Beyond Likert Scales: Convergent Validity of an NLP-Based Future Self-Continuity Assessment from Verbal Data

Recent advances in Large Language Models (LLMs) offer new assessment approaches that can help overcome the limitations of traditional Likert-item scales in measuring complex, subjective constructs. To demonstrate this, we introduce and validate a novel LLM-based methodology for psychological assessment by applying it to Future Self-Continuity (FSC), the perceived connection, including similarity, vividness, and positivity, between present and future selves. We used an LLM (Claude 3.5 Sonnet) to perform natural language processing (NLP) on transcripts of audio responses to 15 theory-based interview prompts. Data from 164 MTurk participants (including 93 with past-year suicide ideation, who were oversampled to examine clinical utility) yielded quantitative NLP-FSC scores that significantly correlated with the Future Self-Continuity Questionnaire (FSCQ; r = .57), supporting convergent validity. A Bland-Altman analysis also indicated acceptable agreement. Replication using one older and two updated LLM versions confirmed the method’s robustness (inter-model total score r = .91, .88, and .84). Exploratory analysis using the Suicidal Behaviors Questionnaire–Revised (SBQR) found that the NLP assessment captured unique variance in the perceived likelihood of a future suicide attempt beyond the FSCQ, suggesting potential clinical implications. This validated NLP approach offers a nuanced assessment of FSC, advancing psychological measurement methodology in research and, potentially, clinical practice.