Towards Measuring the Theory Crisis in Psychology: An LLM-assisted Discourse-analysis Approach

Psychology is theoretically fragmented, yet the field lacks scalable tools for quantifyingwhether theories consolidate or remain contested over time. We introduce a discourse-based approach that tracks theory reception by classifying how published articlesposition themselves relative to a target theory – whether they explicitly support it,oppose it, rely on it tacitly, or engage ambiguously. Using structured literature retrievaland full-text processing, we apply this approach at scale with the aid of a largelanguage model (LLM) used as an annotation support tool, while anchoring allclassifications in expert-defined stance categories. As a case study, we analyze 472articles engaging with memory-decay theory published between 1958 and 2023. LLM-assisted stance classifications show strong agreement with expert judgments usingonly the article abstracts (Cohen’s κ ≈ .80–.83). Across six decades of publications,stance distributions exhibit no systematic increase in tacit acceptance, no sustaineddecline in opposition, and no consistent drift toward consensus, suggesting theoreticalstagnation. Taken together, this work provides a reusable workflow for tracking theoryreception over time using features of scientific discourse. All materials required toreproduce and adapt the pipeline – including code, search strategies, screeningcriteria, and article-level stance labels – are openly available.