When AI Meets Psychology: Current Practices, Methodological Pitfalls, and the PREACT Framework
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Integrating artificial intelligence (AI) into psychological research has rapidly gained popularity over the past decade and outpaced systematic evaluation of the research practices it introduces. In this review, we survey 100 psychological studies incorporating AI components between 2015 and 2025, and evaluated their methodological practices. Our analysis reveals that a notable proportion of studies are exploratory, data-driven, closed-source, and weakly grounded in theory, often relying on prediction-oriented evaluation metrics that risk reverse inference. We further identify key methodological challenges that have direct implications for scientific inference, including tensions between falsifiability and flexibility, instability of learned representations, and ambiguity in causal interpretations. To address these concerns, we propose PREACT, a set of guidelines aimed at improving robustness, interpretability, and reproducibility in AI-integrated psychological research.
