Human vs. Machine: Comparing AI-generated and Human-written Psychological Reports

This study evaluated the ability of ChatGPT-4, an artificial intelligence (AI) chatbot, to generate psychological reports using mock data. Two hundred forty-nine licensed psychologists in the United States, rated the quality of AI-generated and human-generated psychological reports across six dimensions: overall quality, readability, writing style, organizational structure, summary quality, and recommendation quality. The study used a counterbalanced design with eight conditions, presenting human or AI-generated reports for ADHD, intellectual disability, depression, and anxiety. Participants then provided ratings of quality through an online survey. Human-generated reports were preferred for their writing style (p = 0.026), organization (p = 0.042), and overall quality (p = 0.036), while the recommendations in AI-generated reports were favored (p < 0.001). No significant differences were found in readability. Participants were more likely to prefer and approve human-generated reports for clinical use (p = 0.042), though the majority expressed discomfort with both types. While AI shows promise in augmenting psychological report writing, particularly in generating recommendations, human expertise remains crucial, especially in summarizing information. These findings highlight the need for professional guidelines on AI use in psychological practice and updated educational curricula to prepare future psychologists for an evolving technological landscape in mental health care.