Human vs. Machine: A Comparative Analysis of Qualitative Coding by Humans and ChatGPT-4

Generative artificial intelligence (AI) applications are becoming increasingly influential in psychology training, practice, and research. In this study, the procedures (e.g., coding process) and products (e.g., codes; themes; core story) of qualitative content analysis (QCA) conducted by Chat Generative Pre-trained Transformer (ChatGPT)-4 and novice human researchers were compared, and advantages and disadvantages of each approach were considered. Data included open-ended survey responses from trainers (N = 60) in school psychology programs regarding assessment practices during the COVID-19 pandemic. Findings indicated that ChatGPT-4 could conduct QCA with products that were similar, overall, to human coders, and in significantly less time. However, ChatGPT-4’s process was not transparent, and some codes and themes were unclear. Meanwhile, human coding allowed for the selection and implementation of a purposeful methodological approach, and an auditable and systematic process resulting in a coherent narrative. Considerations for the use of AI in qualitative research are considered and discussed, and future research directions are provided.