Prompt engineering meets ‘definition of the situation’ and identity theory: Using ChatGPT to study social media datasets from a qualitative symbolic interactionist perspective

This research note examines the potential of ChatGPT’s large language model for qualitative symbolic interactionist analysis of social media datasets. Using a small sample of 15 posts on mental health issues from the social media platform Reddit, we explore how large language models interpret, organize, and communicate information based on researchers’ prompts. Prompting itself is a process in which researchers provide both a definition of the situation and of relevant identities for a GPT to assume, which directly shape its interpretive capacity and output. We compare ChatGPT-4o, the latest personal subscription version available at the time of writing, and Microsoft AutoGen, a more powerful API version capable of organizing multiple GPT agents to complete research tasks semi-autonomously. We describe and evaluate the structure and content of each model’s output, including how they engage with data thematically, empirically, and conceptually. We also highlight differences in depth and coherence of their analyses. We show how changes to prompts influenced the GPTs’ interpretative actions, yielding both expected and novel analytical insights. Our preliminary findings suggest that GPTs hold promise for advancing qualitative analysis of big data, albeit with some limitations in analytic depth and occasional hallucination errors. The study contributes to nascent symbolic interactionist scholarship on nonhuman agents in qualitative research, positioning GPTs as semi-autonomous research collaborators.