Same answer, different reasons: AI-moderated interviews uncover topics and reasoning that surveys miss

Read the full article See related articles

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

Surveys can reach large samples but capture only pre-defined answers, with identical responses often concealing very different reasoning. We tested whether AI-moderated interviews capture this reasoning at scale. In a pre-registered, randomized, counterbalanced design, 1,953 US adults from a probability-based panel and an opt-in panel completed three AI-moderated interviews as well as a section with closed-ended and open-text survey items. Topics were extracted from transcripts inductively, without a predefined codebook, and validated by a confirmatory follow-up. Participants endorsed 94% of topics as fair descriptions of what they said. 21–51% of topics from the AI interview had no counterpart in the closed-ended battery, and 17–37 percentage points net coverage over open text answers. Among respondents who gave identical closed-ended responses, their reasoning diverged widely in valence and content. AI-moderated interviews surface reasons that closed-ended and open-text items miss, and show how reasoning differs among people who answer alike.

Article activity feed