Engineering of Inquiry: The “Transformation” of Social Science through Generative AI

We increasingly read that generative AI will “transform” the social sciences, but little to no work has conceptualized the conditions necessary to fulfill such a promise. We review recent research on generative AI and evaluate its potential to reshape research practices. As the technology advances, generative AI could support various research tasks, including idea generation, data collection, and analysis. However, we discuss three challenges to an optimistic outlook that focuses solely on accelerating research through practical tools and reducing costs through inexpensive “synthetic” data. First, generative AI raises severe concerns about the validity of conclusions drawn from synthetic data about human populations. Second, possible efficiency gains in the research process may be partially offset by new problems introduced by the technology. Third, applications of generative AI have so far focused on enhancing existing methods, with limited efforts to harness the technology’s unique potential to simulate human behavior in social environments. Sociologists could use sociological theories and methods to develop “generative agents.” A new “trading zone” could emerge where social scientists, statisticians, and computer scientists develop new methodologies to facilitate innovative lines of inquiry and produce scientifically valid conclusions.