The News with ChatGPT: An Audit and Survey Experiment on the Effects of GPT-Enabled News Search on User Attitudes

Policymakers and researchers are increasingly concerned with how algorithmic systems shape public access to political information. AI-powered chatbots, such as OpenAI’s GPT-4o, are being adopted as search tools, yet their impacts on information quality and user attitudes remain unclear. This study evaluates the information retrieval performance and cognitive consequences of GPT-4o News compared to Google News. In the first part, we audited 592,008 news articles retrieved by GPT-4o News and Google News in response to over 80,000 political queries across eleven countries. Our results show that GPT-4o News retrieved significantly less relevant and more controversial content than Google News, despite slightly lower partisanship scores. In the second part, we conducted a bilingual survey experiment (N = 921) in the U.S. and India, randomly assigning participants to one of four retrieval formats. We found that participants exposed to GPT-4o reported higher knowledge gains but also demonstrated lower misinformation discernment, particularly when interacting with a conversational chatbot. These effects were moderated by gender, with non-male participants gaining more knowledge but showing greater declines in discernment. Our findings highlight trade-offs between perceived learning and evaluative accuracy when relying on AI-driven information systems, raising important implications for the design of trustworthy political information retrieval tools.