Conversing with a disagreeing LLM improves people’s inaccurate predictions

Accurately predicting the outcome of future events improves decisions in domains ranging from health to finance, yet prediction errors are common. One reliable way to improve accuracy is to engage with disagreeing views through dialogue, which can reveal weak assumptions and facilitate belief revision. However, people often avoid disagreeing exchanges because they are socially and emotionally costly or inconvenient to seek out. Here, we examine whether a disagreeing large language model (LLM) can provide the benefits of disagreement while sidestepping these interpersonal barriers. Across two experiments (total N = 807 adults making 4,564 predictions; Experiment 2 was incentivized and preregistered), participants: i) predicted the likelihood of various events, ii) discussed each prediction with an LLM by articulating their reasoning, and iii) submitted final predictions. Crucially, we manipulated the LLM’s stance during these discussions: it responded either supportively (agreeing) or critically (disagreeing) to participants’ reasoning. The results showed that, when initial predictions were erroneous, conversing with a disagreeing LLM produced more accurate final predictions than conversing with a supportive one. This advantage was primarily driven by directionally correct revisions, with participants shifting their final predictions more often toward the realized outcome. However, interacting with an LLM was not uniformly beneficial as it decreased accuracy when initial predictions were already accurate. These findings underscore the value of designing LLMs that challenge rather than affirm users’ views and opinions, while highlighting the importance of tailoring their responses to the expected accuracy of initial predictions.