Dissecting the persuasiveness of AI writing in climate change articles: Emotion frames, perceived message qualities, and linguistic traits

Large language models (LLMs) can generate content that rivals, and in some cases even outperforms, the persuasiveness of human writing. While prior research suggests that LLM-generated text persuades through strong reasoning and high perceived credibility, it remains unclear which features ultimately drive this persuasive power, and, in particular, whether it depends on emotion frames – an open question with direct implications for climate change communication. In this pre-registered study, we conduct an online experiment with US-based participants (N = 1,746) to assess the persuasiveness of static climate change articles with the same titles and content, but written by GPT-4, as climate change is a context where emotions have been identified as key drivers of mobilisation. We focus on four emotion frames that capture emotions related to hope, threat, sadness, and anger respectively and find that the AI-written article versions are at least as persuasive as the human-generated article versions. Notably, we find no evidence that article persuasiveness depends on the emotion frames employed, nor that the persuasiveness of AI writing differs between articles using a particular emotion frame. Instead, we find that articles with higher ratings for convincingness and novelty are associated with greater persuasion. We further dissect the persuasiveness of AI writing by conducting an exploratory linguistic analysis of the climate change article sets and find that a key distinction between AI-written and human-written articles is their significantly greater use of words related to psychological drives. While psychological drives was not a statistically significant predictor of climate action, we recommend that this trait receive greater attention in future studies.