Large language models’ interpretation homogeneity and text Analysis: Evaluating the utility of the global flu view platform for Influenza surveillance
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
The advent of Large Language Models (LLMs) has transformed natural language processing and offers new possibilities for analyzing qualitative data in public health research. This study evaluated the utility of Global Flu View (GFV), a participatory surveillance platform for influenza-like illness, using multiple LLMs to analyze stakeholder perceptions. We conducted in-depth interviews with 10 participants comprising GFV partners, advisory group members, and public health researchers across six countries. Interview transcripts were analyzed using four LLMs (ChatGPT, Claude AI, Perplexity, and Gemini) to perform sentiment analysis, with scores ranging from -1 (negative) to +1 (positive). Word pair networks were generated using the Louvain clustering method to identify thematic patterns in stakeholder responses. The analysis revealed consistently positive sentiments toward GFV across all stakeholder groups, with sentiment scores ranging from 0.4 to 0.8. The Friedman test showed that ChatGPT and Gemini produced higher sentiment scores (average rank = 3.5 each) compared to Claude AI (average rank ≈ 1.33) and Perplexity (average rank ≈ 1.67). Word pair networks demonstrated that participants conceptualize GFV as a data-centric tool integrated within broader public health systems, with strong connections between terms related to surveillance, global collaboration, and public health decision-making. While stakeholders expressed optimism about GFV’s potential to enhance global influenza surveillance, they also identified areas for improvement, particularly regarding data sharing and regional implementation. This study represents the first evaluation of GFV using LLMs for sentiment analysis and demonstrates the potential of AI-assisted qualitative analysis in public health research. The findings suggest that GFV is perceived as a valuable tool for global health surveillance, while also highlighting opportunities for platform enhancement and methodological considerations for future AI-assisted qualitative research in public health.
Author summary
In our study, we explored how public health experts view and value a new global health surveillance platform called Global Flu View, which collects data about flu-like illnesses from multiple countries across four continents. We were particularly interested in understanding whether this platform is helping to track and monitor flu-like illnesses worldwide. To do this, we interviewed 10 experts from different countries and used artificial intelligence tools to analyze their responses. This approach allowed us to systematically evaluate their perceptions and sentiment towards the platform. We found that experts generally view the platform positively and see it as a valuable tool for global health monitoring. However, they also pointed out areas where it could be improved, such as making data sharing between countries easier. Our work is important because it helps us understand how new digital tools can support global health surveillance, and it demonstrates how artificial intelligence can help researchers analyze interview data. The insights we gained can help make these kinds of health monitoring platforms more effective and useful for tracking disease outbreaks across the world.
