Development of a Core Outcome Set for Neurological Disorders (COS-Neuro): an AI-Assisted Thematic Framework Analysis
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Neurological disorders affect approximately 3 billion people globally, yet clinical trial success is often hindered by poorly selected outcome measures, impacting trial design, compliance, and interpretation. Over the past 25 years, Core Outcome Sets (COS) have emerged as standardized tools to enhance outcome selection, ensuring comparability across studies and reflecting the priorities of both researchers and patients. Despite the success of COS initiatives in other fields, their development in neurology remains limited, leaving many trialists without disease-specific guidance.
Objectives
This study aimed to develop a COS framework for neurological disorders with the assistance of artificial intelligence (AI) by analysing the frequency and scope of outcomes previously reported in existing COS to identify common themes applicable to neurological research.
Methods
COS-Neuro was developed using AI-assisted thematic framework analysis, complemented by expert review. A modified five-step thematic analysis was conducted without pre-determined codes:
Dataset Gathering – Data was collected from the COMET database, and COS domains for neurological disorders were coded.
Prompt Design & Testing – Large language models (LLMs), including ChatGPT 3.5, Google Gemini 1.5 Flash and Meta Llama-2-70b, were trialled, and prompts refined based on their outputs.
Thematic Analysis – LLMs categorised domains into core areas.
Human Refinement – Experts reviewed LLM-generated core areas and selected those most appropriate for further interpretation.
Clinical Validation – Experts validated the domains, core areas, and concepts.
This approach integrated AI with expert oversight to develop a standardised COS framework for neurological disorders.
Results
Utilising LLMs, particularly ChatGPT, a robust conceptual framework for COS in neurological disorders was developed, based on the existing 112 existing COS. Through adaptation of the OMERACT model, the final framework comprised four concepts, 13 core areas, and 75 domains, as determined by expert consensus.
Conclusion
COS-Neuro establishes AI-assisted framework for developing COS in neurological disorders. This project provides a foundational resource for future COS research and serves a reference for designing trials in areas where established COS are lacking. Furthermore, it sets a precedent for the integration of AI in qualitative analysis in medicine, demonstrating the scalability of approaches like OMERACT for the development of ‘COS of COS’ across various specialties.
