Automated Disease Activity Assessment in Systemic Lupus Erythematosus Using Privacy-Preserving Large Language Models

The Systemic Lupus Erythematosus Disease Activity Index 2000 (SLEDAI-2K) is a crucial but labor-intensive tool for managing SLE. We developed a privacy-preserving, model-agnostic large language model (LLM) framework to automate SLEDAI-2K assessment from real-world electronic health records. The framework was developed on a specialist-verified ground truth of 658 clinical notes and externally validated on 56 MIMIC-IV discharge summaries. Seven open-source LLMs were evaluated using advanced prompting and ensemble strategies. The top-performing model, a two-layered GPT-OSS-120B + verifier, achieved a micro-F1 of 94.2% for descriptor classification and an 86% exact match for SLEDAI-2K scores on the internal set, with corresponding external validation performance of 87.7% and 64%, respectively. To demonstrate clinical utility, the LLMs were deployed on 2,576 serial notes from 108 SLE patients. Patients identified by the LLMs as achieving sustained low disease activity had a significantly lower incidence of stage 3 chronic kidney disease (log-rank p = 0.0053), the need for kidney replacement therapy (p = 0.044), and hospitalization (p = 0.021) over 18.3 years of follow-up. These findings demonstrate that privacy-preserving LLMs, when guided by a well-designed framework, can assist in specialist-level reasoning in autoimmune diseases, offering a scalable solution for clinical decision support and patient management.