Large Language Model–Enhanced Electronic Frailty Assessment and Prognostic Performance in Hospitalized Adults: A Retrospective Cohort Study
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Electronic frailty indices (eFIs) derived from structured electronic health record (EHR) data may incompletely capture functional deficits. Large language models (LLMs) can extract frailty-related information from clinical notes, but whether improved frailty ascertainment improves prognostic performance is uncertain.
Objective
To compare frailty ascertainment and prognostic performance between standard and LLM-enhanced eFIs among hospitalized adults.
Design
Retrospective cohort study
Participants
Adults aged ≥50 years enrolled in two studies at an academic medical center and affiliated community hospital between February 2022 and April 2024.
Main Measures
A validated 35-deficit diagnosis-based eFI and an LLM-enhanced eFI incorporating deficits extracted from physical therapy (PT) documentation using GPT-4o. Frailty was defined using an index threshold of ≥0.20. The primary outcome was a composite of discharge to a facility, readmission, or death within 90 days. Multivariable logistic regression models assessed associations with the outcome, and discrimination was compared using area under the receiver operating characteristic curve (AUROC). Secondary analyses examined threshold-based classification and PT-documentation availability; prespecified sensitivity analyses evaluated population and outcome restrictions.
Results
Among 616 hospitalizations, mean (SD) age was 70.9 (10.2) years, 329 (53.4%) had PT documentation, and 320 (51.9%) experienced the composite outcome. Standard and LLM-enhanced eFIs classified 216 (35.1%) and 362 (58.8%) hospitalizations as frail, respectively. Both were independently associated with the outcome (standard eFI: adjusted OR, 1.98 [95% CI, 1.39–2.83]; LLM-enhanced eFI: adjusted OR, 2.46 [95% CI, 1.75–3.47]). Discrimination did not differ significantly (AUROC, 0.636 vs 0.660; difference, 0.024 [95% CI, −0.006 to 0.053]; p =0.12). LLM enhancement increased sensitivity (42.5% to 69.4%) but decreased specificity (73.0% to 52.7%).
Conclusions
LLM enhancement substantially increased frailty ascertainment and sensitivity for adverse outcomes but did not significantly improve prognostic discrimination. Thus, LLM-enhanced frailty assessment may depend on whether the intended use prioritizes broad identification of functional vulnerability or prediction of adverse post-hospital outcomes.
