Diagnostic accuracy of a DenseNet-121 deep learning algorithm for chest radiograph triage in health assessment applicants: a prospective shadow-mode validation study in Nepal

Objectives

To evaluate the diagnostic accuracy of a publicly available DenseNet-121 convolutional neural network (TorchXRayVision) for triaging chest radiographs of health assessment applicants at a tertiary hospital in Nepal.

Design

Prospective, single-centre, shadow-mode diagnostic accuracy validation study. Reported in accordance with the STARD 2015 checklist and STARD-AI/DECIDE-AI guidelines.

Setting

Department of Radiology and Imaging, Patan Academy of Health Sciences / Patan Hospital, Lalitpur, Nepal.

Participants

826 consecutive health assessment applicants (foreign employment Pre-Departure Medical Examination and student migration) undergoing chest radiography between 5 June and 20 June 2026. Two cases were excluded due to DICOM technical failure.

Index test

DenseNet-121 algorithm (TorchXRayVision library, densenet121-res224-all pretrained weights). A maximum aggregated pathology probability score was derived per radiograph and compared against a post-hoc derived threshold of 0.6258 (selected as the highest threshold achieving the pre-specified ≥95% sensitivity criterion).

Reference standard

Single-reader-per-case review by one of three radiologists — two board-certified radiodiagnosticians (LS: 276 cases; DM: 275 cases) and one radiology resident (UB: 275 cases) — each blinded to AI output, using a standardised data collection worksheet capturing binary classification (abnormal/normal) and free-text findings.

Results

Of 826 radiographs, 41 (4.97%) were classified as abnormal by the reference standard. At the post-hoc derived threshold of 0.6258, the DenseNet-121 algorithm achieved: sensitivity 95.12% (95% CI 83.9–98.7%), specificity 77.2% (95% CI 74.1–80.0%), area under the receiver operating characteristic curve (AUROC) 0.9583 (95% bootstrap CI 0.9225–0.9843), NPV 99.67% (95% Wilson CI 98.8–99.9%), PPV 17.89% (95% Wilson CI 13.4–23.5%), and Cohen’s κ 0.237 (95% bootstrap CI 0.174–0.304). Brier score was 0.3621 (null Brier 0.0472) and ECE was 0.564, confirming calibration failure due to score compression (range 0.52–0.72) despite preserved discrimination.

Cross-validated results

Ten-fold cross-validation yielded bias-corrected sensitivity 95.12% (95% Wilson CI 83.9–98.7%; optimism 0.00 pp) and specificity 75.80% (95% Wilson CI 72.7–78.7%; optimism +1.40 pp), confirming primary metrics are not materially inflated by circular optimisation.

Conclusions

The DenseNet-121 algorithm demonstrated high sensitivity and excellent discrimination for chest radiograph triage in a Nepali health-assessment population, supporting its potential as a rule-out tool (NPV 99.67%). Systematic score compression—preserved discrimination despite calibration shift—is a quantifiable marker of LMIC distributional shift. Prospective local calibration studies are warranted before operational deployment.

Ethics approval

Institutional Review Committee of Patan Academy of Health Sciences (Protocol No. drs2606052243; expedited review, 5 June 2026). Individual written informed consent was waived by the IRC.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Deep learning algorithms trained on large chest radiograph datasets achieve radiologist-level performance on well-defined pathology classification tasks in high-income country settings.

• Most prospective validation studies have been conducted in high-income countries using datasets demographically and technically similar to the training data; evidence from South Asian LMIC settings is scarce.

• Distributional shift—where model performance degrades on out-of-distribution populations—is a recognised but incompletely characterised challenge for AI deployment in low- and middle-income countries

WHAT THIS STUDY ADDS

• A publicly available DenseNet-121 algorithm (TorchXRayVision) achieved sensitivity >95% and AUROC >0.95 for chest radiograph triage in a Nepali health-assessment population, supporting clinical utility as a rule-out tool with NPV 99.67%.

• Systematic score compression (range 0.52–0.72; SD ∼0.026) is identified and quantified as a marker of LMIC distributional shift; critically, this compression did not compromise discrimination (AUROC 0.9583).

• Shadow-mode prospective validation of an open-source AI triage tool is shown to be feasible in a resource-limited LMIC radiology department using fully offline, opensource Python infrastructure.