Average accuracy hides where a brain-imaging psychosis risk model fails across clinical high-risk cohorts

Brain-imaging models predicting psychiatric outcomes are rarely tested where they would be used. Clinical high risk for psychosis is a demanding test case because services differ in whom they recruit and how they record transition. We re-estimated a published structural magnetic resonance imaging (MRI) risk model and applied it unchanged to four independent cohorts (685 participants, 106 transitions). Discrimination was weak (area under the curve, 0.56) and near chance in the two cohorts not previously used to evaluate the model (0.53 and 0.56). Predicted risks did not transfer (calibration slope, 0.17). Exploratory performance landscapes, which map local discrimination across participant characteristics, showed that discrimination varied with negative-symptom severity, also in external cohorts, whereas apparent age- and brain-age-related variation depended on single cohorts; no global test survived multiple-testing correction. Reporting where a model works, not only its average accuracy, can reveal when one diagnostic label hides several prediction problems.