A habitat-pathology-clinical transformer model for precision prognostication of neoadjuvant immunochemotherapy in esophageal squamous cell carcinoma

Esophageal squamous cell carcinoma (ESCC) exhibits substantial inter-patient heterogeneity in response to neoadjuvant immunochemotherapy (NICT), underscoring the need for reliable pre-treatment predictors to guide individualized therapeutic decision-making. In this multicenter study, 102 patients with advanced ESCC receiving NICT were enrolled from two independent institutions. Pre-treatment contrast-enhanced CT images, whole-slide histopathology images, and clinical variables were collected. A habitat-based radiomics strategy was applied to characterize intratumoral heterogeneity, while pathomics features were extracted using a weakly supervised multi-instance learning framework. Optimal single-modality models were identified through systematic feature selection and machine-learning screening. A Transformer-based architecture was subsequently developed to integrate radiomic, pathomic, and clinical signatures into a unified multimodal prediction model (HPTrans). Habitat-based radiomics achieved the best performance among imaging models, with the SVM classifier yielding an AUC of 0.723 (95% CI: 0.580–0.866) in the external validation cohort. For pathomics, a ResNet50-based multi-instance learning model combined with logistic regression demonstrated stable predictive capability (AUC = 0.766, 95% CI: 0.639–0.894). HPTrans outperformed all single-modality approaches, achieving AUC of 0.921 (95% CI: 0.889–0.952) in the training cohort and 0.824 (95% CI: 0.720–0.928) in the external validation cohort. Collectively, HPTrans can accurately predict the efficacy of NICT for patients with advanced ESCC, providing a new and reliable predictive reference for the selection of neoadjuvant treatment regimens for these patients.