Harnessing the Power of Multiple-Instance Learning for Conversion Therapy Outcome Prediction from Pretreatment CT images of Patients with Gastric Cancer

Background: Gastric cancer is a leading cause of cancer-related death worldwide. Conversion therapy for gastric cancer is a treatment strategy that aims to convert unresectable gastric cancer into resectable ones. Early noninvasive evaluation of patients who would benefit from conversion therapy remains a challenge but is essential for personalized treatment in the setting of locally advanced gastric cancer (LAGC). In this study, we aim to develop a reliable deep learning prediction model based on CT images for conversion therapy outcome in patients with LAGC. Methods: Data from LAGC patients, who had CT scans within two weeks prior to conversion therapy, were retrospectively analyzed. We propose a novel approach to predict the conversion therapy outcome using multiple-instance learning (MIL), which is a deep learning framework that can handle data with ambiguous labels, where only bag level labels are given but labels of instances within the bag are unknown. We first used contrastive learning to train a feature extractor, which was then used to extract features from each image. The extracted features were used to train the MIL model that can predict scan-level outcome. We evaluated the performance of the model on a dataset of 124 patients, and compare it with several baseline methods. Results: All 124 patients were recruited from one Chinese hospital between September 2017 and September 2023. The training cohort (TC, n=99) and validation cohort (VC, n=25) were randomly selected, with the data in VC remaining balanced. Performance metrics included accuracy (ACC), area under the curve (AUC), sensitivity and specificity. The results shown that our method achieves higher ACC, AUC and a better balance between sensitivity and specificity than the baseline methods, with an accuracy of 0.88 and an AUC of 0.92 on VC. We also analyzed the features learned by our model using several visualization approaches to verify efficacy. Conclusion: Our method may provide a new perspective and a useful tool for predicting the conversion therapy outcome of LAGC patients using CT images. Moreover, we believe that our method can be applied to various medical image analysis scenarios due to case-based characteristics and common absence of image-level labels.