A multi-scale segmentation-free self-supervised AI model to characterize the heterogeneity of the brain tumor microenvironment

Brain tumors affect about 1 million people in the U.S., with aggressive types like glioblastoma having very low survival rates due to complex tumor biology and the protective blood-brain barrier. Current treatments are limited in effectiveness, and our understanding of brain tumor biology remains incomplete. High dimensional multiplexed imaging has enabled us to better understand the tumor microenvironment (TME); however, analyses typically rely on cell segmentation, which is error-prone, may discard useful context outside the cell boundary, and neglects complex tissue-wide features. To address this limitation, we developed a segmentation-free, self-supervised representation learning framework that enables us to train directly on multiplexed images using masked image modeling. We used this approach to analyze 389 imaging mass cytometry images from 185 brain tumor patients. To study tissue-wide features, we first trained our model on 64×64 micron tiles capturing neighborhoods of 10-20 cells, which we termed local tumor microenvironments (LTMEs). To further characterize these LTMEs, we trained our model on 16×16 micron tiles centered on individual cells in our dataset, so that each tile captures a single cell and its surrounding area, which we termed single-cell microenvironments (SCMEs). This multi-scale, self-supervised approach enables a detailed analysis of the heterogeneity within the brain TME, examining single cells in their spatial context. In addition to validating known findings, we identified a novel LTME in GBM patients, composed primarily of tumor cells and a few B and T cells, which strongly correlated with increased survival. By analyzing these B cells with our SCME model, we found they were distinct from other GBM B cells, and higher concentrations of these B cells were linked to improved survival. In conclusion, our study introduces a multi-scale, segmentation-free, self-supervised machine learning model that provides unprecedented insights into brain TMEs, enabling discovery of previously unrecognized cell interactions and spatial features that are predictive of patient survival.