Self-supervised learning reveals clinically relevant histomorphological patterns for therapeutic strategies in colon cancer

Self-supervised learning (SSL) automates the extraction and interpretation of histopathology features on unannotated hematoxylin-eosin-stained whole slide images (WSIs). We train an SSL Barlow Twins encoder on 435 colon adenocarcinoma WSIs from The Cancer Genome Atlas to extract features from small image patches (tiles). Leiden community detection groups tiles into histomorphological phenotype clusters (HPCs). HPC reproducibility and predictive ability for overall survival are confirmed in an independent clinical trial ( N  = 1213 WSIs). This unbiased atlas results in 47 HPCs displaying unique and shared clinically significant histomorphological traits, highlighting tissue type, quantity, and architecture, especially in the context of tumor stroma. Through in-depth analyses of these HPCs, including immune landscape and gene set enrichment analyses, and associations to clinical outcomes, we shine light on the factors influencing survival and responses to treatments of standard adjuvant chemotherapy and experimental therapies. Further exploration of HPCs may unveil additional insights and aid decision-making and personalized treatments for colon cancer patients.