FROM CANCER MOLECULAR SUBTYPE TO AI HYPE: BENCHMARKING AI IN CANCER MOLECULAR SUBTYPING

Background

Cancer molecular subtype classification is an essential component of precision oncology which provides insights into cancer prognosis and guides targeted therapy. Despite the growing applications of AI for cancer molecular subtype classification, challenges persist due to non-standardized dataset configurations, diverse omics modalities, and inconsistent evaluation measures. These issues limit the comparability, reproducibility, and generalizability of AI classifiers across different cancers and hinder the development of robust and accurate AI-driven tools.

Results

This study benchmarks 35 unique AI classifiers across 153 datasets, covering 8 omics modalities and 20 different cancers. Particularly, it investigates 6 different research questions, and based on comprehensive performance analyses of the 35 AI classifiers it elucidates the research questions with the following answers: (i) Out of 17 different configurations for 5/8 omics modalities, RPPA (RPPA), Gistic2-all-data-by-genes (CNV), HM27 (Meth), and HiSeqV2-exon (Exon) configurations consistently yield better performance; (ii) In terms of 8 omics modalities, RNASeq, miRNA, CNV, and Exon generally achieve higher macro-accuracy compared to Meth., Array, SNP and RPPA; (iii) SNP and RPPA modalities are prone to biases due to technical noise and data imbalance; (iv) Traditional machine learning (ML) models (SVM, XGB, HGB) perform best on small and low-dimensional datasets, while deep learning (DL) models (ResNet18, CNN, NN, MLP) excel on large and high-dimensional datasets; (v) SVM achieves the highest mean macro-accuracy across all classifiers, with NN, ResNet18, DEEPGENE, and MLP also demonstrate strong performance; and (vi) DL classifiers show superior macro accuracy as compared to ML classifiers in 12 out of 20 cancers.

Conclusions

The findings offer key insights to guide the development of standardized, robust, and efficient AI-driven pipelines for cancer molecular subtype classification. This study enhances reproducibility and facilitates better comparison across AI methods, ultimately advancing precision oncology.

Key Points

• This study benchmarks 35 unique AI classifiers, ranging from simpler ML models such as Support Vector Machines (SVM), Histogram-Based Gradient Boosting (HGB), and K-Nearest Neighbors (KNN), to complex DL classifiers including Convolutional Neural Networks (CNNs), computer vision models like DenseNet and ResNet, sequential models such as Recurrent Neural Networks (RNN), Gated Recurrent Units (GRU), Long Short-Term Memory networks (LSTM), and their hybrid combinations (e.g., CNN-LSTM, CNN-GRU), as well as transformer-based models, across 153 datasets spanning 8 omics modalities and 20 cancers. It identifies optimal data configurations and evaluates the performance of these classifiers in cancer molecular subtype classification.

• The study highlights biases in specific omics modalities: SNP, RPPA, and Array exhibit higher variability and precision-recall imbalances, while RNASeq, miRNA, Exon, and CNV deliver more consistent and reliable results.

• ML models (e.g., SVM, XGB, HGB) demonstrate strong performance on smaller datasets with fewer features, whereas DL models (e.g., ResNet18, CNN, NN, MLP, and DEEPGENE transformer) excel in handling high-dimensional datasets with large sample sizes.

• The findings provide critical insights for developing robust, standardized AI pipelines for precision oncology, enhancing reproducibility and enabling meaningful cross-method comparisons.