Comparative Study of Anatomical and Learned Features in AI Models for Structural Brain MRI

In this work, we comprehensively evaluate three popular feature-extractionparadigms in AI-based neuroimaging modeling: (1) computation of anatomicalsurfaces and volumes, (2) supervised learning with convolutional neural net-works (CNNs), and (3) unsupervised pretraining of vision transformer (ViT)foundation models, followed by supervised finetuning. Our study is based on18 publicly available datasets containing 3D structural T1-weighted MRI scansfrom approximately 80,000 participants across seven distinct clinical tasks. Weobserve that a linear model based on anatomical features matches the diag-nostic performance of complex nonlinear features learned by sophisticated AIframeworks, including foundation models trained on thousands of scans. Con-versely, CNNs and pretrained ViTs learn features that implicitly capture relevantanatomical information, bypassing the need for explicit feature extraction. Build-ing upon these insights, we propose Anatomy Segmentation Pretraining (ASP),a novel method to incorporate anatomical information during foundation-modelpretraining, which outperforms existing models in biological age estimation.