Predicting conversion from mild cognitive impairment to Alzheimer’s disease using a Vision Transformer and hippocampal MRI slices

Convolutional neural networks (CNNs) have been the standard for computer vision tasks and are frequently applied in medical conditions, such as in Alzheimer’s disease (AD). Recently, Vision Transformers (ViTs) have been introduced, which provide a strong alternative to CNNs by discarding the convolution approach in favor of the attention mechanism. This allows modeling global and distant relationships within distinct parts of an image without relying on the strong inductive biases present in CNNs. A common precursor stage of AD is a syndrome called mild cognitive impairment (MCI). However, not all individuals diagnosed with MCI progress to AD. The establishment of reliable classification models that predict converters versus non-converters would be a valuable tool to support clinical decision-making, such as enabling early treatment. Hence, in this investigation a transfer learning approach was used by applying a pretrained ViT model, fine-tuned on the ADNI dataset comprising 575 subjects with MCI. We included baseline T1-weighted structural MRI data from 299 stable MCI and 276 progressive MCI individuals, who developed Alzheimer’s disease within 36 months. Inputs to the model were three normalized axial slices covering areas of the hippocampal region, consisting of the combined gray and white matter segmentations. The final model was evaluated over multiple runs to obtain stable performance estimates, yielding an average area under the receiver operating characteristic curve (AUC-ROC) on the test set of 0.74 ± 0.02 (mean ± SD), an accuracy of 0.69 ± 0.03, a sensitivity of 0.65 ± 0.07, a specificity of 0.72 ± 0.06, and a F1-score for the pMCI class of 0.67 ± 0.04. By specifically focusing on axial slices covering the hippocampal region, we aimed to target the brain structure often reported as being the first affected by the disease, while our results indicate that a ViT approach achieves reasonable classification accuracy for predicting the conversion from MCI to AD.