An explainable multimodal framework for exploring depression-associated speech representations in Alzheimer’s disease
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Depression in Alzheimer’s disease (AD) is clinically challenging to characterise because affective and cognitive manifestations frequently overlap with the underlying neurodegenerative condition. This study investigates depression-related speech patterns in AD using an explainable multimodal framework that integrates linguistic and acoustic features from elicited picture-description speech. To address the absence of directly observed depression labels in the target AD cohort, we employ a source-informed cross-dataset transfer strategy. Unsupervised clustering identified two distinct speech sub-phenotypes within the AD cohort. By comparing these unsupervised representations to a depression-labelled source corpus, one cluster demonstrated mathematical proximity (measured via cosine similarity in a shared latent space) to the source depression centroid. We explicitly acknowledge that this proximity reflects geometric similarity in a shared latent space, which may encapsulate overlapping domain characteristics alongside affective variance. To validate the structural integrity and separability of these discovered clusters, a proxy supervised classification task was employed. The retained multimodal artificial neural network (ANN) integrated information from both modalities and achieved 92.73% cluster separability accuracy. This demonstrates that the representation-derived two-cluster partition was highly learnable on the independent internal hold-out test set; it does not establish clinical validity, longitudinal stability, or diagnostic validity of the identified sub-phenotypes. For interpretability, Shapley Additive exPlanations (SHAP) analysis was utilised to extract feature attributions and organise influential acoustic and linguistic features into a multimodal taxonomy. Furthermore, a Multimodal SHAP Attribution Magnitude (MSAM) score was derived as an exploratory model-based attribution score rather than a clinically validated severity measure. Ultimately, this study provides an interpretable computational baseline for characterising depression-related speech variance in AD, establishing a structural foundation for future prospective validation using independently labelled clinical cohorts.
Author Summary
Depression can be difficult to recognise in people with Alzheimer’s disease because changes in mood, motivation, language, and behaviour may overlap with symptoms of cognitive impairment. Speech analysis offers a possible way of studying these changes, but datasets containing both Alzheimer’s disease and independently verified depression labels are limited. We therefore explored whether speech representations from an Alzheimer’s disease cohort could be compared with a separate depression-labelled speech dataset. We used acoustic and linguistic features to identify groups of participants with similar speech characteristics and then examined whether these groups could be reliably reproduced by supervised machine-learning models. The best multimodal model reproduced the representation-derived grouping with 92.73% accuracy on an independent participant-level test set. Importantly, this result does not demonstrate that the participants had depression, because depression was not independently assessed in the Alzheimer’s disease cohort. Instead, the study provides an exploratory computational framework for studying cross-corpus speech similarities and identifies features that contributed to the learned speech representations. Prospective studies with clinically assessed depression status are needed before such methods can be considered for clinical use. Such studies should also examine whether the observed speech patterns remain informative after accounting for cognitive impairment, age, medication, speaking task, and recording conditions. This work therefore represents a hypothesis-generating computational step rather than a diagnostic tool.
