Large-scale electrophysiology and deep learning reveal distorted neural signal dynamics after hearing loss
Curation statements for this article:-
Curated by eLife
eLife assessment
This fundamental work uses deep neural networks to simulate activity evoked by a wide range of stimuli and demonstrates systematic differences in latent population representations between hearing-impaired and normal-hearing animals that are consistent with impaired representations of speech in noise. While the evidence supporting the conclusions is compelling, additional analyses would facilitate the generalizability of the neural-network approach. The research will be of interest to auditory neuroscientists and computational scientists.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
Listeners with hearing loss often struggle to understand speech in noise, even with a hearing aid. To better understand the auditory processing deficits that underlie this problem, we made large-scale brain recordings from gerbils, a common animal model for human hearing, while presenting a large database of speech and noise sounds. We first used manifold learning to identify the neural subspace in which speech is encoded and found that it is low-dimensional and that the dynamics within it are profoundly distorted by hearing loss. We then trained a deep neural network (DNN) to replicate the neural coding of speech with and without hearing loss and analyzed the underlying network dynamics. We found that hearing loss primarily impacts spectral processing, creating nonlinear distortions in cross-frequency interactions that result in a hypersensitivity to background noise that persists even after amplification with a hearing aid. Our results identify a new focus for efforts to design improved hearing aids and demonstrate the power of DNNs as a tool for the study of central brain structures.
Article activity feed
-
-
eLife assessment
This fundamental work uses deep neural networks to simulate activity evoked by a wide range of stimuli and demonstrates systematic differences in latent population representations between hearing-impaired and normal-hearing animals that are consistent with impaired representations of speech in noise. While the evidence supporting the conclusions is compelling, additional analyses would facilitate the generalizability of the neural-network approach. The research will be of interest to auditory neuroscientists and computational scientists.
-
Reviewer #1 (Public Review):
This paper investigates the neural correlates of noise-induced hearing loss. The authors use an electrode array to capture neural responses across the inferior colliculus to speech and synthetic sounds in both normal-hearing gerbils, and gerbils with noise-induced hearing loss. They use dimensionality reduction to isolate a low-dimensional response subspace that captures most of the information about the speech signals, and find that this low-dimensional representation is altered considerably by hearing loss (evaluated with CCA). To probe the basis of these differences, the authors train an artificial neural network to predict the subspace responses to arbitrary stimuli, for instance to investigate the consequences of frequency-dependent amplification of sound with a hearing aid, or synthetic test stimuli. …
Reviewer #1 (Public Review):
This paper investigates the neural correlates of noise-induced hearing loss. The authors use an electrode array to capture neural responses across the inferior colliculus to speech and synthetic sounds in both normal-hearing gerbils, and gerbils with noise-induced hearing loss. They use dimensionality reduction to isolate a low-dimensional response subspace that captures most of the information about the speech signals, and find that this low-dimensional representation is altered considerably by hearing loss (evaluated with CCA). To probe the basis of these differences, the authors train an artificial neural network to predict the subspace responses to arbitrary stimuli, for instance to investigate the consequences of frequency-dependent amplification of sound with a hearing aid, or synthetic test stimuli. Using this approach, they find that the representation of sounds in quiet is largely restored by a hearing aid algorithm that amplifies high frequencies to render them audible. However, the representation of sounds in noise also differs between the IC of normal-hearing and hearing-impaired gerbils, and this difference is not eliminated by a hearing aid. Specifically, low-frequency maskers seem to distort the representation of high-frequency sounds (e.g. consonants in speech), even once the high-frequencies have been amplified to compensate for the hearing loss.
Overall, this is a strong paper. The topic is important, the methods are innovative, logical, and rigorous, and the whole thing is exceptionally clearly described. I greatly appreciate the care that clearly went into writing the paper. I have two major concerns. The first seems fairly critical to the paper's conclusions, but I hope can be addressed with some kind of control experiment. The second could potentially be thought of as more of a future direction, but it speaks to the specificity of the conclusions.
1. My main substantive concern is that the conclusions depend critically on believing the predictions of the DNN, and yet it is not clear we should expect it to generalize well to stimuli outside its training distribution. Current artificial neural networks typically work very well for stimuli like those they were trained on, but often do not generalize as well as one might like. The authors recorded responses to speech in quiet and in different noise levels, and show that the trained DNN (trained on these sounds and the associated responses) produces very accurate predictions on held-out sounds from this distribution. But the conclusions depend critically on the DNN predictions for sound processed by a hearing aid, and for synthetic sounds (pure tones, SAM noises) that are quite unlike the training data. The predictions look reasonable in places where we have some prior sense for what to expect (level-dependent frequency tuning to pure tones), which is reassuring, but I am not sure how to be confident that the predictions should be accurate for all of the conditions that are tested, in particular to the results with the simulated hearing aid. I am pretty sure that the predictions will be inaccurate for some types of stimuli (just based on the various pathologies that are known to occur with neural networks). I would hope that this would not be the case for the conditions tested by the authors, but it is hard to be sure, and this makes the conclusions seem a little more vulnerable than I would like.
How do we know that the DNN generalizes beyond its training data well enough to render the conclusions airtight?2. My second concern is the extent to which the results are specific to a) the IC, and b) noise. The authors assert that similar effects would not be present in the nerve, citing a Heinz paper, but I am not sure how clear this evidence is - it is not described in enough detail here to assess. It would be nice to show this, perhaps by repeating their analysis on a model of the nerve with and without simulated hearing loss. One can similarly wonder about the effects in the cortex, especially given the literature on noise invariance (Rabinowitz, Moore, Khalighinejad, Kell...), which would at least be worth discussing. It is similarly unclear whether the results are specific to additive noise. Would similar conclusions hold for any type of distortion? This could be easily addressed by an additional DNN analysis (e.g. with clipping, or segments of speech intermittently replaced by silence, or reverberation).
-
Reviewer #2 (Public Review):
This very interesting study uses a combination of high channel count neural recordings and machine learning to characterize neural representations of complex natural and synthetic sounds in the inferior colliculus. The authors use deep neural networks to model sound evoked activity in a large number of IC multiunits with high accuracy in gerbils with normal hearing and hearing loss. They then use the DNNs to simulate activity evoked by a wide range of stimuli and demonstrate systematic differences in latent population representations between normal hearing and hearing-impaired animals. Models for hearing impaired animals show activity consistent with impaired representations of speech in noise. These results lay the groundwork for a potentially valuable approach to improving signal processing in hearing aids …
Reviewer #2 (Public Review):
This very interesting study uses a combination of high channel count neural recordings and machine learning to characterize neural representations of complex natural and synthetic sounds in the inferior colliculus. The authors use deep neural networks to model sound evoked activity in a large number of IC multiunits with high accuracy in gerbils with normal hearing and hearing loss. They then use the DNNs to simulate activity evoked by a wide range of stimuli and demonstrate systematic differences in latent population representations between normal hearing and hearing-impaired animals. Models for hearing impaired animals show activity consistent with impaired representations of speech in noise. These results lay the groundwork for a potentially valuable approach to improving signal processing in hearing aids and prosthetics.
The large speech dataset and clean hearing loss effects are particularly impressive. While the approach and associated data are novel and likely to be of broad interest, there are some substantial concerns about the study. First, the authors fail to acknowledge substantial previous work on super-threshold activity in cortex of animals with hearing loss, making it appear that they overstate the novelty of the current results. There are also many cases where they fail to clearly report the details of statistics used to support their claims. Finally, while the accuracy of the DNN models is compelling for the speech stimuli in the data set, it is not clear that the comparisons of simulated activity reflect actual neural activity in the stimulus conditions tested.
-
-