Decoding subjective emotional arousal from EEG during an immersive virtual reality experience
Curation statements for this article:-
Curated by eLife
Summary: Hofmann et al. investigate the link between two phenomena, emotional arousal and cortical alpha activity. Although alpha activity is tightly linked to the first reports of electric activity in the brain nearly 100 years ago, a comprehensive characterization of this phenomenon is elusive. One of the reasons is that EEG, the major method to investigate electric activity in the human brain, is susceptible to motion artifacts and, thus, mostly used in laboratory settings. Here, the authors combine EEG with virtual reality (VR) to give experimental participants a roller-coaster ride with high immersion. The ride, literally, leads to large ups and downs in emotional arousal, which is quantified by the subjects during a later rerun. Three different decoding methods were evaluated (Source Power Comodulation, Common Spatial Patterns, and Long Short-Term Memory Recurrent Neural Networks), each of which demonstrated above-chance levels of performance, substantiating a link between lower levels of parietal/occipital alpha and subjective arousal in a quasi-naturalistic setting.
The reviewers both expressed some enthusiasm for the MS:
The study is timely and makes an important contribution to our understanding of the relation of emotions and sensory processing
Of potentially great interest to a broad audience
The embedding in historic literature is excellent. I like it a lot.
This work is notable because the roller-coaster simulation is a well-controlled, yet dynamic manipulation of arousal, and in its comparison of multiple decoding approaches (that can model the dynamics of affective responses). Indeed, this is an interesting proof of concept that shows it is possible to decode affective experience from brain activity measured during immersive virtual reality.
Reviewer #1 opted to reveal their name to the authors in the decision letter after review.
This article has been Reviewed by the following groups
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
- Evaluated articles (eLife)
Abstract
Immersive virtual reality (VR) enables naturalistic neuroscientific studies while maintaining experimental control, but dynamic and interactive stimuli pose methodological challenges. We here probed the link between emotional arousal, a fundamental property of affective experience, and parieto-occipital alpha power under naturalistic stimulation: 37 young healthy adults completed an immersive VR experience, which included rollercoaster rides, while their EEG was recorded. They then continuously rated their subjective emotional arousal while viewing a replay of their experience. The association between emotional arousal and parieto-occipital alpha power was tested and confirmed by (1) decomposing the continuous EEG signal while maximizing the comodulation between alpha power and arousal ratings and by (2) decoding periods of high and low arousal with discriminative common spatial patterns and a long short-term memory recurrent neural network. We successfully combine EEG and a naturalistic immersive VR experience to extend previous findings on the neurophysiology of emotional arousal towards real-world neuroscience.
Article activity feed
-
-
Reviewer #2:
In this work, Hofmann and colleagues conduct a study investigating the relationship between EEG alpha and subjective arousal in naturalistic (as opposed to controlled experimental) settings. Participants completed an immersive virtual reality experience while EEG was recorded, and continuously rated their subjective arousal while a video of the experience was replayed. Three different decoding methods were evaluated (Source Power Comodulation, Common Spatial Patterns, and Long Short-Term Memory Recurrent Neural Networks), each of which demonstrated above chance levels of performance, substantiating a link between lower levels of parietal/occipital alpha and subjective arousal. This work is notable because the roller-coaster simulation is a well-controlled, yet dynamic manipulation of arousal, and in its comparison of …
Reviewer #2:
In this work, Hofmann and colleagues conduct a study investigating the relationship between EEG alpha and subjective arousal in naturalistic (as opposed to controlled experimental) settings. Participants completed an immersive virtual reality experience while EEG was recorded, and continuously rated their subjective arousal while a video of the experience was replayed. Three different decoding methods were evaluated (Source Power Comodulation, Common Spatial Patterns, and Long Short-Term Memory Recurrent Neural Networks), each of which demonstrated above chance levels of performance, substantiating a link between lower levels of parietal/occipital alpha and subjective arousal. This work is notable because the roller-coaster simulation is a well-controlled, yet dynamic manipulation of arousal, and in its comparison of multiple decoding approaches (that can model the dynamics of affective responses). Indeed, this is an interesting proof of concept that shows it is possible to decode affective experience from brain activity measured during immersive virtual reality.
Major concerns:
The authors advocate that naturalistic experiments are needed to study emotional arousal, because "static" manipulations are not well-suited to capture the continuity and dynamics of arousal. This point is well-taken, but no comparisons were made between static and dynamic methods. Thus, although the work succeeds in showing it is possible to use machine learning to decode the subjective experience of arousal during virtual reality, it is not clear what new insights naturalistic manipulations and the machine learning approaches employed have to offer.
The methods used to assess model performance are also a concern. Decoding models were evaluated separately for each subject using 10-fold cross-validation, and inference on performance was made using group-level statistics. Because time-series data are being decoded, if standard cross-validation was performed the results could be overly optimistic. Additionally, hyperparameters were selected to maximize model performance which can also lead to biased estimates. This is particularly problematic because overall decoding performance is not very high.
-
Reviewer #1:
Hofmann et al. investigate the link between two phenomena, emotional arousal and oscillatory alpha activity in the cerebral cortex, which is of central interest in their respective fields. Although alpha activity is tightly linked to the first reports of electric activity in the brain nearly 100 years ago, a comprehensive characterization of this phenomenon is elusive. One of the reasons is that EEG, the major method to investigate electric activity in the human brain, is susceptible to motion artifacts and, thus, mostly used in laboratory settings. Here, the authors combine EEG with a virtual reality setup to give experimental participants a roller-coaster ride with high immersion. The ride, literally, leads to large ups and downs in emotional arousal, which is quantified by the subjects during a later rerun. Next, the …
Reviewer #1:
Hofmann et al. investigate the link between two phenomena, emotional arousal and oscillatory alpha activity in the cerebral cortex, which is of central interest in their respective fields. Although alpha activity is tightly linked to the first reports of electric activity in the brain nearly 100 years ago, a comprehensive characterization of this phenomenon is elusive. One of the reasons is that EEG, the major method to investigate electric activity in the human brain, is susceptible to motion artifacts and, thus, mostly used in laboratory settings. Here, the authors combine EEG with a virtual reality setup to give experimental participants a roller-coaster ride with high immersion. The ride, literally, leads to large ups and downs in emotional arousal, which is quantified by the subjects during a later rerun. Next, the authors decode the degree of emotional arousal as stated in the rerun based on the EEG signals recorded during the VR session. They demonstrate convincingly a negative dependence of alpha activity with the degree of emotional arousal. Further, they demonstrate the differential involvement of parietal and occipital regions in this process. The sequencing of the description of methods and results could be improved upon, is, however, as such perfectly ok. This investigation comes timely, makes an important contribution to our understanding of the relation of emotions and sensory processing.
-
Summary: Hofmann et al. investigate the link between two phenomena, emotional arousal and cortical alpha activity. Although alpha activity is tightly linked to the first reports of electric activity in the brain nearly 100 years ago, a comprehensive characterization of this phenomenon is elusive. One of the reasons is that EEG, the major method to investigate electric activity in the human brain, is susceptible to motion artifacts and, thus, mostly used in laboratory settings. Here, the authors combine EEG with virtual reality (VR) to give experimental participants a roller-coaster ride with high immersion. The ride, literally, leads to large ups and downs in emotional arousal, which is quantified by the subjects during a later rerun. Three different decoding methods were evaluated (Source Power Comodulation, Common Spatial Patterns, …
Summary: Hofmann et al. investigate the link between two phenomena, emotional arousal and cortical alpha activity. Although alpha activity is tightly linked to the first reports of electric activity in the brain nearly 100 years ago, a comprehensive characterization of this phenomenon is elusive. One of the reasons is that EEG, the major method to investigate electric activity in the human brain, is susceptible to motion artifacts and, thus, mostly used in laboratory settings. Here, the authors combine EEG with virtual reality (VR) to give experimental participants a roller-coaster ride with high immersion. The ride, literally, leads to large ups and downs in emotional arousal, which is quantified by the subjects during a later rerun. Three different decoding methods were evaluated (Source Power Comodulation, Common Spatial Patterns, and Long Short-Term Memory Recurrent Neural Networks), each of which demonstrated above-chance levels of performance, substantiating a link between lower levels of parietal/occipital alpha and subjective arousal in a quasi-naturalistic setting.
The reviewers both expressed some enthusiasm for the MS:
The study is timely and makes an important contribution to our understanding of the relation of emotions and sensory processing
Of potentially great interest to a broad audience
The embedding in historic literature is excellent. I like it a lot.
This work is notable because the roller-coaster simulation is a well-controlled, yet dynamic manipulation of arousal, and in its comparison of multiple decoding approaches (that can model the dynamics of affective responses). Indeed, this is an interesting proof of concept that shows it is possible to decode affective experience from brain activity measured during immersive virtual reality.
Reviewer #1 opted to reveal their name to the authors in the decision letter after review.
-
