Neural signatures of vigilance decrements predict behavioural errors before they occur

  1. Hamid Karimi-Rouzbahani
  2. Alexandra Woolgar
  3. Anina N. Rich

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

There are many monitoring environments, such as railway control, in which lapses of attention can have tragic consequences. Problematically, sustained monitoring for rare targets is difficult, with more misses and longer reaction times over time. What changes in the brain underpin these “vigilance decrements”? We designed a multiple-object monitoring (MOM) paradigm to examine how the neural representation of information varied with target frequency and time performing the task. Behavioural performance decreased over time for the rare target (monitoring) condition, but not for a frequent target (active) condition. This was mirrored in the neural results: there was weaker coding of critical information during monitoring versus active conditions. We developed new analyses that can predict behavioural errors from the neural data more than a second before they occurred. This paves the way for pre-empting behavioural errors due to lapses in attention and provides new insight into the neural correlates of vigilance decrements.

Article activity feed

  1. eLife

    ###Reviewer #3:

    I found the question, approach and analysis provide a clever framework for understanding how vigilance changes over time. I believe this work will contribute greatly to the literature. However, I have one main concern in the interpretation of the patterns of results and the a priori assumptions that are made, but never explicitly discussed or justified.

    The introduction makes it clear that the authors acknowledge that there may be multiple sources of interference contributing to declining vigilance over time: the encoding of sensory information, appropriate responses to the stimuli, or a combination of both. In the introduction, it would help if the authors review how infrequent targets affect response patterns.

    In addition, it would help if the theoretical approach and assumptions of the authors were explicitly stated. On p. 23, lines 481-483: The connectivity analysis between the frontal and occipital areas as a way to get at the effect of vigilance is useful, but some consideration of the theoretical justification for this analysis should be added here. The a priori assumption surrounding this analysis should be acknowledged and discussed in the interpretation of the pattern of results (e.g., p. 32, line 658). Based on the analysis between frontal and occipital areas, we have to assume it's the sensory processing alone, but this does not preclude other influences. For instance, effects could also occur on response patterns. These considerations should be added as caveats to the interpretation and to avoid the impression of a confirmation bias.

  2. eLife

    ###Reviewer #2:

    In the manuscript "Neural signatures of vigilance decrements predict behavioural errors before they occur", Karimi-Rouzbahani and colleagues present a study which used a multiple-object monitoring task in combination with magnetoencephalography (MEG) recordings in humans to investigate the neural coding and decoding-based connectivity of vigilance decrements. They found that increasing the rarity of targets led to weaker decoding accuracy for the crucial feature (distance to an object), and weaker decoding was also found for misses compared to correct responses. They also report a drop in decoding-based connectivity between frontal and occipital/parietal regions of interest for misses, and they could predict upcoming performance errors early during a trial based on accumulative decoding accuracy for the relevant target feature.

    This is an interesting study with a quite complex paradigm and a very interesting analysis approach. However, the logic of the approach and the results are rather difficult to unpack, and I am not convinced that it is always correct. My main issues are: Firstly, it is not clear what role eye fixations play here. Participants could freely scan the display, so the retinotopic representations would change depending on where the participants fixate, but at the same time the authors claim that eye position did not matter. Secondly, the display of the results is very dense, and it is not always clear whether decoding for a specific variable was above chance or not. The authors often focused on relative differences, making it difficult to fully understand the meaning of the full pattern of results. Thirdly, the connectivity analysis appears to be a correlation of decoding results between two regions of interest. The more parsimonious interpretation here is that information might have been represented across all channels at this time. Lastly, while this is methodologically interesting work, there is no convincing case made for what exactly the contribution of this study is for theories of vigilance. It seems that the findings can be reduced to that a lack of decodability of relevant target features from brain activity predicts that participants will miss the target. I have outlined my specific comments below.

    1. Methods, Page 11: The authors state that "We did not perform eye-blink artefact removal because it has been shown that blink artefacts are successfully ignored by multivariate classifiers as long as they are not systematically different between decoded conditions (Grootswagers et al., 2017)." I actually doubt that this is really true. Firstly, the cited paper makes a theoretical argument rather than showing this empirically. Secondly, even if this were true, the frequency of eye-related artefacts seems to be of crucial importance for a paradigm that involves moving stimuli (and no fixation). There could indeed be systematic differences between conditions that are then picked up by the classifier (i.e. if more eye-blinks are related to tiredness and in turn decreased vigilance). The authors should show that their results replicate if standard artefact removal is performed on the data.

    2. Relatedly, on page 16 the authors claim that "If the prediction from the MEG decoding was stronger than that of the eye tracking, it would mean that there was information in the neural signal over and above any artefact associated with eye movement." In my view, this statement is problematic: Firstly, such a result might only mean that prediction from MEG decoding is stronger than decoding from eye-movements, but not relate to "artefacts" in general, to which blinks would also count. Secondly, given that the signal underlying both analyses is entirely different (and the number of features), it is not valid to directly compare the results between these analyses.

    3. Results: The Bayes-factor plots in the decoding results figures are so cramped that it is very difficult to actually see the individual dots and to unpack all of this (e.g., Fig 3). I'm wondering whether this complexity could be somehow reduced, maybe by dividing the panels into separate figures? The two top panels in Figure 3B should also include the chance level as in A. It looks like the accuracy is very low for unattended trials, which is only true in comparison to attended trials, but (as also shown in Supplementary Figure 1) it was clearly also encoded in unattended trials, which is very important for interpreting the results.

    4. The section on informational brain connectivity already contains a fair bit of interpretation and discussion in relation to the literature (e.g., "Weaker connectivity between occipital and frontal areas could have led to the behavioural misses observed in this study [...]"). This should be avoided.

    5. Relatedly, if I understand the informational brain connectivity analysis correctly, the authors only show that frontal and occipital/parietal patterns of decoding results are correlated? This means, if one "region" allows for decoding the distance to the object, the other one does too. However, this alone does not equal connectivity. It could simply mean that patterns across the entire brain allow for decoding the same information. For example, it would not be surprising to find that both ROIs correlate more strongly for correct trials (i.e. the brain has obviously represented the relevant information) than for errors (i.e. the brain has failed to represent the information), without this necessarily being related to connectivity at all. The information might simply be spread-out across all channels. The authors show no evidence that only these two (arbitrarily selected) "regions" encode the information while others do not. In my view, to show evidence for meaningful connectivity, a) the spread of information should be limited to small sub-regions, and b) the decoding results in one "region" should predict the results in another region in time (as for DCM).

    6. Predicting miss trials: The implicit assumption here is that there is "less representation" for miss trials compared to correct trials (e.g., of distance to object). But even for miss trials, the representation is significantly above chance. However, maybe the lower accuracy for the miss trials resulted from on average more trials in which the target was not represented at all rather than a weaker representation across all trials. This would call into questions the interpretation of a decline in coding. In other words, on a single trial, a representation might only be present (but could result in a miss for other reasons) or not present (which would be the case for many miss trials), and the lower averages for misses would then be the result of more trials in which the information was completely absent.

    7. Having said that, I am wondering whether the results of the subsequent analysis (predicting misses and correct responses before they occur) might be in conflict with my more pessimistic interpretation. If I understand this correctly, here the classifier predicts Distance to Object for each individual trial, and Fig 6B shows that while there is a clear difference between the correct and miss trials, the latter can still be predicted above chance level but never exceed the threshold? If this is true for all single trials, this would indeed speak for a weak but "unused" representation on miss trials. But for this the authors need to show how many of the miss trials per participant had a chance-level accuracy (i.e. might be truly unrepresented), and how many were above chance but did not exceed the threshold (i.e. might have been "less represented").

    8. In general, it is not clear to me how the brain decoding results were impacted by participants freely looking around on the screen. I am not convinced that decoding from the strongly reduced feature space of eye movements necessarily gives an answer. More detailed analyses of fixations and fixation duration on targets and distractors might indeed be strongly related to behaviour. What is decodable at a given time might just be driven by what participants are looking at.

    9. Discussion: The authors discuss their connectivity results in relation to previous studies on connectivity changes in mind wandering. However, given that the connectivity analysis here is questionable, I'm not sure these results can be meaningfully related.

    10. Overall, even if the issues above are addressed, the study only demonstrates that with less attention to the target, there is less evidence of representations of the relevant features of targets in the brain. The authors also find the expected decrements for rare targets and when participants do not actively monitor the targets. While this is interesting, in particular to directly show this in neural representations, I am not sure whether this is also a conceptually novel contribution to the field. It seems that these general effects are quite well-known from previous work (although demonstrated with different methods)? I am not sure how these findings actually contribute to "theories of vigilance", as claimed by the authors.

  3. eLife

    ###Reviewer #1:

    Karimi-Rouzbahani and colleagues investigate vigilance and sustained monitoring, using a complex and intriguing task in which participants attend to multiple colored dots moving towards the center and occasionally make. They use computationally sophisticated multivariate analyses of MEG data to disentangle attentional factors in this task. The authors demonstrate that they can decode spatial location of the dot (left vs. right) as well as the spatial distance from the critical deflection location, and relate the multivariate decoding ability to features of the task. In addition, they develop methods that can predict errors by accumulating information from distance-based classifiers in the time window preceding behavioral responses. While I was intrigued by this paper, I had numerous questions about the details of their multivariate pattern analyses and the conclusions that they drew from them.

    1. One key finding was that while classifying the direction of the dots was modulated by attention, it was insensitive to many features that were captured by a classifier trained to decode the distance from the deflection. In some ways, I find this very surprising because both are spatial features that seem hard to separate. In addition, the procedures to decode direction vs distance were very different. Therefore, I wonder if there would still be a lack of an effect if the procedure used to train the direction classifier was more analogous or matched?

    2. The distance classifier was trained using only correct trials. Then in the testing stage, it was generalized to either correct or miss trials. While I understand the rationale for using correct trials, I wonder if decoding of error prediction is an artifact of the training sample, reflecting the fact that misses were not included in the training set?

    3. By accumulating classifiers across time, it looks like classifier prediction improves closer to deflection. However, this could also be due to the fact that the total amount of information provided to the classifier increased. I understand the rationale that additional information improves classification, but I wonder if that is because classifiers are relatively poor at distinguishing adjacent distances? Alternatively, perhaps there is a way to control for the total amount of information at different timepoints (e.g., by using a trailing window lag rather than accumulation), or contrast the classifier that derives from accumulating information with the classifier trained moment-by-moment?

    4. The relationship between the vigilance decrement and error prediction. Is vigilance decrement driving the error prediction? That is, if errors increase later on, and the signal goes down, then maybe the classifier is worse. Alternatively, maybe the classifier predictions do not necessarily monotonically decrease throughout the experiment. I wonder if the classifier is equally successful at predicting errors early and late?

    5. When decoding of distance, one thing I found intriguing is that active decoding declines from early to late, even though performance does not decline (or even slightly improves from early to late). This discrepancy seems hard to explain. Is this decline in classification driven by differences in the total signal from early to late?

    6. I noted that classifier performance was extremely high almost immediately after trial onset. Does the classifier perform at chance before the trial onset, or does this reflect sustained but not stimulus-specific information?

  4. eLife

    ##Preprint Review

    This preprint was reviewed using eLife’s Preprint Review service, which provides public peer reviews of manuscripts posted on bioRxiv for the benefit of the authors, readers, potential readers, and others interested in our assessment of the work. This review applies only to version 4 of the manuscript.

    This manuscript is under revision at eLife.

    ###Summary:

    Karimi-Rouzbahani and colleagues investigate vigilance and sustained monitoring, using a multiple-object monitoring task in combination with magnetoencephalography (MEG) recordings in humans to investigate the neural coding and decoding-based connectivity of vigilance decrements. Using computationally sophisticated multivariate analyses of the MEG data, they found that increasing the rarity of targets led to weaker decoding accuracy for the crucial feature (distance to an object), and weaker decoding was also found for misses compared to correct responses.

  5. Version published to 10.1101/2020.06.29.178970 on bioRxiv