History biases reveal novel dissociations between perceptual and metacognitive decision-making

  1. Christopher S.Y. Benwell
  2. Rachael Beyer
  3. Francis Wallington
  4. Robin A. A. Ince

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Human decision-making and self-reflection often depend on context and internal biases. For instance, decisions are often influenced by preceding choices, regardless of their relevance. It remains unclear how choice history influences different levels of the decision-making hierarchy. We employed analyses grounded in information and detection theories to estimate the relative strength of perceptual and metacognitive history biases, and to investigate whether they emerge from common/unique mechanisms. Though both perception and metacognition tended to be biased towards previous responses, we observed novel dissociations which challenge normative theories of confidence. Different evidence levels often informed perceptual and metacognitive decisions within observers, and response history distinctly influenced 1 st (perceptual) and 2 nd (metacognitive) order decision-parameters, with the metacognitive bias likely to be strongest and most prevalent in the general population. We propose that recent choices and subjective confidence represent heuristics which inform 1 st and 2 nd order decisions in the absence of more relevant evidence.

Article activity feed

  1. eLife

    ###Reviewer #3

    The authors address an important open question in decision neuroscience: how do perceptual and metacognitive choices arise from the same sensory input? They perform an experiment that asks observers to report both their percept and associated confidence, and then investigate the role of history biases in both perceptual and metacognitive decisions. They find support for (well-established, e.g. https://elifesciences.org/articles/11946 ) second-order theories of metacognition, which state that metacognitive reports arise from more than just the information used to guide first-order perceptual decisions.

    The methods and statistics are rigorous, and the authors have thoroughly reviewed the relevant literature on the topic. The mutual information analysis is a nice way to quantify the size of different biasing factors. However, the paper presents only limited mechanistic insight into history biases, decision processes or metacognition. It would help to clarify what the main insights are, and how they change our understanding of decision-making and its underlying computational nature.

    Technical suggestions/comments:

    -A recent study (https://elifesciences.org/articles/49834 ) shows the importance of correcting for slow fluctuations within a session, to accurately estimate history biases that are due to the previous trial itself. It would be great to see if the results hold after such a correction, to ensure that the effects are not due to slow fluctuations in choice and metacognitive reports within a session.

    -The answer to whether perceptual and metacognitive decisions arise from the same or distinct processes may depend strongly on the nature of the confidence report. Would the same conclusions hold if choice and confidence were indicated with the same action (e.g. https://elifesciences.org/articles/12192 )?

    -Half of participants base their confidence ratings mostly on confidence history, rather than on evidence. Does this just mean people were not very motivated to do well on the confidence report? Were they rewarded to perform well on both tasks?

    -The authors test the population-level prevalence and effect sizes of perceptual and metacognitive history biases? However, with n = 37 their size does not seem large or diverse enough to infer properties of the general population.

  2. eLife

    ###Reviewer #2

    In this paper, Benwell and colleagues present a perceptual decision-making task with confidence ratings in human subjects to investigate choice and metacognitive history biases. The statistical analyses are solid and thorough. The mutual information analysis is a valuable complement. The paper is comprehensive, clearly written and the figures are informative. It would help if the authors could emphasise their reasoning across the paper: why selecting this particular paradigm to study history biases, and what is the added value of their findings beyond previous work? Many similar data sets (perceptual decision & confidence) have been published and are open. Moreover, it seems important to better justify their hypotheses and motivate the analyses, particularly the final focus on repeat vs. alternation in a paradigm in which these were not manipulated explicitly by the experimenters.

    The authors suggest that history biases are adaptive (building on the stability of real-life environments), whereas at other times, that they are maladaptive ("irrelevant factors such as previous confidence reports"). It would be helpful to explicit the arguments in favour of either interpretation, and to clarify what computational interpretation the present findings favour, if any. There is already a little bit about why it may be advantageous to assume stability. Are there other reasons to think of choice/metacognitive bias as helpful vs. maladaptive? In which contexts? If repeating were a more prevalent bias than alternating in the population, why would this be useful? Relatedly, it would be helpful to further clarify for readers why it is relevant to study choice and confidence history biases, i.e. explain why it is not a simple by-product of experimental designs where experimenters artificially present multiple times very similar decisions.

    It is interesting that the authors comment on the prevalence of each result in their sample, instead of simply reporting statistics on group means, to get a better sense of the strength of the findings. However, it is a bit difficult to generalise about "population prevalence" unless larger samples than the current n=37 are used. Because the experimental design overlaps with previous work, most of these analyses could be re-done on other datasets to address discrepancies between the present findings and that of Urai et al. The confidence database (Rahnev et al., 2020) may provide a useful resource for future work (especially for drawing conclusions about population prevalence based on the current sample of 37 subjects).

    Technical suggestions/comments:

    Could the authors indicate the proportion of errors vs. correct trials on previous repeating vs. alternative trials? If there were more errors on alternating trials, could it be that due to a post error slowing mechanism whereby subjects became more accurate after an alternate, hence the increase in d'? Regarding Fig S1 and the comparison with previous studies, it would be worth discussing the results in relation to a related study in rats (Hermoso, Hyafil et al., 2020 ncomms), for instance examining if the choice bias was overall driven by correct trials and absent after errors (Fig 2A). Finally, in Fig S1 the authors show a choice bias present for both high and low confidence at t-1. Would it be more precise, for concluding about a lack of an influence of confidence, to perform an ordinal regression analysis using the 4 levels of confidence available?

    In Fig 1F, 1G, 1H, could the authors perform psychometric and statistical analyses to actually demonstrate the findings that the authors describe, or rephrase that the confirmation of model predictions are qualitative only? (For instance, showing quantitatively that the slopes are different for high and low confidence in Fig. 1F)

    In particular, when comparing Fig 1F and 1G, are the patterns for confidence and RTs identical with respect to absolute orientation? If so, is this an issue for interpreting confidence data (supposedly not only a strict reflection of RTs but also incorporating information about accuracy)?

    Could the authors comment on (even briefly) the other psychometric parameters (stimulus independent lapse rate and slope)? Why do the stimulus-independent lapses fixed and not fitted with the two other psychometric parameters? Does it change the conclusions if they are fitted? It would be worth checking parameters of the sigmoid psychometric function in Fig S1 (right panel red curve), because the psychometric function looks unusual with an increase at highly positive orientations.

    It is reassuring to see that the correlation results in Fig. S4 are reproduced using an alternative metric of metacognitive efficiency (meta-d'/d'). However, could the authors provide this measure for all other analyses based on meta-d'-d'? I am not asking for a detailed breakdown or new figures, but at least in the text specify whether findings are maintained using this alternative metric.

    Could the authors argue that here we have a true metacognitive history bias, and not a bias due to low-level effects e.g. motor anchoring, use of scale? (See e.g. Foda, H., Barger, K., Navajas, J., & Bahrami, B. (2017). Domain-general idiosyncratic anchoring of metacognition.)

  3. eLife

    ###Reviewer #1

    This preprint by Benwell and colleagues aims at studying 'history' biases during perceptual and metacognitive decision-making. The authors rely on a canonical, well-controlled two-alternative orientation discrimination task with confidence reports on a 4-point scale, tested in 37 human observers. Based on both model-free (mutual information) and model-based (type-2 signal detection theory) analyses, the authors report dissociations between perceptual history biases (related to previous perceptual decisions) and metacognitive history biases (related to previous confidence reports). They interpret these two forms of history biases as 'heuristics' informing type-1 (perceptual) and type-2 (confidence) decisions in the absence of sufficient evidence in the current trial.

    History biases have generated a lot of interest over recent years, and the comparison of perceptual and metacognitive history biases in a single behavioral modeling study is interesting. The standard 2-AFC + 4-point confidence paradigm used by the authors is well suited to address the research question outlined by the authors in the introduction. However, I am not entirely satisfied by the definition of metacognitive history biases used by the authors throughout the manuscript – something which is likely to impact some of the dissociations reported by the authors.

    Indeed, the authors report no effect of confidence at trial t-1 on perceptual bias at trial t (lines 147-149). However, this apparent dissociation appears unsurprising if the response at trial t-1 is not considered in the analysis (as shown in Figure 4). Indeed, a high confidence report in favor of a Left response at trial t-1 is likely to bias the perceptual response at trial t in the opposite direction to a high confidence report in favor of a Right response at trial t-1. In other words, when considering metacognitive history biases, the authors should consider four (not two) types of trials, based on the previous confidence report (high/low) but also the previous response (Left/Right). A true dissociation between previous confidence and the current response would imply that the perceptual history bias (i.e., the effect of the previous response on the current response) is of the same magnitude following low confidence reports and high confidence reports. It is very important that the authors re-compute the effect of metacognitive history biases in this response-dependent fashion. Indeed, Figure 4 will show no effect (i.e., an apparent dissociation) even if the perceptual history bias is strongly increased following high confidence reports than low confidence reports. The manuscript should always consider the direction of the previous response when assessing metacognitive history biases, and only claim dissociations if these previous response-dependent analyses reveal no interaction between the size of perceptual history biases and the previous confidence report (high vs. low).

    Along the same lines, Figure 2C should be modified to plot confidence not as a function of the absolute orientation, but as a function of the orientation signed by the provided response (positive for a stimulus orientation consistent with the provided response, and negative for a stimulus orientation inconsistent with the provided response). It is highly likely that confidence does not scale with the absolute orientation as suggested by Figure 2C, but as a function of the consistency of the stimulus orientation with the provided response (highest confidence for an easy stimulus orientation consistent with the provided response, lowest confidence for an easy stimulus orientation inconsistent with the provided response).

    Regarding the use of mutual information (MI) as a metric for comparing all kinds of effects, I wonder whether the authors could explain more extensively their reasoning. I was under the impression that MI is bounded differently for different kinds of variables (e.g., a binary effect is bounded at 1 bit). Comparing binary variables (Left/Right responses) with non-binary variables (1-4 confidence reports) may thus be problematic, unless I have missed something. To validate the use of MI for claiming "sub-optimal metacognitive performance" (lines 193-195) would require to perform sanity checks such as model simulations with optimal metacognitive performance. The goal of these simulations is to show the difference between the pattern of MI obtained for simulations with genuinely optimal metacognitive performance (and matched perceptual performance to the human subjects) and the pattern of MI obtained for human subjects.

    In the type-2 SDT modeling, the authors now separate Left and Right responses when computing their abs(meta-c minus c) measure. As mentioned above, I believe that the authors should always split Left and Right responses when assessing metacognitive history biases and claiming dissociations between type-1 (perceptual) and type-2 (metacognitive) processes. As above, also, model simulations with optimal metacognitive performance but no history biases (neither perceptual nor metacognitive) may be useful to validate the analyses carried out by the authors on their behavioral data.

    In Figure 4, as in Figure 2C, post high confidence and post low confidence should be further split between post high/low confidence in a Left response and post high/low confidence in a Right response. While a high confidence report in the previous trial may not potentiate perceptual sensitivity (d') on the current trial, it is possible that a high confidence report for a Left response in the previous trial would bias perception in favor of a Left response in the current trial more than a low confidence report for a Left response (and vice versa for a Right response).

    Finally, the repeat vs. alternate effect shown in Figure 5 could be driven by at least two very different mechanisms that cannot be teased apart from presented analyses: 1) the reduced orientation sensitivity in repeat trials proposed by the authors, and 2) a non-zero 'repetition lapse' rate (i.e., a fraction of trials in which the subject blindly repeats his/her previous response). To distinguish between these two accounts, it would be very useful to re-plot Figure 5C. Instead of the absolute orientation, one would use as x-axis the orientation signed by the previous response (positive if the current orientation is in the same direction as the previous response, negative otherwise) and plot on the y-axis the fraction of repeat decisions in the current trial. Fitting a sigmoid function to this curve with a non-zero probability of a blind repetition should afford to tell whether the lower apparent sensitivity in repeat trials shown on Figure 5C is actually triggered by blind repetitions – instead of a lower sensitivity to orientation.

  4. eLife

    ##Preprint Review

    This preprint was reviewed using eLife’s Preprint Review service, which provides public peer reviews of manuscripts posted on bioRxiv for the benefit of the authors, readers, potential readers, and others interested in our assessment of the work. This review applies only to version 2 of the manuscript.

    ###Summary

    This preprint describes a behavioral study of 'history biases' during perceptual and metacognitive decisions. The preprint reports dissociations between history biases across successive perceptual decisions and history biases across successive confidence reports. The question of how perceptual decisions and confidence reports arise from the same sensory input is important, and the statistical analyses of the behavioral data are rigorous and thorough. In particular, the proposed mutual information analysis appears valuable to quantify and compare different biasing effects. However, despite these methodological strengths, the preprint does not reveal substantive new insights into the computational processes which may give rise to the observed effects. Also, the preprint reads at times more like a collection of findings than a coherent study of a targeted research question. In this respect, it would be useful to emphasize the added value of the findings beyond the extensive previous work on history biases and on the relation between type-1 (perceptual) and type-2 (metacognitive) decisions. An explicit motivation of the performed analyses, particularly the focus on the repetition vs. alternation contrast at the end of the results, would be very useful to clarify the reasoning followed by the authors throughout the study.

  5. Version published to 10.1101/737999 on bioRxiv