Neuronal origins of reduced accuracy and biases in economic choices under sequential offers

  1. Weikang Shi
  2. Sebastien Ballesta
  3. Camillo Padoa-Schioppa

  • Curated by eLife

    eLife logo

    Evaluation Summary:

    Padoa-Schioppa and colleagues describe possible neuronal correlates of behavioral biases observed in monkeys making value-based choices when options are presented simultaneously versus sequentially. Building on the lab's previous work detailing functional roles of different neurons in the orbitofrontal cortex, the authors relate different choice biases to different groups of OFC neurons. They propose that these relationships indicate that different biases are likely to arise from specific stages of decision computation. The study results are convincing and represent a significant advance in understanding circuit-level computations underlying decision-making.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #3 agreed to share their name with the authors.)

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Economic choices are characterized by a variety of biases. Understanding their origins is a long-term goal for neuroeconomics, but progress on this front has been limited. Here, we examined choice biases observed when two goods are offered sequentially. In the experiments, rhesus monkeys chose between different juices offered simultaneously or in sequence. Choices under sequential offers were less accurate (higher variability). They were also biased in favor of the second offer (order bias) and in favor of the preferred juice (preference bias). Analysis of neuronal activity recorded in the orbitofrontal cortex revealed that these phenomena emerged at different computational stages. Lower choice accuracy reflected weaker offer value signals (valuation stage), the order bias emerged during value comparison (decision stage), and the preference bias emerged late in the trial (post-comparison). By neuronal measures, each phenomenon reduced the value obtained on average in each trial and was thus costly to the monkey.

Article activity feed

  1. eLife

    Author Response

    Reviewer #2 (Public Review):

    This manuscript from Shi, Ballesta, and Padoa-Schioppa examines the relationship between neural activity in the monkey orbitofrontal cortex (OFC) and various choice patterns that arise in sequential (versus simultaneous) choice. This approach addresses a central question in the study of decision-making: how can one identify value-dependent versus value-independent effects on choice behavior when value is defined from that behavior itself? Here, the authors document three behavioral differences in sequential choice: choosers are nosier, show an order bias, and show a preference bias. Leveraging a conceptual computational framework for OFC activity that the authors have developed over many years, the authors link reduced accuracy to changes in neural valuation in the OFC, order effects to post-valuation decision activity in the OFC, and preference effects to extra-OFC processes. For decision neuroscientists, these findings show specific differences between sequential and simultaneous choice, and suggest the integration of multiple stages (valuation, decision, and post-decision) in the selection process. More broadly, this work shows how an examination of neural activity can shed light on aspects of the decision process that cannot be distinguished by an examination of behavior alone.

    Strengths:

    Overall, this paper presents a novel and thoughtful task design that allows comparison of neural and behavioral value and choice effects. In concert with an established circuit-based framework for parsing different types of OFC response patterns, the authors test and validate a number of hypotheses on the link between neural activity and choice.

    (1) Comparing sequential and simultaneous choice tasks in an interleaved manner is a clever approach to separate valuation and comparison processes in time. While not entirely novel (e.g. see work from the Hayden group), the combination of this approach with the OFC response pattern (offer value, chosen value, chosen juice) framework allows a distinction between valuation and comparison-related effects.

    (2) This paper is the latest in a significant series of related papers on orbitofrontal activity from this group, and cleverly utilizes their expertise in characterizing, analyzing, and conceptualizing different patterns of OFC activity. In addition to the long-established offer value/chosen value/chosen juice categorization, recent papers from this group have established the causal contribution of OFC offer value activity to economic choice and established similar OFC neural contributions to sequential and simultaneous choice tasks.

    (3) Apart from a causal test (e.g. cell type specific stimulation) of the contribution of different neural responses to different choice effects, the next strongest evidence is a demonstration of a consistent relationship across sessions. The authors show such a relationship between offer value coding strength and choice accuracy, between chosen value sequence effects and behavioral order bias, and between chosen juice inhibition and order bias. At the least, these relatively strong effects show a strong correlation between different OFC responses and behavior.

    Thank you for emphasizing these points.

    Weaknesses:

    While the experimental approach and rigor of the analyses are strengths, there are issues of interpretation and generality of analytical approaches that should be clarified.

    (1) The abstract, introduction, and discussion touch on canonical behavioral economic choice effects as a prelude to the behavioral effects documented here, but it's not clear they are so closely related. [A] Many of the effects in the cited literature (framing effects in risky choice, preference reversals, etc.) are robust across different task paradigms, whereas the effects shown here arise specifically from a comparison of choice across different task paradigms (sequential vs. simultaneous). Furthermore, [B] it's not clear that the term "bias" adequately captures the array of effects in the behavioral economic literature (for that matter, [C] one of the main effects in this paper is reduced choice accuracy rather than a bias). [D] The paper would benefit from a clearer conceptual linkage between documented behavioral biases (particularly in humans) and the effects shown here.

    [B] We beg to differ. In our reading of the literature, the term “bias” is very general and it is invoked practically every time choices present some effect that seems idiosyncratic or “irrational”. The list of documented biases is very long – a good reference is the Wikipedia page on cognitive biases (for more scholarly references, see (Gilovich et al., 2002; Kahneman et al., 1982)).

    [A] As for whether biases documented in behavioral economics are robust across task paradigms, that’s really matter of perspectives. For example, we all understand the phenomenon of loss aversion (a.k.a. “status quo bias”) to be very robust and almost intuitive. But before the prospect theory paper of Kahneman and Tversky (1979), that was not at all the case. In the 15 years following that paper, much of what Kahneman and Tversky did was to show how loss aversion affected choices in different domains (Kahneman and Tversky, 2000). Other biases are much less reliable. For example, there is an extensive literature on decoy effects – i.e., violations of the axiom of “independence of irrelevant alternatives”. However, it turns out that the strength and even the direction of decoy effects depend on seemingly minor details (Spektor et al., 2021). In other words, decoy effects are not as robust as one might think. As for the biases dicussed here, our hunch is that the order bias is quite ubiquitous. Indeed, it was already documented using different tasks in different species (Krajbich et al., 2010; Rustichini et al., 2021). The preference bias might also be the manifestation of a rather general phenomenon. Afterall, there is a common intuition that when a decision is difficult we sometimes fail to finalize it, and eventually choose some default option. In conclusion, we think of the two biases discussed here as conceptually very comparable to biases described in behavioral economics.

    [C] We agree that the drop in accuracy is (strictly speaking) not a choice bias, and we carefully chose the title and wrote the whole manuscript to keep that point clear. However, let us note that the drop in accuracy observed under sequential offers could easily be construed as a choice bias – specifically, a bias favoring in any situation the lesser option (lower value). As we conclude the present study, this phenomenon continues to fascinate us. Indeed, while it is clear that the behavioral effect arises at the valuation stage, we still don’t understand why the activity range of offer value cells is reduced under sequential offers. Naively, one might have guessed the opposite – i.e., that when only one offer is on display, the lack of competition translates to stronger offer value signals. We plan to give this issue more thought in the future. One possibility is that the system modulates the activity range of offer value cells depending on the task and/or the behavioral context. If so, differences in choice accuracy measured under sequential versus simultaneous offers would be a manifestation of a more general phenomenon. Of course, this matter remains open for future research.

    [D] The link between the biases discussed here and other biases described in the literature is conceptual. The main point we want to make is this: Over the past 20 years, we have gained some understanding of the neural circuit and mechanisms underlying simple economic choices. While our understanding remains incomplete and object of ongoing research, notions acquired for simple choices can be used to make sense of a broader class of choices. Thus, in principle at least, it is possible to shed light on a variety of traits and biases by observing the activity of particular cell groups. The last paragraph of the ms conveys this point.

    (2) The analyses rely on a particular quantification of choice behavior (probit regression), which interprets choice effects (e.g. relative valuation of the two juices, sigmoid steepness) via specific parameter combinations and relies on specific assumptions about the construction of choice (e.g. cumulative normal distribution, constant sigmoid slope across order effects). This method of quantifying choice behavior is well-documented in previous studies, allowing a comparison to past work. However, given the importance of this approach to both quantifying choice effects and comparing choice to OFC responses, the paper would benefit from directly addressing two issues: (1) how well does probit regression actually capture stochastic choice behavior (in both Task 1 and Task 2), and (2) do the findings rely on specific choice modeling assumptions? The second issue is most important for the order bias effects, which assume a constant sigmoid across conditions - do the authors reach similar conclusions if this assumption is relaxed?

    Thanks for raising this question. We address it more thoroughly below (under “Recommendations for the authors”, point (2)). In a nutshell, when we designed the behavioral analysis, we chose the probit function and the log value ratio model (as opposed to the value difference model) based on general considerations and for consistency with our previous studies. We now conducted a series of control analyses using logit instead of probit and value difference instead of log value ratio. We also repeated all the analyses of neuronal activity using measures for relative value, choice accuracy and order bias derived from these behavioral models. The upshot is that all of our results hold true independently of the regression model used to analyze choices. Thus we kept the results as in the original ms, and we included a new section in the Methods to describe our control analyses (p.16-17).

    (3) There are some issues with the strength and interpretation of the preference bias that need to be addressed. Re: strength and significance of the preference bias, the text seems to overemphasize the dependence of the effect on relative value (rotation of the rho-2 vs rho-1 ellipse) at the cost of the simple task difference (shift in the ellipse above the identity line). Conceptually, a preference bias (an shift in relative value towards the favored item) requires only the task difference, not the dependence on relative value. It would be clearer for example if the main text (pg. 6) presented the statistics (t-test, Wilcoxon) supporting the difference in relative values (rhos) between Tasks 1 and 2. Furthermore, the rotation does not seem as robust: the text states that the result is significant in both animals (p<0.04) but the ANCOVA results (Fig 3C and 3F) suggests that the effect is only significant in Monkey J. Is the preference effect significant only in one animal, and if so, is the effect significant across the combined data?

    Let us refer to Fig.3C. There is no question that the separation between the red and blue lines is statistically significant (order bias). In addition, the two lines appear (a) displaced upwards and (b) rotated counterclockwise compared to the identity line. In our understanding, the question raised by R2 is whether the two effects – displacement (a) and rotation (b) – are both present and both necessary to define the preference bias. We actually gave this issue extensive thought early on, and we concluded that displacement and rotation are not easily dissociable, at least in our data set. The reason is simple: to dissociate them, we would have to make some assumption about the center of rotation. For example, if we assume that the center of rotation is [0, 0], then there clearly is a rotation but the displacement is close to zero. Conversely, if the center of rotation is [1, 1] (which, in some ways, is a more logical assumption), the rotation is still there but the displacement is >0. When we considered these elements, we realized that any choice of a center of rotation would be somewhat arbitrary. Further complicating things, once a center of rotation is chosen, rotation and displacement are non-commutative operations. Importantly, this issue only affects the displacement, meaning that the rotation angle (and its statistical significance) does not depend on choosing any particular center of rotation. In this light, we chose to define the preference bias in a way that is more tight to the rotation than to the displacement, while noting that the net effect of the phenomenon was to bias choices in favor of the preferred juice (hence, the phrase “preference bias”). The only problem with this definition is that it doesn’t do full justice to the phenomenon in monkey G (Fig.3F), where the displacement is more clearly evident than the rotation (indeed, the latter only trends towards statistical significance (p=0.07)). Still, we don’t see a better way to design our analyses. Thus we kept the ms unchanged in this respect.

    (4) On a related note, the authors present and view the effects as detrimental for the animals, but I think they have to more explicitly state how they are defining outcomes. For example, the abstract states "By neuronal measures, each phenomenon reduced the value obtained on average in each trial and was thus costly to the monkey". Does this mean that outcomes are less valuable, with value defined by (offer value cell) firing rates? A clarification is particularly important for the preference bias, where animals show a stronger bias for the preferred option compared to simultaneous choice. At the behavioral level, this effect seems to only be a poorer outcome if one assumes that simultaneous choice demonstrates true values - can it not be assumed that sequential choice demonstrates true preference, and the preference bias reduces performance in simultaneous choice? The authors may have an explanation in mind based on OFC value coding, and it would be helpful to be explicit here.

    Thank you for raising this question. The revised ms includes a new section (Discussion; ‘The cost of choice biases’; p.13) that discusses this important issue. In a nutshell, if in two conditions subjective values are the same but choices are different, in one or both conditions the subject fails to choose the higher value. In that sense, the choice bias is detrimental. Our analyses of neuronal activity indicated that subjective offer values were (a) the same in the two tasks and (b) independent of the presentation offer in Task 2. Hence, both the preference bias and the order bias were detrimental to the animal.

    (5) Finally, at a broad level, the authors rigorously define and test hypotheses about how the different behavioral effects relate to OFC activity within the context of their neurocomputational framework (offer value, chosen value, chosen juice cells arranged in a competitive inhibition network; Fig. 1). However, it should be acknowledged that the primary conclusions - about how the different behavioral effects arise during valuation, comparison, or post-comparison - relies on the assumption that the different OFC response patterns reflect these specific circuit functions, and that OFC is causally related to choice. It would be more balanced if the authors could acknowledge this point in the discussion, and discuss any relevant potential alternative explanations for their findings.

    This issue is addressed above (Essential revision, point 1). In essence, R2 is correct: all our analyses were designed, and all our results are interpreted, under a series of assumptions. Most of these are backed by empirical evidence (e.g., showing that the encoding of decision variables in OFC is categorical in nature). However, one assumption remains a working hypothesis. Specifically, we assume that the cell groups identified in OFC constitute the building blocks of a decision circuit. If so, the activity of different cell groups may be associated with different computational stages. We edited the Discussion to clarify this point (p.11-12). As for possible alternative explanations, we agree that it is a very reasonable question to ask, but we honestly are at a loss addressing it. Indeed, one would never conduct the analyses presented in this ms if not in the framework of Fig.1. Consequently, it is hard to come up with any interpretation for the results without embracing that computational framework. If R2 can propose some alternative interpretation for the results presented in the ms, we would be more than happy to think about it, and possibly revise our thinking.

  2. Version published to 10.7554/elife.75910 on eLife
  3. eLife

    Evaluation Summary:

    Padoa-Schioppa and colleagues describe possible neuronal correlates of behavioral biases observed in monkeys making value-based choices when options are presented simultaneously versus sequentially. Building on the lab's previous work detailing functional roles of different neurons in the orbitofrontal cortex, the authors relate different choice biases to different groups of OFC neurons. They propose that these relationships indicate that different biases are likely to arise from specific stages of decision computation. The study results are convincing and represent a significant advance in understanding circuit-level computations underlying decision-making.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #3 agreed to share their name with the authors.)

  4. eLife

    Reviewer #1 (Public Review):

    This manuscript describes three decision biases, and attempts to relate them to neural recordings from OFC. They analyze neural recordings from a previous experiment, in which animals were either presented with simultaneous or sequential juice offers. They describe three behavioral observations: (1) choices are noisier for sequential offers; (2) for sequential offers, monkeys exhibit an order bias, in which they are more likely to choose the second offer; (3) for sequential offers, monkeys exhibit a preference bias, in which they are more likely to choose the preferred juice. They argue that the first bias (decision noise) occurs at the valuation stage, as they observe reduced value signals in the sequential task. They argue that the order bias occurs at the "value comparison" stage, because neural correlates are reflected in chosen value cells, not offer value cells, shortly preceding and after the second offer. Finally, they argue that the preference bias is a post-decision bias. Understanding the neural correlates of various decision biases is a goal of the fields of decision-making and neuroeconomics. I think the data are of high quality and the findings are interesting. However, I think there are questions about the methodology used to classify neurons as participating in these different cognitive processes (valuation, value comparison, post-decision). This methodology arbitrarily forces neurons to only participate in one process, when in reality individual neurons could support or encode several or all of these processes. In my view, this is the major weakness of the study, but is also central to the interpretation of the results.

  5. eLife

    Reviewer #2 (Public Review):

    This manuscript from Shi, Ballesta, and Padoa-Schioppa examines the relationship between neural activity in the monkey orbitofrontal cortex (OFC) and various choice patterns that arise in sequential (versus simultaneous) choice. This approach addresses a central question in the study of decision-making: how can one identify value-dependent versus value-independent effects on choice behavior when value is defined from that behavior itself? Here, the authors document three behavioral differences in sequential choice: choosers are nosier, show an order bias, and show a preference bias. Leveraging a conceptual computational framework for OFC activity that the authors have developed over many years, the authors link reduced accuracy to changes in neural valuation in the OFC, order effects to post-valuation decision activity in the OFC, and preference effects to extra-OFC processes. For decision neuroscientists, these findings show specific differences between sequential and simultaneous choice, and suggest the integration of multiple stages (valuation, decision, and post-decision) in the selection process. More broadly, this work shows how an examination of neural activity can shed light on aspects of the decision process that cannot be distinguished by an examination of behavior alone.

    Strengths:

    Overall, this paper presents a novel and thoughtful task design that allows comparison of neural and behavioral value and choice effects. In concert with an established circuit-based framework for parsing different types of OFC response patterns, the authors test and validate a number of hypotheses on the link between neural activity and choice.

    (1) Comparing sequential and simultaneous choice tasks in an interleaved manner is a clever approach to separate valuation and comparison processes in time. While not entirely novel (e.g. see work from the Hayden group), the combination of this approach with the OFC response pattern (offer value, chosen value, chosen juice) framework allows a distinction between valuation and comparison-related effects.

    (2) This paper is the latest in a significant series of related papers on orbitofrontal activity from this group, and cleverly utilizes their expertise in characterizing, analyzing, and conceptualizing different patterns of OFC activity. In addition to the long-established offer value/chosen value/chosen juice categorization, recent papers from this group have established the causal contribution of OFC offer value activity to economic choice and established similar OFC neural contributions to sequential and simultaneous choice tasks.

    (3) Apart from a causal test (e.g. cell type specific stimulation) of the contribution of different neural responses to different choice effects, the next strongest evidence is a demonstration of a consistent relationship across sessions. The authors show such a relationship between offer value coding strength and choice accuracy, between chosen value sequence effects and behavioral order bias, and between chosen juice inhibition and order bias. At the least, these relatively strong effects show a strong correlation between different OFC responses and behavior.

    Weaknesses:

    While the experimental approach and rigor of the analyses are strengths, there are issues of interpretation and generality of analytical approaches that should be clarified.

    (1) The abstract, introduction, and discussion touch on canonical behavioral economic choice effects as a prelude to the behavioral effects documented here, but it's not clear they are so closely related. Many of the effects in the cited literature (framing effects in risky choice, preference reversals, etc.) are robust across different task paradigms, whereas the effects shown here arise specifically from a comparison of choice across different task paradigms (sequential vs. simultaneous). Furthermore, it's not clear that the term "bias" adequately captures the array of effects in the behavioral economic literature (for that matter, one of the main effects in this paper is reduced choice accuracy rather than a bias). The paper would benefit from a clearer conceptual linkage between documented behavioral biases (particularly in humans) and the effects shown here.

    (2) The analyses rely on a particular quantification of choice behavior (probit regression), which interprets choice effects (e.g. relative valuation of the two juices, sigmoid steepness) via specific parameter combinations and relies on specific assumptions about the construction of choice (e.g. cumulative normal distribution, constant sigmoid slope across order effects). This method of quantifying choice behavior is well-documented in previous studies, allowing a comparison to past work. However, given the importance of this approach to both quantifying choice effects and comparing choice to OFC responses, the paper would benefit from directly addressing two issues: (1) how well does probit regression actually capture stochastic choice behavior (in both Task 1 and Task 2), and (2) do the findings rely on specific choice modeling assumptions? The second issue is most important for the order bias effects, which assume a constant sigmoid across conditions - do the authors reach similar conclusions if this assumption is relaxed?

    (3) There are some issues with the strength and interpretation of the preference bias that need to be addressed. Re: strength and significance of the preference bias, the text seems to overemphasize the dependence of the effect on relative value (rotation of the rho-2 vs rho-1 ellipse) at the cost of the simple task difference (shift in the ellipse above the identity line). Conceptually, a preference bias (an shift in relative value towards the favored item) requires only the task difference, not the dependence on relative value. It would be clearer for example if the main text (pg. 6) presented the statistics (t-test, Wilcoxon) supporting the difference in relative values (rhos) between Tasks 1 and 2. Furthermore, the rotation does not seem as robust: the text states that the result is significant in both animals (p<0.04) but the ANCOVA results (Fig 3C and 3F) suggests that the effect is only significant in Monkey J. Is the preference effect significant only in one animal, and if so, is the effect significant across the combined data?

    (4) On a related note, the authors present and view the effects as detrimental for the animals, but I think they have to more explicitly state how they are defining outcomes. For example, the abstract states "By neuronal measures, each phenomenon reduced the value obtained on average in each trial and was thus costly to the monkey". Does this mean that outcomes are less valuable, with value defined by (offer value cell) firing rates? A clarification is particularly important for the preference bias, where animals show a stronger bias for the preferred option compared to simultaneous choice. At the behavioral level, this effect seems to only be a poorer outcome if one assumes that simultaneous choice demonstrates true values - can it not be assumed that sequential choice demonstrates true preference, and the preference bias reduces performance in simultaneous choice? The authors may have an explanation in mind based on OFC value coding, and it would be helpful to be explicit here.

    (5) Finally, at a broad level, the authors rigorously define and test hypotheses about how the different behavioral effects relate to OFC activity within the context of their neurocomputational framework (offer value, chosen value, chosen juice cells arranged in a competitive inhibition network; Fig. 1). However, it should be acknowledged that the primary conclusions - about how the different behavioral effects arise during valuation, comparison, or post-comparison - relies on the assumption that the different OFC response patterns reflect these specific circuit functions, and that OFC is causally related to choice. It would be more balanced if the authors could acknowledge this point in the discussion, and discuss any relevant potential alternative explanations for their findings.

  6. eLife

    Reviewer #3 (Public Review):

    This manuscript by Shi and colleagues describes possible neuronal correlates of behavioral biases observed in monkey, when options are presented sequential versus simultaneously. These behavioral effects are that for sequential presentations, the monkeys show: (1) less accurate choices, (2) a preference for the second (more recently) presented option, (3) a preference for the more preferred juice type, independent of amount. The paper builds on a long series of work from the Padoa-Schioppa lab that has identified 3 different functional signals in OFC: (1) option value, (2) chosen value, (3) chosen juice type. Important for the logic of the analysis in this paper, the option value cells can be interpreted as input signals that drive the decision process (a comparison of the value of each option), while the chosen value and juice type signals reflect the outcome of the decision process. The fact that these different cell types represent different stages of the decision process is used here to investigate at which functional stage neuronal activity correlates with behavioral biases. Using this general approach, the authors find that: (1) diminished accuracy is due specifically to lower value sensitivity of option value cells, (2) order bias emerges at the comparison stage, but is not driven by input signals, (3) preference bias is not reflected during the comparison stage (immediately following the presentation of the second option), but seems to emerge later in the trial during a waiting period, before the choice is indicated. The paper is written very clearly and the underlying logic for the different analysis is very well described. The results are for the most part convincing. It is particularly noteworthy that the authors were able to identify very specific mappings between neuronal signals and behavioral effects by showing systematically that one particular class of neurons is correlated with the effect, but others are not. That is very impressive. (Of course, the authors were also lucky, because in principle, an effect could have easily arise from multiple stages of the decision process.) Overall the paper is great. This is one of the first examples, in which suboptimal choices can be convincingly related to fluctuations in the underlying circuit for value estimation and comparison.

  7. Version published to 10.1101/2021.11.07.467645v1 on bioRxiv