Neuronal activity in dorsal anterior cingulate cortex during economic choices under variable action costs

  1. Xinying Cai
  2. Camillo Padoa-Schioppa

  • Curated by eLife

    eLife logo

    Evaluation Summary:

    This manuscript examines how effort is integrated into economic decisions by recording neural activity from the dorsal anterior cingulate cortex (ACC) in monkeys, while requiring animals to choose between different juice types offered in variable amounts and with different action costs. The ACC is a relevant area because some theories have suggested it is important for evaluating or selecting among potential actions during decision-making, although evidence supporting this idea has been inconsistent. The main results provide evidence against the notion that ACC contributes to evaluation of potential actions. Instead, neurons predominantly coded for post-decision variables, such as cost of the chosen target and the juice type of the chosen offer, but not pre-decision variables, such as offer values. This is in contrast to OFC encoding in the same task (and same subjects), in which neurons encoded the effort associated with choice options. The authors conclude that ACC is unique in representing more post-decision variables than OFC, and in its encoding of outcomes in several reference frames (chosen juice, chosen cost, and chosen action). Together, the results are convincing and highlight potentially unique roles of ACC neurons in learning and decision making.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #2 agreed to share their name with the authors.)

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The role of the dorsal anterior cingulate cortex (ACCd) in decision making has often been discussed but remains somewhat unclear. On the one hand, numerous studies implicated this area in decisions driven by effort or action cost. On the other hand, work on economic choices between goods (under fixed action costs) found that neurons in ACCd encoded only post-decision variables. To advance our understanding of the role played by this area in decision making, we trained monkeys to choose between different goods (juice types) offered in variable amounts and with different action costs. Importantly, the task design dissociated computation of the action cost from planning of any particular action. Neurons in ACCd encoded the chosen value and the binary choice outcome in several reference frames (chosen juice, chosen cost, chosen action). Thus, this area provided a rich representation of post-decision variables. In contrast to the OFC, neurons in ACCd did not represent pre-decision variables such as individual offer values in any reference frame. Hence, ongoing decisions are unlikely guided by ACCd. Conversely, neuronal activity in this area might inform subsequent actions.

Article activity feed

  1. Version published to 10.7554/elife.71695 on eLife
  2. eLife

    Author Response:

    Reviewer #2:

    Cai & Padoa-Schioppa recorded from macaque dorsal anterior cingulate cortex (ACCd) while requiring animals to choose between different juice types offered in variable amounts and with different action costs. Authors compared neural activity in ACCd (present study) with previous, directly comparable, findings on this same task when recording in macaque orbitofrontal cortex. The behavioral task is very powerful and the analyses of both the choice behavior and neural data are rigorous. Authors conclude that ACCd is unique in representing more post-decision variables and in its encoding of chosen value and binary outcome in several reference frames (chosen juice, chosen cost, and chosen action), not offer value, like OFC. Indeed, the encoding of choice outcomes in ACCd was skewed toward a cost-based reference frame. Overall, this is important new information about primate ACCd. I have only a few suggestions to enhance clarity. Figures 5 and 7 are maximally informative, but it is not clear that Figure 6 adds much to the reported Results. It is also suggested to abbreviate the comparison with Hosokawa et al. as it presently takes up 3 paragraphs in the Discussion: it is clear the methods and task designs were different enough to not be so easily compared with the present study. An additional suggestion would be to include mention of the comparison with OFC in the abstract and possibly also in the title, since the finding and direct comparison in Figure 7 are some of the most novel and interesting effects of the paper. Other suggestions are minor, and have to do with definition of time windows, variables, and additional papers that authors may cite for a well-rounded Discussion.

    Please refer to Essential Revisions point #4. And we added “In contrast to the OFC” in the abstract to highlight the difference between these two regions.

    Essential Revisions Point #4 Response:

    We shortened the discussion from 3 paragraphs to 1 paragraph as follows.

    "In another study, Hosokawa, Kennerley et al. (2013) compared the neuronal coding in ACCd and OFC in a choice task involving cost-benefit tradeoff. Our findings differ in two aspects. First, Hosokawa et. al. (2013) reported contralateral action value coding in ACCd while we did not discover significant offer value coding in either spatial- or action-based reference frames in our ACCd recordings. Second, they reported that there was no action-based value representation in the OFC therefore concluded that OFC does not integrate action cost in economic choice. Two elements may help explain the discrepancies between our findings in ACCd and OFC (Cai and Padoa-Schioppa 2019) and those of Hosokawa et. al. (2013). First, we recall that Hosokawa et. al. (2013) only tested value-related variables such as the benefit, cost and discounted value in action-based reference frame. Most importantly, they did not test the variable that is related to the saccade direction, which is highly correlated with the spatial value signal. As a consequence, contralateral value signal may not be significant if chosen target location was included in their regression analysis. Indeed, in our analysis, saccade direction (or chosen target location) was identified as one of the variables that explained a significant portion of neuronal activity in ACCd (Cai and Padoa-Schioppa 2012, Cai and Padoa-Schioppa 2019).The second and often overlooked aspect is that value may be encoded in schemes other than the action-based reference frame. In their study, each unique combination of reward quantity and cost was presented by a unique picture. Thus, information on good attributes were conveyed to the animal with an “integrated” visual representation. Accordingly, a distinct group of neurons may have been recruited to encode the reward and cost conjunctively represented by a unique fractal, which would result in 16 groups of offer value coding neurons."

    Reviewer #3:

    Cai and Padoa-Schioppa present a paper titled 'Neuronal Activity in Dorsal Anterior Cingulate Cortex during Economic Choices under Variable Action Costs'. They used a binary choice task where both offers indicated the reward type, reward amount, and the action cost (but not the specific action.) Variable action costs were then operationalized by placing targets on concentric circles of different radius. Here, and in a previous study that included OFC recordings (Cai and Padoa-Schioppa, 2019), monkeys integrated action costs into their decisions. Single-unit recordings in ACCd revealed that neurons predominantly coded for post-decision variables, such as cost of the chosen target and the juice type of the chosen offer, but not pre-decision variables, such as offer values. Given this finding, the authors compared the percentage of neurons in OFC and ACCd that coded for decision variables. In OFC neurons, the activity was mostly restricted to the offer presentation phase, whereas ACCd neurons showed sustained coding of chosen value and costs that lasted until the appearance of the saccade targets. Overall, this is an interesting study that provides evidence that decision-related signals evolve from coding offer values in the OFC to representing chosen costs in the ACC. This finding could highlight the roles of ACC neurons in learning and decision making. We have only a few questions.

    1. Do any of the variables used in this study correlate with a conflict? When the authors previously studied ACC, they discarded the conflict monitoring hypothesis - a hypothesis that is well established for ACC hemodynamic responses - for ACC single cell activity based on neural data from 'difficult' decisions (Cai and Padoa-Schioppa, 2012). The definition of difficulty they used, then, was descriptive and based on reaction times (RTs). They defined the most difficult trials as those trials with the longest RTs and discovered that those trials had options with similar offer values. This definition of choice difficulty appears to be contrived from evidence accumulation models/tasks, where normatively harder judgments elicit longer RTs. However, there is no normative economic reason that trials with similar offer values are more difficult or should cause conflict. After all, according to theory, choosing between two options with the same value is as easy as flipping a coin. Here, it seems like the authors could have a more fitting definition of conflict. For example, conflict can be operationalized by considering trials when the animal must choose between a high value/high-cost option and a low-value/low-cost option. In that case, the costs and benefits are in conflict. What do the RTs look like? Do the RTs indicate conflict resolution? If so, is this reflected in neuronal responses?

    We thank the reviewer for raising this important point. First, we would like to clarify that both in this study and in our previous study of ACC (Cai and Padoa-Schioppa 2012) we imposed a delay between offer presentation and the go signal. Such delay is critical to disentangle value comparison from action selection. However, the delay effectively dissociates reaction times from the decision difficulty. Normally, we operationalize the decision difficulty (or conflict) with the variable value ratio = chosen value / unchosen value. In an early behavioral study conducted in capuchin monkeys, where no delay was imposed between offer presentation and the go signal, we found that reaction times were strongly correlated with the value ratio, as one would naturally expect (Padoa-Schioppa, Jandolo et al. 2006). In the previous study of ACC (Cai and Padoa-Schioppa 2012) we referenced that earlier result but, again, we did not analyze reaction times.

    Coming to the present study, we addressed this question by including in the variable selection analyses the two variables value ratio and cost/benefit conflict = cost of A * sign(offer value A – offer value B) (see also Table 2). The results of the updated analysis are illustrated in the new Figure 4, which we include here below. In essence, including these two variables did not affect the results of the variable selection analysis. That is, both the stepwise and best-subset methods selected the variables chosen value, chosen cost, chosen juice, chosen offer location only and chosen target location only.

    Figure 4. Population summary of ANCOVA (all time windows). (A) Explained responses. Row and columns represent, respectively, time windows and variables. In each location, the number indicates the number of responses explained by the corresponding variable in that time window. For example, chosen value (juice) explained 34 responses in the post-offer time window. The same numbers are also represented in gray scale. Note that each response could be explained by more than one variable and thus could contribute to multiple bins in this panel. (B) Best fit. In each location, the number indicates the number of responses for which the corresponding variable provided the best fit (highest R2 in that time window. For example, chosen value (juice) provided the best fit for 40 responses in the late-delay time window. The numerical values are also represented in gray scale. In this plot, each response contributes to at most one bin.

    1. The authors claimed that the ACCd neurons integrated juice identity, juice quantity and action costs later in the trial. As they acknowledge, the evidence for this claim is marginal. The conclusion the authors made in line 211, therefore, could be moderated. Given that the model containing cost-related variables is more complex, it is equally valid and more appropriately to write '… we cannot reject the null hypothesis that action cost was not integrated by chosen value responses later in the trial.

    We acknowledge the complexity of this claim. However, results from previous studies (Kennerley, Dahmubed et al. 2009, Kennerley and Wallis 2009, Hosokawa, Kennerley et al. 2013) are in favor of establishing a null hypothesis of integration rather than non-integration. Therefore, we feel that it is more appropriate to keep the null hypothesis of cost integration while in the meantime acknowledging that in our study the evidence for cost integration is rather weak.

  3. eLife

    Evaluation Summary:

    This manuscript examines how effort is integrated into economic decisions by recording neural activity from the dorsal anterior cingulate cortex (ACC) in monkeys, while requiring animals to choose between different juice types offered in variable amounts and with different action costs. The ACC is a relevant area because some theories have suggested it is important for evaluating or selecting among potential actions during decision-making, although evidence supporting this idea has been inconsistent. The main results provide evidence against the notion that ACC contributes to evaluation of potential actions. Instead, neurons predominantly coded for post-decision variables, such as cost of the chosen target and the juice type of the chosen offer, but not pre-decision variables, such as offer values. This is in contrast to OFC encoding in the same task (and same subjects), in which neurons encoded the effort associated with choice options. The authors conclude that ACC is unique in representing more post-decision variables than OFC, and in its encoding of outcomes in several reference frames (chosen juice, chosen cost, and chosen action). Together, the results are convincing and highlight potentially unique roles of ACC neurons in learning and decision making.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #2 agreed to share their name with the authors.)

  4. eLife

    Reviewer #1 (Public Review):

    This manuscript examines how effort is integrated into economic decisions by recording neural activity from the dorsal anterior cingulate cortex (ACC) in monkeys. The ACC is a relevant area because some theories have suggested it is important for evaluating or selecting among potential actions during decision-making, although evidence supporting this idea has been inconsistent. The study design and analyses follow a long line of research from this group that has mainly focused on responses of OFC neurons during economic choices, so that the results in the present study can be directly compared to OFC which broadens the potential interpretation.

    In the task, subjects made choices between variable amounts of different flavors of juice that required different amounts of effort to obtain. The level of effort was indicated before the direction of movement, effectively separating the effort evaluation from action planning. The main results provide evidence against the notion that ACC contributes to evaluation of potential actions or computation of effort-based choices. Although neural signals correlating with effort evaluation were found, these were nearly all related to post-choice variables (i.e. the information encoded depended on which option had been chosen, meaning that they more likely reflect the outcome of the choice rather than the inputs to it). This is in contrast to OFC encoding in the same task (and same subjects), in which neurons encoded the effort associated with choice options. This contrast supports the role of OFC, but not ACC, in economic choices involving action evaluation, but leaves open the possibility that ACC retrospectively evaluates or tracks past choices that involve different effort costs.

    Overall, the study is well designed and executed, and the comparison with previously published OFC data provides an important contrast. Although the evidence against the notion that ACC is uniquely involved in evaluating actions to make a decision is reasonably strong, the impact of this result is modest given that there have been a number of previous publications refuting this perspective. I have a few specific comments below, but the major consideration in my view is whether the idea that ACC contributes to decisions by evaluating potential actions is still prominent enough for this study to be of high impact. Though I do think it could provide something of a final word on the issue.

  5. eLife

    Reviewer #2 (Public Review):

    Cai & Padoa-Schioppa recorded from macaque dorsal anterior cingulate cortex (ACCd) while requiring animals to choose between different juice types offered in variable amounts and with different action costs. Authors compared neural activity in ACCd (present study) with previous, directly comparable, findings on this same task when recording in macaque orbitofrontal cortex. The behavioral task is very powerful and the analyses of both the choice behavior and neural data are rigorous. Authors conclude that ACCd is unique in representing more post-decision variables and in its encoding of chosen value and binary outcome in several reference frames (chosen juice, chosen cost, and chosen action), not offer value, like OFC. Indeed, the encoding of choice outcomes in ACCd was skewed toward a cost-based reference frame. Overall, this is important new information about primate ACCd. I have only a few suggestions to enhance clarity. Figures 5 and 7 are maximally informative, but it is not clear that Figure 6 adds much to the reported Results. It is also suggested to abbreviate the comparison with Hosokawa et al. as it presently takes up 3 paragraphs in the Discussion: it is clear the methods and task designs were different enough to not be so easily compared with the present study. An additional suggestion would be to include mention of the comparison with OFC in the abstract and possibly also in the title, since the finding and direct comparison in Figure 7 are some of the most novel and interesting effects of the paper. Other suggestions are minor, and have to do with definition of time windows, variables, and additional papers that authors may cite for a well-rounded Discussion.

  6. eLife

    Reviewer #3 (Public Review):

    Cai and Padoa-Schioppa present a paper titled 'Neuronal Activity in Dorsal Anterior Cingulate Cortex during Economic Choices under Variable Action Costs'. They used a binary choice task where both offers indicated the reward type, reward amount, and the action cost (but not the specific action.) Variable action costs were then operationalized by placing targets on concentric circles of different radius. Here, and in a previous study that included OFC recordings (Cai and Padoa-Schioppa, 2019), monkeys integrated action costs into their decisions. Single-unit recordings in ACCd revealed that neurons predominantly coded for post-decision variables, such as cost of the chosen target and the juice type of the chosen offer, but not pre-decision variables, such as offer values. Given this finding, the authors compared the percentage of neurons in OFC and ACCd that coded for decision variables. In OFC neurons, the activity was mostly restricted to the offer presentation phase, whereas ACCd neurons showed sustained coding of chosen value and costs that lasted until the appearance of the saccade targets. Overall, this is an interesting study that provides evidence that decision-related signals evolve from coding offer values in the OFC to representing chosen costs in the ACC. This finding could highlight the roles of ACC neurons in learning and decision making. We have only a few questions.

    1. Do any of the variables used in this study correlate with a conflict? When the authors previously studied ACC, they discarded the conflict monitoring hypothesis - a hypothesis that is well established for ACC hemodynamic responses - for ACC single cell activity based on neural data from 'difficult' decisions (Cai and Padoa-Schioppa, 2012). The definition of difficulty they used, then, was descriptive and based on reaction times (RTs). They defined the most difficult trials as those trials with the longest RTs and discovered that those trials had options with similar offer values. This definition of choice difficulty appears to be contrived from evidence accumulation models/tasks, where normatively harder judgments elicit longer RTs. However, there is no normative economic reason that trials with similar offer values are more difficult or should cause conflict. After all, according to theory, choosing between two options with the same value is as easy as flipping a coin. Here, it seems like the authors could have a more fitting definition of conflict. For example, conflict can be operationalized by considering trials when the animal must choose between a high value/high-cost option and a low-value/low-cost option. In that case, the costs and benefits are in conflict. What do the RTs look like? Do the RTs indicate conflict resolution? If so, is this reflected in neuronal responses?

    2. The authors claimed that the ACCd neurons integrated juice identity, juice quantity and action costs later in the trial. As they acknowledge, the evidence for this claim is marginal. The conclusion the authors made in line 211, therefore, could be moderated. Given that the model containing cost-related variables is more complex, it is equally valid and more appropriately to write '... we cannot reject the null hypothesis that action cost was not integrated by chosen value responses later in the trial.

  7. Version published to 10.1101/2021.06.14.448291v2 on bioRxiv
  8. Version published to 10.1101/2021.06.14.448291v1 on bioRxiv