A process model account of the role of dopamine in intertemporal choice
Curation statements for this article:-
Curated by eLife
eLife assessment
This study presents an important reanalysis of a prior dataset testing effects of D2 antagonism on choices in a delay discounting task. While the prior report using standard analyses showed no effects, the current study used a DDM to examine more carefully possible effects on different subcomponents of the decision process. This approach revealed compelling evidence of contrasting effects of D2 blockade on the effect of reward size differences and bias on choice behavior, findings which should be of broad interest to neuroscientists trying to understand dopamine function or the factors influencing choice behavior.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
- Neuroscience (eLife)
Abstract
Theoretical accounts disagree on the role of dopamine in intertemporal choice and assume that dopamine either promotes delay of gratification by increasing the preference for larger rewards or that dopamine reduces patience by enhancing the sensitivity to waiting costs. Here, we reconcile these conflicting accounts by providing empirical support for a novel process model according to which dopamine contributes to two dissociable components of the decision process, evidence accumulation and starting bias. We re-analyzed a previously published data set where intertemporal decisions were made either under the D2 antagonist amisulpride or under placebo by fitting a hierarchical drift diffusion model that distinguishes between dopaminergic effects on the speed of evidence accumulation and the starting point of the accumulation process. Blocking dopaminergic neurotransmission not only strengthened the sensitivity to whether a reward is perceived as worth the delay costs during evidence accumulation (drift rate) but also attenuated the impact of waiting costs on the starting point of the evidence accumulation process (bias). In contrast, re-analyzing data from a D1 agonist study provided no evidence for a causal involvement of D1R activation in intertemporal choices. Taken together, our findings support a novel, process-based account of the role of dopamine for cost-benefit decision making, highlight the potential benefits of process-informed analyses, and advance our understanding of dopaminergic contributions to decision making.
Article activity feed
-
-
Author Response
Reviewer #1 (Public Review):
In the current study, the authors reanalyze a prior dataset testing effects of D2 antagonism on choices in a delay discounting task. While the prior report using standard analysis, showed no effects, the current study used a DDM to examine more carefully possible effects on different subcomponents of the decision process. This approach revealed contrasting effects of D2 blockade on the effect of reward size differences and bias. Effects were uncorrelated, suggesting separate mechanisms perhaps. The authors speculate that these opposing effects explain the variability in effects across studies, since they mean that effects would depend on which of these factors is more important in a particular design. Overall the study is novel and well-executed, and the explanation offers interesting …
Author Response
Reviewer #1 (Public Review):
In the current study, the authors reanalyze a prior dataset testing effects of D2 antagonism on choices in a delay discounting task. While the prior report using standard analysis, showed no effects, the current study used a DDM to examine more carefully possible effects on different subcomponents of the decision process. This approach revealed contrasting effects of D2 blockade on the effect of reward size differences and bias. Effects were uncorrelated, suggesting separate mechanisms perhaps. The authors speculate that these opposing effects explain the variability in effects across studies, since they mean that effects would depend on which of these factors is more important in a particular design. Overall the study is novel and well-executed, and the explanation offers interesting insight into neural processes.
We thank the reviewer for judging our study as interesting and well-executed.
Reviewer #2 (Public Review):
The authors aim to test the hypothesis that dopamine mediates the evaluation of temporal costs in intertemporal choice in humans, with a specific goal of synthesizing the competing accounts and previous results regarding whether dopamine increases or decreases evaluation of delays in comparing differently delayed future rewards. To do this, they computationally dissect the impact of the drug amisulpride, a D2R antagonist, using a variant of a sequential sampling model, the drift-diffusion model (DDM), that is well established in decision-making literature as a cognitive process model of choice. This model allows the dissociation of starting bias from the rate at which decision evidence is integrated ('drift'), which the authors map to different accounts of the role of dopamine: the temporal proximity of an outcome is proposed to impact bias, while the cost of a delay to impact the drift rate of evidence evaluation/accumulation. Consistent with previous results, and perhaps integrating conflicting findings, the authors find that d2R blockade impacts both bias and drift rate in a cohort of 50 participants, demonstrating dopaminergic action at this receptor is implicated in dissociable components of intertemporal choice, with D2R block reducing the bias towards sooner, more temporally proximate rewards as well as enhancing the contrast between reward magnitudes irrespective of delay, effectively diminishing the effect of delay in the drug condition. These effects are consistent across a small subset of alternative models, confirming the multiple cognitive mechanisms through which D2R block impacts intertemporal choice is a robust feature of decisions on this task.
Overall, this study is a detailed dissection of the specific effects of amisulpride on a type of future-oriented, hypothetical intertemporal choice, and provides consistent evidence integrating conflicting accounts that implicate dopaminergic signaling on evaluation of the cognitive costs, such as a delay, on choice. However the specificity of the empirical intervention and the task design limits the interpretation of the broader dopaminergic mechanisms at play in intertemporal choice, especially given the complexity of receptor specificity of this drug, dopamine precursor availability and individual differences and the specifics of the intertemporal choice in this task. As it stands, the results contribute an interesting, synthesized account of how D2R manipulation can impact evaluation of delays in multiple ways, that will likely be useful for motivating future studies and more detailed computational assessments of the cognitive process-level components of intertemporal choice more generally.
We thank the reviewer for the positive overall evaluation of our study. We revised the manuscript according to the reviewer’s comments, addressing also the receptor specificity of amisulpride and the specifics of the administered intertemporal choice task, which further improved the quality of the manuscript.
The focus of this study is important, and delineating the role of DA in intertemporal choice is of high relevance given DA disfunction is prevalent in many psychiatric disorders and a key target of pharmacological treatment. While the hypotheses of the current study are framed with respect to "costs", the task used by the authors reduces these to evaluation of a hypothetical delay, one which the participants do not necessarily experience in the context of the task. In some respects this is reasonable, given the prevalence of this task paradigm in testing temporal aspects of choice in humans in an economic sense. However, humans are also notoriously subject to framing effects and the impact of instructions in cognitive tasks like these, which can limit the generality of the conclusions, and in particular the specific ways in which a delay can be interpreted as costly (for eg cost as loss of potential earnings, cost as effortful waiting, cost as computational/simulation cost in future evaluation). Given the hypothesis recruits the idea of cost in assessing the role of dopamine, testing for generality in the effects of amisulpride in related but differently framed tasks seems critical for making this link in a general sense, and in connecting it to the previous studies in the literature the authors point to as demonstrating conflicting effects.
We agree that it is important to discuss whether our findings for delay costs can be generalized to other costs types as well, such as risk, social costs, effort, or opportunity costs. Based on a recent literature review (Soutschek, Jetter, & Tobler, 2022), we speculate that dopamine may moderate proximity effects also for risk and social costs but not for effortful rewards, though we emphasize that these hypotheses still require more direct empirical evidence. We also discuss the issue that delays can be perceived as costly in different ways. While in some tasks participants actually experience the waiting time until reward delivery, such that delayed rewards are associated with opportunity costs, in our current task paradigm delayed rewards were virtually free of opportunity costs as participants could engage in other reward-related behaviors during the waiting time. Previous studies suggest that lower tonic dopamine levels reduce the sensitivity to opportunity costs (Niv et al., 2007), which seems in line with our finding that amisulpride decreases the influence of delays on the starting bias parameter. Nevertheless, we emphasize that further evidence is needed to decide whether dopamine shows similar effects for experienced and non-experienced waiting costs. In the revised manuscript, we discuss the cost specificity of our findings on p.22:
“An important question refers to whether our findings for delay costs can be generalized to other types of costs as well, including risk, social costs (i.e., inequity), effort, and opportunity costs. In a recent review, we proposed that dopamine might also moderate proximity effects for reward options differing in risk and social costs, whereas the existing literature provides no evidence for a proximity advantage for effort-free over effortful rewards (Soutschek et al., 2022). However, these hypotheses need to be tested more explicitly by future investigations. Dopamine has also been ascribed a role for moderating opportunity costs, with lower tonic dopamine reducing the sensitivity to opportunity costs (Niv et al., 2007). While this appears consistent with our finding that amisulpride (under the assumption of postsynaptic effects) reduced the impact of delay on the starting bias, it is important to note that choosing delayed rewards did not involve any opportunity costs in our paradigm, given that participants could pursue other rewards during the waiting time. Thus, it needs to be clarified whether our findings for delayed rewards without experienced waiting time can be generalized to choice situations involving experienced opportunity costs.”
Further, while the study aims to test the actions of dopamine broadly, the empirical manipulation is limited to the action of amisulpride, a D2R anatgonist. There is little to no discussion of, or control for, the relationship between dopaminergic action at D2 receptors (the site of amisulpride effects) and wider mechanisms of dopaminergic action at other sites eg D1-like receptors, and the interplay between activation at these two receptor types alongside baseline levels of dopamine concentration. This is necessary for a comprehensive account of dopamine effects on intertemporal choice as the authors aim to test, as opposed to a specific test of the role of the D2 receptor, which is what the study achieves. On a related note, in some preparations at least, amisulpride also acts at some of the 5-HT receptors, raising the possibility of a non-dopaminergic mechanism by which this drug might impact intertemporal decisions. This possibility, while it would not be expected to act without dopaminergic effects as well, is consistent with established effects of serotonin on waiting behaviors and patience. Granted, the limits of pharmacology in humans does not necessarily mean this can be controlled for, it should be kept in mind with a systemic manipulation such as this.
We agree with the reviewer that it is important to distinguish between the contributions of D1 and D2 receptors to decision making, given that these receptor families are hypothesized to have dissociable functional roles. We therefore re-analyzed also data on the impact of a D1 agonist on intertemporal decision making (previous findings for this data set were published in Soutschek et al., 2020, Biological Psychiatry). This analysis provided no evidence for significant effects of D1R stimulation on parameters from a drift diffusion model. This suggests that D2R, rather than D1R, activation mediates the impact of proximity on intertemporal choices.
In the revised manuscript, we report the findings for the D1 agonist study on p.16:
“To assess the receptor specificity of our findings, we conducted the same analyses on the data from a study (published previously in Soutschek et al. (2020)) testing the impact of three doses of a D1 agonist (6 mg, 15 mg, 30 mg) relative to placebo on intertemporal choices (between-subject design). In the intertemporal choice task used in this experiment, the SS reward was always immediately available (delay = 0), contrary to the task in the D2 experiment where the delay of the SS reward varied from 0-30 days. Again, the data in the D1 experiment were best explained by DDM-1 (DICDDM-1 = 19,657) compared with all other DDMs (DICDDM-2 = 20,934; DICDDM-3 = 21,710; DICDDM-5 = 21,982; DICDDM-6 = 19,660; note that DDM-4 was identical with DDM-1 for the D1 agonist study because the delay of the SS reward was 0). Neither the best-fitting nor any other model yielded significant drug effects on any drift diffusion parameter (see Table 4 for the best-fitting model). Also model-free analyses conducted in the same way as for the D2 antagonist study revealed no significant drug effects (all HDI95% included zero). There was thus no evidence for any influence of D1R stimulation on intertemporal decisions.”
We discuss the specificity of D2 receptors for moderating the proximity bias on p.17: “This finding represents first evidence for the hypothesis that tonic dopamine moderates the impact of proximity (e.g., more concrete versus more abstract rewards) on cost-benefit decision making (Soutschek et al., 2022; Westbrook & Frank, 2018). Pharmacological manipulation of D1R activation, in contrast, showed no significant effects on the decision process. This provides evidence for the receptor specificity of dopamine’s role in intertemporal decision making (though as caveat it is worth keeping the differences between the tasks administered in the D1 and the D2 studies in mind).”
We also agree that amisulpride acts also on 5-HT7 receptors, such that it remains unclear whether also such effects contribute to the observed result pattern. We discuss this limitation in the revised manuscript on p.21:
“Lastly, while the actions of amisulpride on D2/D3 receptors are relatively selective, it also affects serotonergic 5-HT7 receptors (Abbas et al., 2009). Because serotonin was related to impulsive behavior (Mori, Tsutsui-Kimura, Mimura, & Tanaka, 2018), it is worth keeping in mind that amisulpride effects on serotonergic, in addition to dopaminergic, activity might contribute to the observed result pattern.”
Overall the modeling methods are robust and appropriate for the specific test of decision impacts of D2R blockade, and include several prima facie variable alternative models for comparison. Some caution is warranted, since there are not many trials per subject, and some trials are discarded as well as outliers, which raises the question of power. Given the models are fit hierarchically, which gives both group-level and individual-level parameter estimates, the elements are there to probe more deeply into individual differences, and to test how reliably this approach can dissociate the dual effects of bias and drift rate at the individual level, and perhaps correlate it with other informative subject measures of either dopamine activity/capacity or other dopamine-dependent behaviors. Alternative DDMs might also capture some of this individual variation, with meaningful differences potentially in model comparison at the individual level. It should be noted that the scope of these models do not exhaust the ways in which proximity (here, temporal) of rewards and contrast between choice options might be incorporated into a cognitive process model account of choice; all alternatives here rest on the same implicit 2-alternative forced choice assumption of the DDM, and the assumptions of this model are not here tested against other accounts of choice, for example the linear ballistic accumulator (LBA) and its derivatives. Further, the concept of proximity as a global feature of a trial (on average, how soon are these options overall?) is never tested on my read of the alternative models.
We thank the reviewer for these interesting suggestions. First, to explore whether measures of dopaminerigc activity correlate with individual differences in drug effects on DDM parameters, we now report correlations between DDM parameters and performance in the digit span backward task as proxy for dopamine synthesis capacity (Cools et al., 2008). None of these correlation analyses showed significant results. In the revised manuscript, we report these analyses on p.13:
“However, we observed no evidence that individual random coefficients for the drug effects on the drift rate or on the starting bias correlated with body weight, all r < 0.22, all p > 0.10. There were also no significant correlations between DDM parameters and performance in the digit span backward task as proxy for baseline dopamine synthesis capacity (Cools, Gibbs, Miyakawa, Jagust, & D'Esposito, 2008), all r < 0.17, all p > 0.22. There was thus no evidence that pharmacological effects on intertemporal choices depended on body weight as proxy of effective dose or working memory performance as proxy for baseline dopaminergic activity.”
Regarding model comparisons on the individual level, we note that the hierarchical Bayesian modelling approach allows (to the best of our knowledge) computing indices of model fit like DIC only on the group, not the individual level (while accounting for individual differences). However, we agree with the reviewer that theoretically different models might work best in different individuals (depending, for example, on the individual sensitivity to proximity). While such fine-grained model comparisons on the individual level are beyond the scope of the current study (and might not yield robust results given the limited number of trials for each participant), we now discuss this limitation in the revised manuscript (p.17-18):
“We note that the hierarchical modelling approach allowed us to compare models on the group level only, such that in some individuals behavior might better be explained by a different model than DDM-1. Such model comparisons on the individual level, however, were beyond the scope of the current study and might not yield robust results given the limited number of trials per individual.”
Likewise, linear ballistic accumulator (LBA) models represent a further class of process models with different assumptions on the mechanisms underlying the choice process than DDMs. In LBAs, evidence is accumulated separately for each choice alternative, whereas DDMs assume only one accumulation process which integrates attributes from two choice options, limiting the use of DDMs to two-alternative forced-choice scenarios. Nevertheless, proximity effects might be incorporated also in LBA models via modulating the starting point of the option-specific accumulators as a function of proximity. To the best of our knowledge, there is no built-in function in JAGS that allows estimating LBA models in a hierarchical Bayesian fashion (in contrast to, e.g., STAN), such that in the context of the current study it is difficult to directly compare our DDM-based approach with LBA models. It is importance to emphasize, however, that similar to other studies we do not make any claims about whether the choice process per se is best explained by DDMs or LBA models; instead, we focus on how rewards and delay costs affect different components of the decision process within a class of decision models. Nevertheless, we discuss such alternative modelling approaches in the revised manuscript on p.18:
“We also emphasize that alternative process models like the linear ballistic accumulator (LBA) model make different assumptions than DDMs, for example by positing the existence of separate option-specific accumulators rather than only one as assumed by DDMs. However, proximity effects as investigated in the current study might be incorporated in LBA models as well by varying the starting points of the accumulators as function of proximity.”
Lastly, we thank the reviewer for the interesting suggestion to assess whether the starting bias parameter is affected by the overall proximity of offers (sum of delays) instead of by the difference in proximity between the options. We ran a further DDM to test this hypothesis, but this model explained the data worse (DIC = 9,492) than the original DDM (DIC = 9,478). Nevertheless, also the overall proximity DDM yielded a significant amisulpride effect on the impact of reward magnitude on the drift rate, HDImean = 0.83, HDI95% = [0.04; 1.75], underlining the robustness of this effect. In the revised manuscript, we report this analysis on p.12:
“In a further model (DDM-4), we explored whether the starting bias is affected by the overall proximity of the options (sum of delays, Delaysum) rather than the difference in proximity (Delaydiff; see Table 3 for an overview over the parameters included in the various models). Importantly, our original DDM-1 (DIC = 9,478) explained the data better than DDM-2 (DIC = 9,481), DDM-3 (DIC = 10,224), or DDM-4 (DIC = 9,492). Nevertheless, amisulpride moderated the impact of Magnitudediff on the drift rate also in DDM-2, HDImean = 0.86, HDI95% = [0.18; 1.64], and DDM-4, HDImean = 0.83, HDI95% = [0.04; 1.75], and amisulpride also lowered the impact of Delaydiff on the starting bias in DDM-3, HDImean = -0.02, HDI95% = [-0.04; -0.001]. Thus, the dopaminergic effects on these subcomponents of the choice process are robust to the exact specification of the DDM.”
Reviewer #3 (Public Review):
Soutschek and Tobler provide an intriguing re-analysis of inter-temporal choice data on amisulpride versus placebo which provides evidence for an as-yet untested hypothesis that dopamine interacts with proximity to bias choices.
The modeling methods are sound with a robust and reasonably exhaustive set of models for comparison, with good posterior predictive checks at the single subject level, and decent evidence of parameter recoverability. Importantly, they show that while there is no main effect of drug on the proportion of larger, later (LL) versus smaller, sooner (SS) choices, this obscures conflicting-directional effects on drift rate versus starting point bias which are under-the-hood, yet anticipated by the hypothesis of interest.
We thank the reviewer for judging our findings as intriguing and the modelling approach as robust and convincing.
While I have no major concerns about methodology, I think the Authors should consider an alternative interpretation - albeit an interpretation which would actually support the hypothesis in question more directly than their current interpretation. Namely, the Authors should re-consider the possibility that amisulpride's effects are mediated primarily by acting at pre-synaptic receptors. If the D2R antagonist were to act pre-synaptically, it would drive more versus less post-synaptic dopamine signaling.
There are multiple reason for this inference. First, the Authors observe that the drug increases sensitivity to differences in the relative offer amounts (in terms of effects on the drift rate). With respect to the canonical model of dopamine signaling in the direct versus indirect pathway, greater post-synaptic signaling should amplify sensitivity to reward benefits - which is what the Authors observe.
Second, the Authors also observe an effect on the starting bias which may also be consistent with an increase in post-synaptic dopamine signaling. Note that according to the Westbrook & Frank hypothesis, a proximity bias in delay discounting should favor the SS over the LL reward, yet the Authors primarily observe a starting bias in the direction of the LL reward. This contradiction can be resolved with the ancillary assumption that, independent of any choice attribute, participants are on average predisposed to select the LL option. Indeed, the Authors observe a reliable non-zero intercept in their logistic regression model indicating that participants selected the LL more often, on average. As such, the estimated starting point may reflect a combination of a heightened predisposition to select the LL option, opposed by a proximity bias towards the sooner option. Perhaps the estimated DDM starting point is positive because the predisposition to select the LL option has a larger effect on choices than the proximity bias towards sooner rewards does in this data set. To the extent that amisulpride increases post-synaptic dopamine signaling (by antagonizing pre-synaptic D2Rs) it should amplify the proximity bias arising from the differences in delay, shifting the starting bias towards the SS option. Indeed, this is also what the Authors observe.
Note that it remains unclear why an increase in post-synaptic dopamine signaling would amplify one kind of proximity bias (towards sooner over later rewards) without amplifying the other (towards a predisposition to select the LL option). Perhaps the cognitive / psychological nature of the sooner bias is more amenable to interacting with dopamine signaling than the latter. Or maybe proximity bias effects are most sensitive to dopamine signaling when they are smaller, and the LL predisposition bias is already at ceiling in the context of this task. These assumptions would help explain why a potential increase in post-synaptic dopamine signaling both amplified the proximity effect of delay when it was smallest (when the differences in delay were smaller), and also failed to amplify the predisposition to select the LL option (which may already be maxed out). More importantly, the assumption that there are opposing proximity biases would also help explain why there is a negative effect of delay magnitude on the estimated starting point on placebo. Namely - as the delay gets larger, the psychological proximity of sooner over later rewards grows, counteracting the proximity bias arising from choice predisposition / repetition.
We thank the reviewer for suggesting this alternative interpretation of our data. We agree that the administered dose of 400 mg amisulpride can show both postsynaptic (reducing D2R activation) and presynaptic effects (enhancing D2R activation), which in many studies makes it difficult to decide whether the observed behavioral effects are caused by presynaptic or postsynaptic mechanisms.
The reviewer suggests that the observed stronger influence of reward magnitudes on drift rates under amisulpride compared with placebo speaks in favor of presynaptic effects, because according to theoretical accounts higher dopamine levels should increase reward seeking (e.g., Frank & O’Reilly, 2006). On the other hand, Figure 2C suggests that amisulpride (compared with placebo) increased the preference only for relatively high, above-average rewards. If the difference between reward magnitudes was below average, amisulpride reduced rather than increased the preference for the larger reward. In our view, this is consistent with the hypothesis that D2R activation implements a cost control, with higher D2R activation increasing the attractiveness of costly rewards and lower D2R activation reducing it. In other words, under low dopamine levels individuals should decide for the costlier reward only if the magnitude of the costlier reward is sufficiently large compared with the lower, less costly reward. In fact, this is exactly what we find in our data according to Figure 2C. In our view, the amisulpride effect on drift rates is thus compatible with both presynaptic and postsynaptic mechanisms of action, depending on the underlying conceptual account of dopamine, as we now discuss in the revised manuscript.
According to the reviewer, also the observed influence of amisulpride on the starting bias speaks in favor of increased rather than reduced dopamine levels. We agree with the reviewer that the result pattern for the starting bias is somewhat complex and seems to combine the effects of two different biases: a general tendency to choose LL over SS rewards (intercept of starting bias where the difference in delays is close to zero), and a shift towards the SS option under placebo if one options has a strong (temporal) proximity advantage over the other. Amisulpride shows opposite effects on the two different biases, as it shifts the intercept of the starting bias further away from the LL option but also reduces the proximity advantage of the SS over the LL reward for larger differences in delay. The reviewer writes that “To the extent that amisulpride increases post-synaptic dopamine signaling (by antagonizing pre-synaptic D2Rs) it should amplify the proximity bias arising from the differences in delay, shifting the starting bias towards the SS option. Indeed, this is also what the Authors observe.” In contrast to that statement, in our study amisulpride reduced rather than increased the starting bias arising from delay (as in Figure 2K the regression line is flatter under amisulpride compared with placebo, despite the differences regarding the intercept). We believe that the amisulpride effects on both the intercept and the delay-dependent slope can be explained via postsynaptic effects: First, the shift of the intercept of the starting bias (small differences in proximity) from the LL towards the SS option under amisulpride is consistent with the assumption that lower dopamine reduces the preference for larger reward (e.g., Beeler & Mourra, 2018; Salamone & Correa, 2012). Second, the finding that amisulpride weakens the proximity advantage of SS over LL rewards (delay-dependent slope) is consistent with the proximity account by Westbrook & Frank (2018) according to which lower tonic dopamine should reduce proximity effects. Thus, if we assume that the result pattern for the starting bias parameter is driven by dopaminergic effects on two separate decision biases (as suggested by the reviewer), we believe that both effects can better be explained by pharmacologically reduced rather than increased dopamine levels.
In the revised manuscript, we extensively discuss the question as to whether the observed drug effects are caused by postsynaptic versus presynaptic effects. We clarify that the amisulpride effect on drift rates seems consistent with both presynaptic and postsynaptic effects (depending on the underlying conceptual account). We moreover discuss that the starting bias effects may reflect the interaction between two different bias types, and the drug effects on both bias types can more easily be reconciled with postsynaptic than presynaptic effects. On balance, we believe that the observed effects are more likely to reflect lower as compared to higher dopamine levels, but the extended discussion of this issue gives all readers the opportunity to weigh the arguments for and against these alternatives. If the reviewer should not agree with some aspects of our argumentation as outlined above, we would of course be happy to modify the discussion according to the reviewer’s advice.
In the revised manuscript, we modified the discussion of presynaptic versus postsynaptic effects as follows (p.20-21):
“While higher doses of amisulpride (as administered in the current study) antagonize post-synaptic D2Rs, lower doses (50-300 mg) were found to primarily block pre-synaptic dopamine receptors (Schoemaker et al., 1997), which may result in amplified phasic dopamine release and thus increased sensitivity to benefits (Frank & O'Reilly, 2006). At first glance, the stronger influence of differences in reward magnitude on drift rates under amisulpride compared with placebo might therefore speak in favor of presynaptic (higher dopamine levels) rather than postsynaptic mechanisms of action in the current study. On the other hand, one could argue that amisulpride reduced the preference for the LL reward if the gain from the costlier LL option compared with the SS option was small (as suggested by Figure 2C), which is consistent with the cost control hypothesis of dopamine (Beeler & Mourra, 2018). The impact of amisulpride on the drift rate thus appears ambiguous regarding the question of pre- versus postsynaptic effects. The result pattern for the starting bias parameter, in turn, suggests the presence of two distinct response biases, reflected by the intercept and the delay-dependent slope of the bias parameter (see Figure 2K), which are both under dopaminergic control but in opposite directions. First, participants seem to have a general bias towards the LL option in the current task (intercept), which is reduced under amisulpride compared with placebo, consistent with the assumption that dopamine strengthens the preference for larger rewards (Beeler & Mourra, 2018; Salamone & Correa, 2012; Schultz, 2015). Second, amisulpride reduced the proximity advantage of SS over LL rewards with increasing differences in delay, as predicted by the proximity account of tonic dopamine (Westbrook & Frank, 2018). On balance, the current results thus appear more likely under the assumption of postsynaptic rather than presynaptic effects. Unfortunately, the lack of a significant amisulpride effect on decision times (which should be reduced or increased as consequence of presynaptic or postsynaptic effects, respectively) sheds no additional light on the issue.”
Regardless of the final interpretation, showing that pharmacological intervention into striatal dopamine signaling can simultaneously modify a starting point bias and drift rate (in opposite directions - thus having systematic effects on choice biases without altering the average proportion of LL choices) provides crucial first evidence for the hypothesis that dopamine and proximity interact to influence decision-making. These results thereby enrich our understanding of the neuromodulatory mechanisms influencing inter-temporal choice, and take an important step towards resolving prior contradictions in this literature. They also have implications for how striatal dopamine might impact decision-making in diverse domains of impulsivity beyond inter-temporal choice, ranging from cognitive neuroscience (e.g. in numerous cognitive control tasks) to psychiatry (treating diverse disorders of impulse control).
We thank the reviewer for highlighting the importance of the current findings for understanding dopamine’s role in decision making.
-
eLife assessment
This study presents an important reanalysis of a prior dataset testing effects of D2 antagonism on choices in a delay discounting task. While the prior report using standard analyses showed no effects, the current study used a DDM to examine more carefully possible effects on different subcomponents of the decision process. This approach revealed compelling evidence of contrasting effects of D2 blockade on the effect of reward size differences and bias on choice behavior, findings which should be of broad interest to neuroscientists trying to understand dopamine function or the factors influencing choice behavior.
-
Reviewer #1 (Public Review):
In the current study, the authors reanalyze a prior dataset testing effects of D2 antagonism on choices in a delay discounting task. While the prior report using standard analysis, showed no effects, the current study used a DDM to examine more carefully possible effects on different subcomponents of the decision process. This approach revealed contrasting effects of D2 blockade on the effect of reward size differences and bias. Effects were uncorrelated, suggesting separate mechanisms perhaps. The authors speculate that these opposing effects explain the variability in effects across studies, since they mean that effects would depend on which of these factors is more important in a particular design. Overall the study is novel and well-executed, and the explanation offers interesting insight into neural …
Reviewer #1 (Public Review):
In the current study, the authors reanalyze a prior dataset testing effects of D2 antagonism on choices in a delay discounting task. While the prior report using standard analysis, showed no effects, the current study used a DDM to examine more carefully possible effects on different subcomponents of the decision process. This approach revealed contrasting effects of D2 blockade on the effect of reward size differences and bias. Effects were uncorrelated, suggesting separate mechanisms perhaps. The authors speculate that these opposing effects explain the variability in effects across studies, since they mean that effects would depend on which of these factors is more important in a particular design. Overall the study is novel and well-executed, and the explanation offers interesting insight into neural processes.
-
Reviewer #2 (Public Review):
The authors aim to test the hypothesis that dopamine mediates the evaluation of temporal costs in intertemporal choice in humans, with a specific goal of synthesizing the competing accounts and previous results regarding whether dopamine increases or decreases evaluation of delays in comparing differently delayed future rewards. To do this, they computationally dissect the impact of the drug amisulpride, a D2R antagonist, using a variant of a sequential sampling model, the drift-diffusion model (DDM), that is well established in decision-making literature as a cognitive process model of choice. This model allows the dissociation of starting bias from the rate at which decision evidence is integrated ('drift'), which the authors map to different accounts of the role of dopamine: the temporal proximity of an …
Reviewer #2 (Public Review):
The authors aim to test the hypothesis that dopamine mediates the evaluation of temporal costs in intertemporal choice in humans, with a specific goal of synthesizing the competing accounts and previous results regarding whether dopamine increases or decreases evaluation of delays in comparing differently delayed future rewards. To do this, they computationally dissect the impact of the drug amisulpride, a D2R antagonist, using a variant of a sequential sampling model, the drift-diffusion model (DDM), that is well established in decision-making literature as a cognitive process model of choice. This model allows the dissociation of starting bias from the rate at which decision evidence is integrated ('drift'), which the authors map to different accounts of the role of dopamine: the temporal proximity of an outcome is proposed to impact bias, while the cost of a delay to impact the drift rate of evidence evaluation/accumulation. Consistent with previous results, and perhaps integrating conflicting findings, the authors find that d2R blockade impacts both bias and drift rate in a cohort of 50 participants, demonstrating dopaminergic action at this receptor is implicated in dissociable components of intertemporal choice, with D2R block reducing the bias towards sooner, more temporally proximate rewards as well as enhancing the contrast between reward magnitudes irrespective of delay, effectively diminishing the effect of delay in the drug condition. These effects are consistent across a small subset of alternative models, confirming the multiple cognitive mechanisms through which D2R block impacts intertemporal choice is a robust feature of decisions on this task.
Overall, this study is a detailed dissection of the specific effects of amisulpride on a type of future-oriented, hypothetical intertemporal choice, and provides consistent evidence integrating conflicting accounts that implicate dopaminergic signaling on evaluation of the cognitive costs, such as a delay, on choice. However the specificity of the empirical intervention and the task design limits the interpretation of the broader dopaminergic mechanisms at play in intertemporal choice, especially given the complexity of receptor specificity of this drug, dopamine precursor availability and individual differences and the specifics of the intertemporal choice in this task. As it stands, the results contribute an interesting, synthesized account of how D2R manipulation can impact evaluation of delays in multiple ways, that will likely be useful for motivating future studies and more detailed computational assessments of the cognitive process-level components of intertemporal choice more generally.
The focus of this study is important, and delineating the role of DA in intertemporal choice is of high relevance given DA disfunction is prevalent in many psychiatric disorders and a key target of pharmacological treatment. While the hypotheses of the current study are framed with respect to "costs", the task used by the authors reduces these to evaluation of a hypothetical delay, one which the participants do not necessarily experience in the context of the task. In some respects this is reasonable, given the prevalence of this task paradigm in testing temporal aspects of choice in humans in an economic sense. However, humans are also notoriously subject to framing effects and the impact of instructions in cognitive tasks like these, which can limit the generality of the conclusions, and in particular the specific ways in which a delay can be interpreted as costly (for eg cost as loss of potential earnings, cost as effortful waiting, cost as computational/simulation cost in future evaluation). Given the hypothesis recruits the idea of cost in assessing the role of dopamine, testing for generality in the effects of amisulpride in related but differently framed tasks seems critical for making this link in a general sense, and in connecting it to the previous studies in the literature the authors point to as demonstrating conflicting effects.
Further, while the study aims to test the actions of dopamine broadly, the empirical manipulation is limited to the action of amisulpride, a D2R anatgonist. There is little to no discussion of, or control for, the relationship between dopaminergic action at D2 receptors (the site of amisulpride effects) and wider mechanisms of dopaminergic action at other sites eg D1-like receptors, and the interplay between activation at these two receptor types alongside baseline levels of dopamine concentration. This is necessary for a comprehensive account of dopamine effects on intertemporal choice as the authors aim to test, as opposed to a specific test of the role of the D2 receptor, which is what the study achieves. On a related note, in some preparations at least, amisulpride also acts at some of the 5-HT receptors, raising the possibility of a non-dopaminergic mechanism by which this drug might impact intertemporal decisions. This possibility, while it would not be expected to act without dopaminergic effects as well, is consistent with established effects of serotonin on waiting behaviors and patience. Granted, the limits of pharmacology in humans does not necessarily mean this can be controlled for, it should be kept in mind with a systemic manipulation such as this.
Overall the modeling methods are robust and appropriate for the specific test of decision impacts of D2R blockade, and include several prima facie variable alternative models for comparison. Some caution is warranted, since there are not many trials per subject, and some trials are discarded as well as outliers, which raises the question of power. Given the models are fit hierarchically, which gives both group-level and individual-level parameter estimates, the elements are there to probe more deeply into individual differences, and to test how reliably this approach can dissociate the dual effects of bias and drift rate at the individual level, and perhaps correlate it with other informative subject measures of either dopamine activity/capacity or other dopamine-dependent behaviors. Alternative DDMs might also capture some of this individual variation, with meaningful differences potentially in model comparison at the individual level. It should be noted that the scope of these models do not exhaust the ways in which proximity (here, temporal) of rewards and contrast between choice options might be incorporated into a cognitive process model account of choice; all alternatives here rest on the same implicit 2-alternative forced choice assumption of the DDM, and the assumptions of this model are not here tested against other accounts of choice, for example the linear ballistic accumulator (LBA) and its derivatives. Further, the concept of proximity as a global feature of a trial (on average, how soon are these options overall?) is never tested on my read of the alternative models.
-
Reviewer #3 (Public Review):
Soutschek and Tobler provide an intriguing re-analysis of inter-temporal choice data on amisulpride versus placebo which provides evidence for an as-yet untested hypothesis that dopamine interacts with proximity to bias choices.
The modeling methods are sound with a robust and reasonably exhaustive set of models for comparison, with good posterior predictive checks at the single subject level, and decent evidence of parameter recoverability. Importantly, they show that while there is no main effect of drug on the proportion of larger, later (LL) versus smaller, sooner (SS) choices, this obscures conflicting-directional effects on drift rate versus starting point bias which are under-the-hood, yet anticipated by the hypothesis of interest.
While I have no major concerns about methodology, I think the Authors …
Reviewer #3 (Public Review):
Soutschek and Tobler provide an intriguing re-analysis of inter-temporal choice data on amisulpride versus placebo which provides evidence for an as-yet untested hypothesis that dopamine interacts with proximity to bias choices.
The modeling methods are sound with a robust and reasonably exhaustive set of models for comparison, with good posterior predictive checks at the single subject level, and decent evidence of parameter recoverability. Importantly, they show that while there is no main effect of drug on the proportion of larger, later (LL) versus smaller, sooner (SS) choices, this obscures conflicting-directional effects on drift rate versus starting point bias which are under-the-hood, yet anticipated by the hypothesis of interest.
While I have no major concerns about methodology, I think the Authors should consider an alternative interpretation - albeit an interpretation which would actually support the hypothesis in question more directly than their current interpretation. Namely, the Authors should re-consider the possibility that amisulpride's effects are mediated primarily by acting at pre-synaptic receptors. If the D2R antagonist were to act pre-synaptically, it would drive more versus less post-synaptic dopamine signaling.
There are multiple reason for this inference. First, the Authors observe that the drug increases sensitivity to differences in the relative offer amounts (in terms of effects on the drift rate). With respect to the canonical model of dopamine signaling in the direct versus indirect pathway, greater post-synaptic signaling should amplify sensitivity to reward benefits - which is what the Authors observe.
Second, the Authors also observe an effect on the starting bias which may also be consistent with an increase in post-synaptic dopamine signaling. Note that according to the Westbrook & Frank hypothesis, a proximity bias in delay discounting should favor the SS over the LL reward, yet the Authors primarily observe a starting bias in the direction of the LL reward. This contradiction can be resolved with the ancillary assumption that, independent of any choice attribute, participants are on average predisposed to select the LL option. Indeed, the Authors observe a reliable non-zero intercept in their logistic regression model indicating that participants selected the LL more often, on average . As such, the estimated starting point may reflect a combination of a heightened predisposition to select the LL option, opposed by a proximity bias towards the sooner option. Perhaps the estimated DDM starting point is positive because the predisposition to select the LL option has a larger effect on choices than the proximity bias towards sooner rewards does in this data set. To the extent that amisulpride increases post-synaptic dopamine signaling (by antagonizing pre-synaptic D2Rs) it should amplify the proximity bias arising from the differences in delay, shifting the starting bias towards the SS option. Indeed, this is also what the Authors observe.
Note that it remains unclear why an increase in post-synaptic dopamine signaling would amplify one kind of proximity bias (towards sooner over later rewards) without amplifying the other (towards a predisposition to select the LL option). Perhaps the cognitive / psychological nature of the sooner bias is more amenable to interacting with dopamine signaling than the latter. Or maybe proximity bias effects are most sensitive to dopamine signaling when they are smaller, and the LL predisposition bias is already at ceiling in the context of this task. These assumptions would help explain why a potential increase in post-synaptic dopamine signaling both amplified the proximity effect of delay when it was smallest (when the differences in delay were smaller), and also failed to amplify the predisposition to select the LL option (which may already be maxed out). More importantly, the assumption that there are opposing proximity biases would also help explain why there is a negative effect of delay magnitude on the estimated starting point on placebo. Namely - as the delay gets larger, the psychological proximity of sooner over later rewards grows, counteracting the proximity bias arising from choice predisposition / repetition.
Regardless of the final interpretation, showing that pharmacological intervention into striatal dopamine signaling can simultaneously modify a starting point bias and drift rate (in opposite directions - thus having systematic effects on choice biases without altering the average proportion of LL choices) provides crucial first evidence for the hypothesis that dopamine and proximity interact to influence decision-making. These results thereby enrich our understanding of the neuromodulatory mechanisms influencing inter-temporal choice, and take an important step towards resolving prior contradictions in this literature. They also have implications for how striatal dopamine might impact decision-making in diverse domains of impulsivity beyond inter-temporal choice, ranging from cognitive neuroscience (e.g. in numerous cognitive control tasks) to psychiatry (treating diverse disorders of impulse control).
-