Different brain systems support learning from received and avoided pain during human pain-avoidance learning
Curation statements for this article:-
Curated by eLife
Evaluation Summary:
This manuscript is of particular interest to readers in the field of pain research. The identification of separate brain systems associated with learning from unexpected pain and learning from unexpected pain relief contributes to understanding of pain avoidance learning. The combination of behavioral data, neuroimaging and computational modeling provide support for many of the central claims of the paper, however weaknesses in the experimental design limit the support for the claims based on the results of the pharmacological manipulation.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Both unexpected pain and unexpected pain absence can drive avoidance learning, but whether they do so via shared or separate neural and neurochemical systems is largely unknown. To address this issue, we combined an instrumental pain-avoidance learning task with computational modeling, functional magnetic resonance imaging (fMRI), and pharmacological manipulations of the dopaminergic (100 mg levodopa) and opioidergic (50 mg naltrexone) systems ( N = 83). Computational modeling provided evidence that untreated participants learned more from received than avoided pain. Our dopamine and opioid manipulations negated this learning asymmetry by selectively increasing learning rates for avoided pain. Furthermore, our fMRI analyses revealed that pain prediction errors were encoded in subcortical and limbic brain regions, whereas no-pain prediction errors were encoded in frontal and parietal cortical regions. However, we found no effects of our pharmacological manipulations on the neural encoding of prediction errors. Together, our results suggest that human pain-avoidance learning is supported by separate threat- and safety-learning systems, and that dopamine and endogenous opioids specifically regulate learning from successfully avoided pain.
Article activity feed
-
-
Evaluation Summary:
This manuscript is of particular interest to readers in the field of pain research. The identification of separate brain systems associated with learning from unexpected pain and learning from unexpected pain relief contributes to understanding of pain avoidance learning. The combination of behavioral data, neuroimaging and computational modeling provide support for many of the central claims of the paper, however weaknesses in the experimental design limit the support for the claims based on the results of the pharmacological manipulation.
(This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)
-
Reviewer #1 (Public Review):
In an experimental human study with pharmacological functional magnetic resonance imaging (fMRI), Jepma and colleagues aimed to dissect neural brain circuits and neurochemical underpinnings of two mechanisms of pain avoidance learning, namely learning from unexpected pain and learning from unexpected pain relief. By combining behavioral data from a probabilistic pain avoidance learning task with computational reinforcement learning modelling the authors demonstrated that unexpected pain has a stronger influence on participants' decision-making and learning than unexpected pain relief. Interestingly, the difference in learning rates from unexpected pain and pain relief disappeared with pharmacologically transiently increasing phasic dopamine using a single dose (100mg, p.o.) of levodopa as well as with …
Reviewer #1 (Public Review):
In an experimental human study with pharmacological functional magnetic resonance imaging (fMRI), Jepma and colleagues aimed to dissect neural brain circuits and neurochemical underpinnings of two mechanisms of pain avoidance learning, namely learning from unexpected pain and learning from unexpected pain relief. By combining behavioral data from a probabilistic pain avoidance learning task with computational reinforcement learning modelling the authors demonstrated that unexpected pain has a stronger influence on participants' decision-making and learning than unexpected pain relief. Interestingly, the difference in learning rates from unexpected pain and pain relief disappeared with pharmacologically transiently increasing phasic dopamine using a single dose (100mg, p.o.) of levodopa as well as with blocking mu-opioid receptors using a single dose (50mg, p.o.) of naltrexone. Both interventions specifically increased learning from unexpected pain relief with no effect on learning from unexpected pain. In contrast, these pharmacological manipulations did not show any effects on neural data recorded during task performance using fMRI. Nevertheless, distinct patterns of brain responses were found related to pain prediction errors (ie. pain expected but not received) and pain relief prediction errors (ie. no pain expected but pain received).
Despite the known relevance of pain avoidance in the development and maintenance of chronic pain, the mechanisms of pain avoidance learning are still not well understood. A better understanding could contribute to an improvement of therapeutic approaches targeting pain avoidance in chronic pain. Accordingly, the provided results by Jepma and colleagues offer important and novel insights. Data and results largely support the authors' conclusions.
The combination of a validated behavioral task with pharmacological interventions, fMRI, and computational modelling is a strength of the present manuscript. This approach allows a comprehensive investigation of underlying mechanisms, enabling high-quality conclusions. Nevertheless, potential insights are somewhat hinder by the small sample size in combination with a between subject experimental design. Particularly for such research questions, a within-design has many advantages and allow better conclusions. For example, a within comparison on drug effects would have been of high interest in this context and might have reduced error variance in the present data.
The use of computational modelling approaches has many advantages in the context of investigations of learning processes. For example, reinforcement models, as used in the present study, are well validated and enable to draw conclusions on underlying mechanisms. As such they provide a much deeper level of mechanistic understanding compared to direct standard group comparisons of behavioral outcomes. This is obvious in the present study as well, because the authors demonstrate a convincing and reasonable differentiation of learning rates based on unexpectedly received pain or pain relief, which cannot be observed in the simple outcome measures of the task. This is also compellingly resembled in the fMRI results, outlining two distinct brain systems mediating learning from unexpected pain and learning from unexpected pain relief.
Although the fMRI data well supports the idea of two brain systems mediating different learning mechanisms, the pharmacological manipulation does not show any effects on brain responses. While such a dissociations has been reported before in similar contexts, the present study lacks a manipulation check for the pharmacological manipulation or a check of individual differences in the responses to the drugs. Many factors such as height/weight, genetic markup etc. can influence how levodopa and naltrexone is metabolized and thus utilized on individual levels. The doses used in this study resemble standard dosages in similar experiments and positive effects have been often reported, but as well strong individual differences in the response to the drugs have been reported as well. The effects on learning from unexpected pain relief suggest that the pharmacological interventions were effective, albeit they are in part in contrast to the hypotheses. Nevertheless, without a manipulation check (e.g. other independent behavioral results, autonomic measures known to be influence by dopamine/opioids, blood samples, prolactin for dopamine, etc.) the negative results on all other behavioral outcomes and the fMRI data have to be viewed with caution.
-
Reviewer #2 (Public Review):
Jepma et al. report an interesting manuscript studying how we learn from pain and its avoidance. The authors use an instrumental pain avoidance task where participants are required to choose between two stimuli, one of which is followed by painful thermal stimulation to the leg and the other is not. The probabilities of receiving pain drifted across trials using random walks. The authors combined this with pharmacological manipulation of the dopamine (via oral levodopa) or opioid (via oral naltrexone) systems and also with computational modelling of Q-learning rules and neuroimaging via fMRI. So, this is an ambitious and well conceived manuscript.
There are real strengths here. The manuscript is theoretically motivated, addresses a fundamental question about how we learn, and is generally well executed. The …
Reviewer #2 (Public Review):
Jepma et al. report an interesting manuscript studying how we learn from pain and its avoidance. The authors use an instrumental pain avoidance task where participants are required to choose between two stimuli, one of which is followed by painful thermal stimulation to the leg and the other is not. The probabilities of receiving pain drifted across trials using random walks. The authors combined this with pharmacological manipulation of the dopamine (via oral levodopa) or opioid (via oral naltrexone) systems and also with computational modelling of Q-learning rules and neuroimaging via fMRI. So, this is an ambitious and well conceived manuscript.
There are real strengths here. The manuscript is theoretically motivated, addresses a fundamental question about how we learn, and is generally well executed. The task is well controlled, the modelling choices seem appropriate, the imaging and its analyses are broad but well defended and choices in analysis strategies are well defined. The manuscript is well written. I did enjoy reading the manuscript.
The results have some interest. The modelling and neuroimaging data suggest important dissociations between learning about pain and learning about its absence - the modelling suggests faster learning rates for learning from pain than its avoidance. The imaging suggests that these two forms of learning are associated with different networks, with a known network linked to learning about pain but a novel network linked to learning about avoided pain.
These are worthwhile knowledge gains. The idea that different rate parameters govern learning about events that are present versus those that are absent is an old one. It is built into most error-correcting learning rules since Rescorla-Wagner and it makes sense. However, it was useful to see it supported here. The finding that different networks of brain regions were associated with the learning from pain versus avoided of pain was also interesting. The networks linked to the former made sense based on the literature. The networks linked to the latter were more novel and notably did not include classic 'relief' brain regions.
However, there were also important weaknesses here, at least on my readings.
I struggled as a reader to understand how the modelling actually related to the behavior and imaging. That is, there is a real disconnect in the manuscript for me between what is observed (behavior) what is inferred (modelling as well as it basis for correlations with fMRI data).
There were no differences in behavior reported between the two kinds of trials (learning from received pain versus avoided pain) effects, no effects of the drugs on behavioral performance, and no differential effect on learning from received pain versus avoided pain. I have no problems with reporting null effects, but here the reader is left wondering: if there are no behavioral differences reported, then why does the modelling predict that there should be? How accurate is the model given that it clearly predicts slower learning from avoided than received pain in the controls and faster learning from avoided pain under naltrexone and levodopa compared to control? In other words, what is it about the modelling that yields differences in learning rates between the two behavioral conditions and between the vehicle, levodopa, and naltrexone conditions when the behavioral data shown do not? Of course, it could be that the task was too easy - the modelling may be prescient and perhaps possible learning rate differences would be picked up under more difficult (more cues) and weaker probabilistic conditions. Perhaps there are behavioral data (reaction times?) not reported that do actually show differences in learning rate between learning from received pain versus avoided pain or show differences between the drug conditions?
I may have misunderstood all of this and am happy to be corrected. If not, think this issue needs to be addressed and would need new data that is hopefully already in hand to do convincingly (such as choice reaction times) to show some difference in behavior between learning from received pain versus avoided pain and/or some effects of the pharmacological manipulations on these.
In the absence of the data the manuscript seems to have three parts:
1. A more compelling set of findings reporting imaging differences between learning from received pain versus avoided pain that are interesting because they suggest a novel network of brain regions for the latter compared to the literature.
2. A set of null findings that neither pharmacological manipulation affected behavior or these imaging findings.
3. A less compelling set of findings that link the above to possible underlying differences in learning rate parameters.The first could be of interest but the latter two need to be strengthened, in my opinion.
I had other minor points (e.g., consider the literature on opioid and dopamine receptor manipulations in the ventral striatum on aversive prediction errors because this suggests the opposite to the literature cited for the midbrain; is the word 'appetitive' in the title really appropriate given the findings in the manuscript), but these are less important than the above.
-
