Neural dynamics underlying self-control in the primate subthalamic nucleus

  1. Benjamin Pasquereau
  2. Robert S Turner

  • Curated by eLife

    eLife logo

    eLife assessment

    This valuable study by Pasquereau and Turner examined the activity neurons in the subthalamic nucleus (STN) while monkeys performed a task in which they had to withhold their response during a delay period whose length was defined by a specific cue. The results indicate that the activity of STN neurons was modulated by reward size and delay. The results are potentially important for understanding how STN regulates behavior such as self-control, but the reviewers thought that the study is incomplete as the analyses, at least in the presented forms, have some potential problems and some analyses require clarification.

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The subthalamic nucleus (STN) is hypothesized to play a central role in neural processes that regulate self-control. Still uncertain, however, is how that brain structure participates in the dynamically evolving estimation of value that underlies the ability to delay gratification and wait patiently for a gain. To address that gap in knowledge, we studied the spiking activity of neurons in the STN of monkeys during a task in which animals were required to remain motionless for varying periods of time in order to obtain food reward. At the single-neuron and population levels, we found a cost–benefit integration between the desirability of the expected reward and the imposed delay to reward delivery, with STN signals that dynamically combined both attributes of the reward to form a single integrated estimate of value. This neural encoding of subjective value evolved dynamically across the waiting period that intervened after instruction cue. Moreover, this encoding was distributed inhomogeneously along the antero-posterior axis of the STN such that the most dorso-posterior-placed neurons represented the temporal discounted value most strongly. These findings highlight the selective involvement of the dorso-posterior STN in the representation of temporally discounted rewards. The combination of rewards and time delays into an integrated representation is essential for self-control, the promotion of goal pursuit, and the willingness to bear the costs of time delays.

Article activity feed

  1. Version published to 10.7554/elife.83971 on eLife
  2. eLife

    Author Response

    Reviewer #1 (Public Review):

    • The statistical procedures used are not completely described and may not be appropriate.

    We revised the text in Methods and Results sections to give more details about the methods used.

    -As only two levels of delay were tested, it is not possible to directly test whether the subjective discounting function is hyperbolic or exponential and hence whether the delay is encoded subjectively or objectively.

    We agree with the reviewer. A higher number of task parameters may offer a better resolution to evaluate the discounting functions. Fortunately, this does not affect our main results.

    • The task has several variable interval lengths (hold in: 1.2-2.8 s, short delay: 1.8-2.3 s, long delay: 3.5-4s) that frustrate interpretation. The distribution of these delays is not described, for example as it reads it seems possible that some long delay rewards are delivered with shorter latency between cue and reward than some short delay rewards (1.2 + 3.5 = 4.7s vs. 2.8+2.3 = 5.1 s).

    We revised the text to address that ambiguity. In the new version of the manuscript, we describe short versus long delays considering the total delay intervals between instruction cue onset and reward delivery [short delay (3.5-5.6s) and long delay (5.2-7.3s)]. Within each delay category, individual delays were distributed in a gaussian fashion such that the two delay ranges overlapped for 9% of trials. These details are now described in the revised Methods section (pg. 22).

    -The authors have not considered that if the delay value is encoding, then the value, both objectively and subjectively, may be changing as the delay elapses. The variation of these task intervals may have an effect on the value of delay.

    In the present study, we report a dynamic integration between the desirability of the expected reward and the imposed delay to reward delivery across the waiting period. Our results (e.g. see Fig. 6) do not fit with simple linear (or logarithmic) effects corresponding to continuous regular changes as the delay elapses. We found different types of interactions (Discounting± and Compounding±) at different periods of the hold period and in different single units. We did not find a way to model all these types of interactions with this type of approach.

    Reviewer #2 (Public Review):

    • Plots of "rejection rate" (trials where the monkeys failed to wait until the rewards) as a function of delay and reward size seem to indicate that the monkeys understood the visual cue. The rejection rates were very low (less than 4% for almost all conditions) which indicates that the monkeys did not have a hard time inhibiting their behavior. It also meant that the authors could not compare trials where the monkeys successfully waited with trials where they failed to wait. This missing comparison weakens the link between the neurophysiological observations and the conclusions the authors made about the signals they observed.

    Here, our main goal was to describe the dynamic STN signals engaged during the waiting period without studying action-related activities. In the discussion (pg. 20), we clearly wrote ‘Further research is needed to determine whether the neural signals identified here causally drive animals’ behavior or rather just participate to reflect or evaluate the current situation.’ Consequently, our conclusions were already tempered by that point.

    In addition, we address the same limitation by writing (pg. 20): “An important avenue for future research will be to determine how STN signals, such as those described here, change when animals run out of patience and finally decide to stop waiting. To do this, however, smaller reward sizes and longer delays might be used to promote more escape behaviors during the delay interval.”

    • The authors examined the STN activity aligned to the start of the delay and also aligned to the reward. Most of the "delay encoding" in the STN activity was observed near the end of the waiting period. The trouble with the analysis is that a neuron that responded with exactly the same response on short and long trials could appear to be modulated by delay. This is easiest to see with a diagram, but it should be easy to imagine a neural response that quickly rose at the time of instruction and then decayed slowly over the course of 2 seconds. For long trials, the neuron's activity would have returned to baseline, but for short trials, the activity would still be above baseline. As such, it is not clear how much the STN neurons were truly modulated by delay.

    We agree with the reviewers. Our original analyses using two-time windows had the potential to introduce biases in the detection of neuronal activities modulated by the delay. To overcome this issue, we modified the time frame of all of our analyses (neuronal activity, eye position, EMG). Now, the revised version of the manuscript only reports activities across one-time window aligned to the time of instruction cue delivery (i.e., -1 to 3.5s relative to instruction cue onset). This time frame corresponds to the minimum possible interval between instruction cues and reward delivery. We have revised all of the figures and we re-calculated all of the statistics using that one analysis window. Despite these major modifications, our key findings were not changed substantially. We found the same pattern in STN activities, with a strong encoding of reward (48% of neurons) preceding a late encoding of delay (39% of neurons). We also updated the text in Methods and Results sections to reflect the revised analyses.

    • Another concern is the presence of eye movement variables in the regressions that determine whether a neuron is reward or delay encoding. If the task variables modulated eye movements (which would not be surprising) and if the STN activity also modulated eye movements, then, even if task variables did not directly modulate STN activity, the regression would indicate that it did. This is commonly known as "collider bias". This is, unfortunately, a common flaw in neuroscience papers.

    Because the presence of eye variables did not influence how neurons were selected by the GLM, we do not think it likely that our analysis was susceptible to “collider bias”. Nonetheless, to control for that possibility directly, we have now repeated the GLM analyses with eye movement variables excluded. Results are shown in a new figure (Fig.4 – supplementary 1). Exclusion of eye parameters produced results that are very similar to those from the GLM that included eye parameters (differences <3 degrees). We have added text to the manuscript describing this added control analysis.

  3. eLife

    eLife assessment

    This valuable study by Pasquereau and Turner examined the activity neurons in the subthalamic nucleus (STN) while monkeys performed a task in which they had to withhold their response during a delay period whose length was defined by a specific cue. The results indicate that the activity of STN neurons was modulated by reward size and delay. The results are potentially important for understanding how STN regulates behavior such as self-control, but the reviewers thought that the study is incomplete as the analyses, at least in the presented forms, have some potential problems and some analyses require clarification.

  4. eLife

    Reviewer #1 (Public Review):

    Pasquereau and Turner investigated the encoding of reward and delay information in subthalamic (STN) neurons in behaving macaques. They record during a forced-choice task with three levels of reward and two levels of delay, using rejection rates to model subjective value. Task-dependent neurons, those which encoded reward and/or delay, were identified with a sliding-window regression model. They then investigated the time course of reward and delay information using a principal component analysis approach. They find that the strength of the first and four principal components varies systematically along the anteroposterior axis of the STN, suggesting a spatial distribution of value coding. These data, recorded in a controlled task, add to the understanding of STN function.

    The data, collected from a well-defined brain area and with appropriate motor and oculomotor controls included during a straight-forward task, are a good foundation for investigating STN function. However, the statistical procedures used are not completely described and may not be appropriate, particularly in the sliding window analysis. Given this analysis underlies some of the further analyses, it must be clarified or corrected for the conclusions to stand. Further, the analysis only explores the encoding of delay at the time of a cue and does not consider how the value of delay may change over time.

    The sliding window analysis, a common approach in investigating time-course data, necessitates multiple comparisons (188 time-bins here) and so requires a controlling procedure to keep the family-wise error-rate low. The authors describe, not completely, how the pre-instruction period was used to establish the boundaries for significance for each coefficient. The pre-instruction period, by the authors' own account, is a period of lower variance and so it would be expected that the boundaries for significance would be lower and the number of task-dependent neurons is therefore an overestimate. The shuffling process the authors use when they determine significance in their principal components analysis is a more appropriate method.

    The task design and analysis provide a limited test of delay encoding. As only two levels of delay were tested, it is not possible to directly test whether the subjective discounting function is hyperbolic or exponential and hence whether the delay is encoded subjectively or objectively. Further, the task has several variable interval lengths (hold in: 1.2-2.8 s, short delay: 1.8-2.3 s, long delay: 3.5-4s) that frustrate interpretation. The distribution of these delays is not described, for example as it reads it seems possible that some long delay rewards are delivered with shorter latency between cue and reward than some short delay rewards (1.2 + 3.5 = 4.7s vs. 2.8+2.3 = 5.1 s). The authors have not considered that if the delay value is encoding, then the value, both objectively and subjectively, may be changing as the delay elapses. The variation of these task intervals may have an effect on the value of delay.

    The principal components analysis is an interesting way to explore patterns of encoding and the spatial distribution of these patterns. In particular, the finding that Discounting- neurons, those whose firing rate increases with increasing reward cues and decreases with increasing delay cues, are preferentially found in the posterior STN, which the authors demonstrate with both the principal component analysis and the sliding-window classification analysis, challenges previous ideas of STN organization.

  5. eLife

    Reviewer #2 (Public Review):

    The manuscript "Neural dynamics underlying self-control in the primate subthalamic (STN) nucleus" builds on a substantial literature indicating a role for the STN in impulsive actions, i.e. responding too early in tasks that require patience. The authors trained two monkeys to move a cursor to a target and then hold still, waiting for a reward. A visual cue indicated the reward magnitude and time interval that the monkeys were required to wait on each trial in order to get the reward. Understanding the mechanism by which the STN supports behavioral inhibition is important since the STN is a common target for deep brain stimulation for both neurological and psychiatric disorders. The authors claim that their results indicate that the STN integrates reward and delay information and that this representation is anatomically varied along the axis of the STN.

    Plots of "rejection rate" (trials where the monkeys failed to wait until the rewards) as a function of delay and reward size seem to indicate that the monkeys understood the visual cue. The rejection rates were very low (less than 4% for almost all conditions) which indicates that the monkeys did not have a hard time inhibiting their behavior. It also meant that the authors could not compare trials where the monkeys successfully waited with trials where they failed to wait. This missing comparison weakens the link between the neurophysiological observations and the conclusions the authors made about the signals they observed.

    The authors examined the STN activity aligned to the start of the delay and also aligned to the reward. Most of the "delay encoding" in the STN activity was observed near the end of the waiting period. The trouble with the analysis is that a neuron that responded with exactly the same response on short and long trials could appear to be modulated by delay. This is easiest to see with a diagram, but it should be easy to imagine a neural response that quickly rose at the time of instruction and then decayed slowly over the course of 2 seconds. For long trials, the neuron's activity would have returned to baseline, but for short trials, the activity would still be above baseline. As such, it is not clear how much the STN neurons were truly modulated by delay.

    Another concern is the presence of eye movement variables in the regressions that determine whether a neuron is reward or delay encoding. If the task variables modulated eye movements (which would not be surprising) and if the STN activity also modulated eye movements, then, even if task variables did not directly modulate STN activity, the regression would indicate that it did. This is commonly known as "collider bias". This is, unfortunately, a common flaw in neuroscience papers.

    Overall, while the work is potentially interesting, these methodological issues weaken the link between the data and the conclusions of the paper.

  6. eLife

    Reviewer #3 (Public Review):

    The authors have been challenged to figure out the neural processing of delay discounting during waiting for upcoming reward outcomes after behavioral controls in the subthalamic nucleus, where unique brain regions as a part of the basal ganglia for cognitive and motor functions. They described the activity property of STN neurons for the delay gratification at the single neuron level and population level, using both conventional and recently developing approaches. The finding is novel, but the details of the analysis are sometimes inaccurate and needed to be improved. Their claims are now partially supported. If their analyses are improved, their findings have a significant impact on understanding the neural basis of delay discounting, which is one of the predominant behavioral characteristics among organisms.

  7. Version published to 10.1101/2022.10.24.513512v1 on bioRxiv