The impact of task context on predicting finger movements in a brain-machine interface

Article activity feed

1. 

   Version published to 10.7554/elife.82598 on eLife

   Jun 7, 2023
2. 

   eLife

   Mar 16, 2023

   Author Responses

   Reviewer #1 (Public Review):

   This work aimed at investigating how a BMI decoding performance is impacted by changing the conditions under which a motor task is performed. They recorded motor cortical activity using multielectrode arrays in two monkeys executing a finger flexion and extension task in four conditions: normal (no load, neutral wrist position), loaded (manipulandum attached to springs or rubber bands to resist flexion), wrist (no load, flexed wrist position) or both (loaded and flexed wrist). They found, as expected, that BMI decoders trained and tested on data sets collected during the same conditions performed better at predicting kinematics and muscle activity than others trained and tested across conditions. They also report that the performance of monkeys a BMI task involving the online control of a …

   Author Responses

   Reviewer #1 (Public Review):

   This work aimed at investigating how a BMI decoding performance is impacted by changing the conditions under which a motor task is performed. They recorded motor cortical activity using multielectrode arrays in two monkeys executing a finger flexion and extension task in four conditions: normal (no load, neutral wrist position), loaded (manipulandum attached to springs or rubber bands to resist flexion), wrist (no load, flexed wrist position) or both (loaded and flexed wrist). They found, as expected, that BMI decoders trained and tested on data sets collected during the same conditions performed better at predicting kinematics and muscle activity than others trained and tested across conditions. They also report that the performance of monkeys a BMI task involving the online control of a virtual hand was almost unaffected by changing either the actual manipulandum conditions as above or switching between decoders trained from data collected under different conditions. As for the neuronal activity, they found a mix of changes across task contexts. Interestingly, a principal component analysis revealed that activity in each context falls within well-aligned manifolds, and that the context-dependent variance in neuronal activity strongly correlated to the amplitude of muscle activity.

   Strengths

   The current study expands on previous findings about BMI decoders generalizability and contributes scientifically in at least three important ways.

   First, their results are obtained from monkeys performing a fine finger control task with up to two degrees of freedom. This provides a powerful setting to investigate fine motor control of the hand in primates. The authors use the accuracy of BMI decoders between data sets as a measure of stationarity in the neuronsto-fingers mapping, which provides a reliable assessment. They show that changes in wrist angle or finger load affect the relationship between cortical neurons and otherwise identical movements. Interestingly, this result holds up for both kinematics and muscle activity predictions, albeit being stronger for the latter.

   Second, their results confirming that neuronal activity recorded during different task conditions lies effectively within a common manifold is interesting. It supports prior observations, but in the specific context of finger movements.

   Third, the dPCA results provide interesting and perhaps unexpected information about the fact that amplitude of muscle activity (or force) is clearly present in the motor cortical activity. This is possibly one of the most interesting findings because extracting a component from neural activity that can related robustly to muscle activity across context would provide great benefits to the development of BMIs for functional electrical stimulation.

   Overall, the analyses are well designed and the interpretation of the results is sound.

   Weaknesses

   I found the discussion about the possible reasons why offline decoders are more sensitive to context than online decoders very interesting. Nonetheless, as the authors recognize, the possibility that the BMI itself causes a change in context, "in the plant", limits their interpretation. It could mean for the monkeys to switch from one suboptimal decoder to another, causing a ceiling effect occluding generalization errors.

   Overall, several new and original results were obtained through these experiments and analyses. Nonetheless, I found it difficult to extract a clear unique and strong take-home message. The study comes short of proposing a new way to improve BMIs generalizability or precisely identifying factors that influence decoders generalizability.

   We thank the reviewer for the positive comments. Relating these results to BMI design and interpreting the adaptation to contexts during online trials comprised a bulk of the essential revisions from the eLife editorial staff. More details can be found in common response #2 and essential revisions #1-3. To summarize, we added an analysis of neural activity during online trials to provide insight into how the monkeys were adapting. We have expanded the discussion of online adaptation, as detailed in essential revision #2. We also expanded discussion of how both the online and offline results might affect BMI design, as detailed in essential revision #3.

   Reviewer #2 (Public Review):

   The authors motivate this study by the medical need to develop brain-machine interfaces (BMIs) to restore lost arm and hand function, for example through functional electrical stimulation. More specifically, they are interested in developing BMI decoding algorithms that work across a variety of "contexts" that a BMI user would encounter out in the real world, for example having their hand in different postures and manipulating a variety of objects. They note that in different contexts, the motor cortex neural activity patterns that produce the desired muscle outputs may change (including neurons' specific relationship to different muscles' activations), which could render a static decoder trained in a different context inaccurate.

   To test whether this potential challenge is indeed the case, this study tested BMI control of virtual (onscreen) fingers by two rhesus macaques trained to perform 1 or 2 degree-of-freedom non-grasping tasks either by moving their fingers, or just controlling the virtual finger kinematics with neural activity. The key experimental manipulations were context shifts in the form of springs on the fingers or flexion of the wrist (or both). BMI performance was then evaluated when these context changes were present, which builds on this group's previous demonstration of accurate finger BMI without any context shifts.

   The study convincingly shows the aforementioned context shifts do cause large changes in measured firing rates. When neural decoding accuracy (for both muscle and position/velocity) is evaluated across these context changes, reconstruction accuracy is substantially impaired. The headline finding, however, is that that despite this, BMI performance is, on aggregate, not substantially reduced. Although: it is noteworthy that in a second experiment paradigm where the decoder was trained on the spring or wrist-manipulated context and tested in a normal context, there were quite large performance reductions in several datasets as quantified by multiple performance measures; this asymmetry in the results is not really explored much further. The changes in neural activity due to context shifts appear to be relatively modest in magnitude and can be fit well as simple linear shifts (in the neural state space), and the authors posit that this would make it feasible (in future work) to find context-invariant neural readouts that would result in more robust muscle activity decoders.

   An additional novel contribution of this study is showing that these motor cortical signals support quite accurately decode muscle activations during non-prehensile finger movements (and also that the EMG decoding was more negatively affected by context shifts than kinematics decoding); previous work decoded finger kinematics but not these kinetics. Note that this was demonstrated with just one of the two monkeys (the second did not have muscle recordings).

   This is a rigorous study, its main results are well-supported, and it does not make major claims beyond what the data support.

   One of its limitations is that while the eventual motivating goal is to show that decoders are robust across a variety of tasks of daily living, only two specific types of context shifts are tested here, and they are relatively simple and potentially do not result in as strong a neural change as could be encountered in realworld context shifts. This is by no means a major flaw (simplifying experimental preparations are a standard and prudent way to make progress). But the study could point this out a bit more prominently that their results do not preclude that more challenging context shifts will be encountered by BMI users, and this study in its current form does not indicate how strong a perturbation the tested context shifts are relative to the full possible range of hand movement context shifts that would be encountered during human daily living activities.

   A second limitation is that while the discrepancy between large offline decoding performance reduction and small online performance reduction are attributed to rapid sensorimotor adaptation, this process is not directly examined in any detail.

   Third, the assessment of how neural dynamics change in a way that preserves the overall shape of the dynamics is rather qualitative rather than quantitative, and that this implementation of a more contextagnostic finger BMI is left for future work.

   We thank the reviewer for the positive comments. We agree that the paper could discuss how this work impacts a wider range of movements and we now include more discussion to that point as detailed in the responses to feedback below. We also acknowledge that the paper did not directly examine online adaptation and we have now included an analysis aimed at answering how the monkeys adapted to the context changes during online tasks.

   Reviewer #3 (Public Review):

   In this manuscript the authors ask whether finger movements in non-human primates can be predicted from neural activity recorded from the primary motor cortex. This question is driven by an ultimate goal of using neural decoding to create brain-computer interfaces that can restore upper limb function using prosthetics or functional electrical stimulation systems. More specifically, since functional use of the hand (real or prosthetic) will ultimately require generating very different grasp forces for different objects, these experiments use a constant set of finger kinematics, but introduce different force requirements for the finger muscles using several different techniques. Under these different conditions (contexts), the study examines how population neural activity changed and uses decoder analyses to look at how these different contexts affect offline predictions of muscle forces and finger kinematics, as well as the animals' ability to use different decoders to control 1 or 2-DOF online. In general, the study found that when linear models were trained on one context from offline data, they did not generalize well to the other context. However, when performance was tested online (monkeys controlling a virtual hand in real time using neural activity related to movement of their own hands) with a ReFIT Kalman filter, the animals were able to complete the task effectively, even with a decoder trained without the springs or wrist perturbation. The authors show data to support the idea that neural activity was constrained to the same manifold in the different contexts, which enabled the animals to rapidly change their behavior to achieve the task goals, compared to the more complex requirement of having to learn entirely new patterns of neural activity. This work takes studies that have been conducted for upper-limb movements and extends them to include hand grasp, which is important for creating decoders for brain-computer interfaces. Finally, the authors show using dPCA can extract features during changes in context that may be related to the activity of specific muscles that would allow for improved decoders.

   Strengths

   The issue of hand control, and how it compares to arm control, is an important question to tackle in sensorimotor control and in the development of brain-computer interfaces. Interestingly, the experiments use two very different ways of changing the muscle force requirements for achieving the same finger movements; springs attached to a manipulandum and changes in wrist posture. Using both paradigms the decoder analysis clearly shows that linear models trained without any manipulation do not predict muscle forces or finger kinematics well, clearly illustrating the limitations of common linear decoders to generalize to scenarios that might encompass real grasping activities that require forceful interactions. Using a welldescribed real-time decoder (ReFIT Kalman Filter), the authors show that this performance decrease observed offline is easily overcome in online testing. The metrics used to make these claims are welldescribed, and the likely explanations for these findings are described well. A particular strength of this manuscript is that, at least for these relatively simple movements and contexts, a component of neural activity (identified using dPCA) is identified that is significantly modulated by the task context in a way that sensibly represents the changes in muscle activity that would be required to complete the task in the new contexts. We thank the reviewer for the positive comments.

   Weaknesses

   The differences between exemplar data sets and comprehensively tested contexts was difficult to follow. There are many references to how many datasets or trials were used for a particular experiment, but overall, this is fragmented across the manuscript. As a result, it is difficult to assess how generalizable the results of the manuscript were across time or animal, or whether day-to-day variations, or the different data collection schedules had an effect.

   Thank you for the comment, we have added in the number of sessions in results in multiple places throughout the paper. For example, starting line 274 in the results:

   "During these 10 sessions the context changes were tested 15 times: four times for the wrist context, seven times for the spring context, and four times for the combined wrist and spring context."

   The introduction allocates a lot of space to discussing the concepts of generating (computing) movements as opposed to representing movements and relates this to ideas of neural dynamics. The distinction between these as described in the introduction is not very clear, nor is it clear what specific hypothesis this leads to for these experiments. Further, this line of thinking is not returned to in the discussion, so the contribution of these experiments to ideas raised in the introduction are unclear.

   Thank you for the comment, we have written a new paragraph relating these results to the concept of generating movement. Starting line 452 of the discussion:

   "During the offline tasks, many channels changed neural activity with context, with 20.9% to 61.7% of tuned SBP channels modulating activity with context (Table I). The magnitude of these shifts were relatively small, especially when compared to the large changes in required muscle activation (Figure 2D-E), with weak trends to require greater activation for resisted flexion and lesser for assisted extension (Figure 7B-C). Additionally, the neural manifolds underlying movements in each context were well-aligned (Figure 7D). Using dPCA we found that while a large proportion of neural variance was explained by dPCA components that did not change with context, a significant proportion of the neural variance is associated with components that are context-dependent (Figure 8B). Visually, the context components are shifting the trajectories without changing the overall shape and the shift in neural activity is strongly correlated with muscle activations in new contexts (Figure 8C). This agrees with other studies which found lower variance activity may be related to the actual motor commands (Gallego et al., 2018; Russo et al., 2018; Saxena et al., 2022)."

   The complexity of the control that was possible in this task (1 or 2 DOF finger flexion/extension) was low. Further, the manipulations that were used to control context were simple and static. Both these factors likely contribute to the finding that there was little change in the principal angles of the high-variance principal components. While this is not a criticism of the specific results presented here, the simplicity of the task and contexts, contrasted with the complexity of hand control more generally, especially for even moderately dexterous movements, makes it unclear how well the finding of stable manifolds will scale. On a related point, it is unclear whether the feature, identified using dPCA, that could account for changes in muscle activity, could be robustly captured in more realistic behaviors. It is stated that future work is needed, but at this point, the value of identifying this feature is highly speculative.

   Thank you for the comment, we have included more discussion to relate these results to decoder development in general as described in essential revision #3 from the editor.

   The maintained control in online BMI trials could also be explained by another factor, which I don't think was explicitly described by either of the two suggestions. Prism goggle experiments introduce a visual shift can be learned quickly, and some BCI experiments have introduced simple rotations in the decoder output (e.g. Chase et. al. 2012, J Neurophys). This latter case is likely similar in concept to in-manifold perturbations. Regardless, the performance can be rapidly rescued by simply re-aiming, which is a simple behavioral adaptation. In a 1DOF or 2DOF control case like used in these experiments, with constant visual feedback on performance, the change in context could likely be rapidly learned by the animals, maybe even within a single trial. In other words, the high performance in the online case may be a consequence of the relatively simple task demands, and the simple biomechanical solution to this problem (push harder). What is the expectation that the results seen in these experiments would be relevant to more realistic situations that require grasp and interaction?

   Thank you for the suggestion, we agree that the quick adaptation is likely related to re-aiming. To this end, we have included a re-aiming analysis, as described in essential revisions #1 and #2 from the editor and common response #2, to look into the quick adjustment.

   Some of the figures were difficult to read and the captions contained some minor incorrect information. The primary purpose of some of the figures was not immediately clear from the caption. For example, the bar plots in Figures 5 and 6 were very small and difficult to read. This also made distinguishing the data from the two different animals challenging.

   Thank you for the comments, multiple figures have been edited to increase legibility and a review of text has been done to fix errors and improve interpretability.

   There is no specific quantification of the data in Figures 4D and 5D. In Figure 4D it seems apparent that the vast majority of the points are below the unity line. But, it remains unclear, particularly in Figure 5D whether the correlations between the two contexts truly are different or not in a way that would allow conclusive statements.

   Thank you for the comments, Figure 4D has been moved to the supplement and 5D has now been replaced by figures analyzing the neural activity patterns during the online task.

   Read the original source
3. 

   eLife

   Dec 8, 2022

   eLife assessment

   This study provides valuable findings about how brain machine interfaces cope with changes in context, an important consideration for deploying such devices in the real world. The evidence supporting the claims is solid although increasing the number and range of contexts investigated would have strengthened the study. The work will be of interest to motor neuroscientists and engineers developing brain machine interfaces and will be useful for future development of such devices.

   Read the original source
4. 

   eLife

   Dec 8, 2022

   Reviewer #1 (Public Review):

   This work aimed at investigating how a BMI decoding performance is impacted by changing the conditions under which a motor task is performed. They recorded motor cortical activity using multielectrode arrays in two monkeys executing a finger flexion and extension task in four conditions: normal (no load, neutral wrist position), loaded (manipulandum attached to springs or rubber bands to resist flexion), wrist (no load, flexed wrist position) or both (loaded and flexed wrist). They found, as expected, that BMI decoders trained and tested on data sets collected during the same conditions performed better at predicting kinematics and muscle activity than others trained and tested across conditions. They also report that the performance of monkeys a BMI task involving the online control of a virtual hand was …

   Reviewer #1 (Public Review):

   This work aimed at investigating how a BMI decoding performance is impacted by changing the conditions under which a motor task is performed. They recorded motor cortical activity using multielectrode arrays in two monkeys executing a finger flexion and extension task in four conditions: normal (no load, neutral wrist position), loaded (manipulandum attached to springs or rubber bands to resist flexion), wrist (no load, flexed wrist position) or both (loaded and flexed wrist). They found, as expected, that BMI decoders trained and tested on data sets collected during the same conditions performed better at predicting kinematics and muscle activity than others trained and tested across conditions. They also report that the performance of monkeys a BMI task involving the online control of a virtual hand was almost unaffected by changing either the actual manipulandum conditions as above or switching between decoders trained from data collected under different conditions. As for the neuronal activity, they found a mix of changes across task contexts. Interestingly, a principal component analysis revealed that activity in each context falls within well-aligned manifolds, and that the context-dependent variance in neuronal activity strongly correlated to amplitude of muscle activity.

   Strengths:

   The current study expands on previous findings about BMI decoders generalizability and contributes scientifically in at least three important ways.

   First, their results are obtained from monkeys performing a fine finger control task with up to two degrees of freedom. This provides a powerful setting to investigate fine motor control of the hand in primates. The authors use the accuracy of BMI decoders between data sets as a measure of stationarity in the neurons-to-fingers mapping, which provides a reliable assessment. They show that changes in wrist angle or finger load affect the relationship between cortical neurons and otherwise identical movements. Interestingly, this result hold up for both kinematics and muscle activity predictions, albeit being stronger for the latter.

   Second, their results confirming that neuronal activity recorded during different task conditions lies effectively within a common manifold is interesting. It supports prior observations, but in the specific context of finger movements.

   Third, the dPCA results provide interesting and perhaps unexpected information about the fact that amplitude of muscle activity (or force) is clearly present in the motor cortical activity. This is possibly one of the most interesting findings because extracting a component from neural activity that can related robustly to muscle activity across context would provide great benefits to the development of BMIs for functional electrical stimulation.
   Overall, the analyses are well designed and the interpretation of the results is sound.

   Weaknesses:

   I found the discussion about the possible reasons why offline decoders are more sensitive to context than online decoders very interesting. Nonetheless, as the authors recognize, the possibility that the BMI itself causes a change in context, "in the plant", limits their interpretation. It could mean for the monkeys to switch from one suboptimal decoder to another, causing a ceiling effect occluding generalization errors.

   Overall, several new and original results were obtained through these experiments and analyses. Nonetheless, I found it difficult to extract a clear unique and strong take-home message. The study comes short of proposing a new way to improve BMIs generalizability or precisely identifying factors that influence decoders generalizability.

   Read the original source
5. 

   eLife

   Dec 8, 2022

   Reviewer #2 (Public Review):

   The authors motivate this study by the medical need to develop brain-machine interfaces (BMIs) to restore lost arm and hand function, for example through functional electrical stimulation. More specifically, they are interested in developing BMI decoding algorithms that work across a variety of "contexts" that a BMI user would encounter out in the real world, for example having their hand in different postures and manipulating a variety of objects. They note that in different contexts, the motor cortex neural activity patterns that produce the desired muscle outputs may change (including neurons' specific relationship to different muscles' activations), which could render a static decoder trained in a different context inaccurate.

   To test whether this potential challenge is indeed the case, this study tested …

   Reviewer #2 (Public Review):

   The authors motivate this study by the medical need to develop brain-machine interfaces (BMIs) to restore lost arm and hand function, for example through functional electrical stimulation. More specifically, they are interested in developing BMI decoding algorithms that work across a variety of "contexts" that a BMI user would encounter out in the real world, for example having their hand in different postures and manipulating a variety of objects. They note that in different contexts, the motor cortex neural activity patterns that produce the desired muscle outputs may change (including neurons' specific relationship to different muscles' activations), which could render a static decoder trained in a different context inaccurate.

   To test whether this potential challenge is indeed the case, this study tested BMI control of virtual (on-screen) fingers by two rhesus macaques trained to perform 1 or 2 degree-of-freedom non-grasping tasks either by moving their fingers, or just controlling the virtual finger kinematics with neural activity. The key experimental manipulations were context shifts in the form of springs on the fingers or flexion of the wrist (or both). BMI performance was then evaluated when these context changes were present, which builds on this group's previous demonstration of accurate finger BMI without any context shifts.

   The study convincingly shows the aforementioned context shifts do cause large changes in measured firing rates. When neural decoding accuracy (for both muscle and position/velocity) is evaluated across these context changes, reconstruction accuracy is substantially impaired. The headline finding, however, is that that despite this, BMI performance is, on aggregate, not substantially reduced. Although: it is noteworthy that in a second experiment paradigm where the decoder was trained on the spring or wrist-manipulated context and tested in a normal context, there were quite large performance reductions in several datasets as quantified by multiple performance measures; this asymmetry in the results is not really explored much further.

   The changes in neural activity due to context shifts appear to be relatively modest in magnitude and can be fit well as simple linear shifts (in the neural state space), and the authors posit that this would make it feasible (in future work) to find context-invariant neural readouts that would result in more robust muscle activity decoders.

   An additional novel contribution of this study is showing that these motor cortical signals support quite accurately decode muscle activations during non-prehensile finger movements (and also that the EMG decoding was more negatively affected by context shifts than kinematics decoding); previous work decoded finger kinematics but not these kinetics. Note that this was demonstrated with just one of the two monkeys (the second did not have muscle recordings).

   This is a rigorous study, its main results are well-supported, and it does not make major claims beyond what the data support. One of its limitations is that while the eventual motivating goal is to show that decoders are robust across a variety of tasks of daily living, only two specific types of context shifts are tested here, and they are relatively simple and potentially do not result in as strong a neural change as could be encountered in real-world context shifts. This is by no means a major flaw (simplifying experimental preparations are a standard and prudent way to make progress). But the study could point this out a bit more prominently that their results do not preclude that more challenging context shifts will be encountered by BMI users, and this study in its current form does not indicate how strong a perturbation the tested context shifts are relative to the full possible range of hand movement context shifts that would be encountered during human daily living activities.

   A second limitation is that while the discrepancy between large offline decoding performance reduction and small online performance reduction are attributed to rapid sensorimotor adaptation, this process is not directly examined in any detail. Third, the assessment of how neural dynamics change in a way that preserves the overall shape of the dynamics is rather qualitative rather than quantitative, and that this implementation of a more context-agnostic finger BMI is left for future work.

   Read the original source
6. 

   eLife

   Dec 8, 2022

   Reviewer #3 (Public Review):

   In this manuscript the authors ask whether finger movements in non-human primates can be predicted from neural activity recorded from the primary motor cortex. This question is driven by an ultimate goal of using neural decoding to create brain-computer interfaces that can restore upper limb function using prosthetics or functional electrical stimulation systems. More specifically, since functional use of the hand (real or prosthetic) will ultimately require generating very different grasp forces for different objects, these experiments use a constant set of finger kinematics, but introduce different force requirements for the finger muscles using several different techniques. Under these different conditions (contexts), the study examines how population neural activity changed and uses decoder analyses to …

   Reviewer #3 (Public Review):

   In this manuscript the authors ask whether finger movements in non-human primates can be predicted from neural activity recorded from the primary motor cortex. This question is driven by an ultimate goal of using neural decoding to create brain-computer interfaces that can restore upper limb function using prosthetics or functional electrical stimulation systems. More specifically, since functional use of the hand (real or prosthetic) will ultimately require generating very different grasp forces for different objects, these experiments use a constant set of finger kinematics, but introduce different force requirements for the finger muscles using several different techniques. Under these different conditions (contexts), the study examines how population neural activity changed and uses decoder analyses to look at how these different contexts affect offline predictions of muscle forces and finger kinematics, as well as the animals' ability to use different decoders to control 1 or 2-DOF online. In general, the study found that when linear models were trained on one context from offline data, they did not generalize well to the other context. However, when performance was tested online (monkeys controlling a virtual hand in real time using neural activity related to movement of their own hands) with a ReFIT Kalman filter, the animals were able to complete the task effectively, even with a decoder trained without the springs or wrist perturbation. The authors show data to support the idea that neural activity was constrained to the same manifold in the different contexts, which enabled the animals to rapidly change their behavior to achieve the task goals, compared to the more complex requirement of having to learn entirely new patterns of neural activity. This work takes studies that have been conducted for upper-limb movements and extends them to include hand grasp, which is important for creating decoders for brain-computer interfaces. Finally, the authors show using dPCA can extract features during changes in context that may be related to the activity of specific muscles that would allow for improved decoders.

   Strengths:

   The issue of hand control, and how it compares to arm control, is an important question to tackle in sensorimotor control and in the development of brain-computer interfaces. Interestingly, the experiments use two very different ways of changing the muscle force requirements for achieving the same finger movements; springs attached to a manipulandum and changes in wrist posture. Using both paradigms the decoder analysis clearly shows that linear models trained without any manipulation do not predict muscle forces or finger kinematics well, clearly illustrating the limitations of common linear decoders to generalize to scenarios that might encompass real grasping activities that require forceful interactions. Using a well-described real-time decoder (ReFIT Kalman Filter), the authors show that this performance decrease observed offline is easily overcome in online testing. The metrics used to make these claims are well-described, and the likely explanations for these findings are described well. A particular strength of this manuscript is that, at least for these relatively simple movements and contexts, a component of neural activity (identified using dPCA) is identified that is significantly modulated by the task context in a way that sensibly represents the changes in muscle activity that would be required to complete the task in the new contexts.

   Weaknesses:

   The differences between exemplar data sets and comprehensively tested contexts was difficult to follow. There are many references to how many datasets or trials were used for a particular experiment, but overall, this is fragmented across the manuscript. As a result, it is difficult to assess how generalizable the results of the manuscript were across time or animal, or whether day-to-day variations, or the different data collection schedules had an effect.

   The introduction allocates a lot of space to discussing the concepts of generating (computing) movements as opposed to representing movements and relates this to ideas of neural dynamics. The distinction between these as described in the introduction is not very clear, nor is it clear what specific hypothesis this leads to for these experiments. Further, this line of thinking is not returned to in the discussion, so the contribution of these experiments to ideas raised in the introduction are unclear.

   The complexity of the control that was possible in this task (1 or 2 DOF finger flexion/extension) was low. Further, the manipulations that were used to control context were simple and static. Both these factors likely contribute to the finding that there was little change in the principal angles of the high-variance principal components. While this is not a criticism of the specific results presented here, the simplicity of the task and contexts, contrasted with the complexity of hand control more generally, especially for even moderately dexterous movements, makes it unclear how well the finding of stable manifolds will scale. On a related point, it is unclear whether the feature, identified using dPCA, that could account for changes in muscle activity, could be robustly captured in more realistic behaviors. It is stated that future work is needed, but at this point, the value of identifying this feature is highly speculative.

   The maintained control in online BMI trials could also be explained by another factor, which I don't think was explicitly described by either of the two suggestions. Prism goggle experiments introduce a visual shift can be learned quickly, and some BCI experiments have introduced simple rotations in the decoder output (e.g. Chase et. al. 2012, J Neurophys). This latter case is likely similar in concept to in-manifold perturbations. Regardless, the performance can be rapidly rescued by simply re-aiming, which is a simple behavioral adaptation. In a 1DOF or 2DOF control case like used in these experiments, with constant visual feedback on performance, the change in context could likely be rapidly learned by the animals, maybe even within a single trial. In other words, the high performance in the online case may be a consequence of the relatively simple task demands, and the simple biomechanical solution to this problem (push harder). What is the expectation that the results seen in these experiments would be relevant to more realistic situations that require grasp and interaction?

   Some of the figures were difficult to read and the captions contained some minor incorrect information. The primary purpose of some of the figures was not immediately clear from the caption. For example, the bar plots in Figures 5 and 6 were very small and difficult to read. This also made distinguishing the data from the two different animals challenging.

   There is no specific quantification of the data in Figures 4D and 5D. In Figure 4D it seems apparent that the vast majority of the points are below the unity line. But, it remains unclear, particularly in Figure 5D whether the correlations between the two contexts truly are different or not in a way that would allow conclusive statements.

   Read the original source
7. 

   Version published to 10.1101/2022.08.26.505422v1 on bioRxiv

   Aug 26, 2022