Dissecting muscle synergies in the task space

  1. David O'Reilly
  2. Ioannis Delis

  • Curated by eLife

    eLife logo

    eLife assessment

    The work by O'Reilly and Delis is important to extend the synergy ideas using methods from signal processing and information theory to cluster muscles and task parameters, thereby advancing our understanding of the modular architecture of motor control. The method is innovative, and the findings are compelling from theoretical and practical perspectives. The work will be of broad interest to motor control and neural engineering researchers.

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The muscle synergy is a guiding concept in motor control research that relies on the general notion of muscles ‘ working together ’ towards task performance. However, although the synergy concept has provided valuable insights into motor coordination, muscle interactions have not been fully characterised with respect to task performance. Here, we address this research gap by proposing a novel perspective to the muscle synergy that assigns specific functional roles to muscle couplings by characterising their task-relevance. Our novel perspective provides nuance to the muscle synergy concept, demonstrating how muscular interactions can ‘ work together ’ in different ways: (1) irrespective of the task at hand but also (2) redundantly or (3) complementarily towards common task-goals. To establish this perspective, we leverage information- and network-theory and dimensionality reduction methods to include discrete and continuous task parameters directly during muscle synergy extraction. Specifically, we introduce co-information as a measure of the task-relevance of muscle interactions and use it to categorise such interactions as task-irrelevant (present across tasks), redundant (shared task information), or synergistic (different task information). To demonstrate these types of interactions in real data, we firstly apply the framework in a simple way, revealing its added functional and physiological relevance with respect to current approaches. We then apply the framework to large-scale datasets and extract generalizable and scale-invariant representations consisting of subnetworks of synchronised muscle couplings and distinct temporal patterns. The representations effectively capture the functional interplay between task end-goals and biomechanical affordances and the concurrent processing of functionally similar and complementary task information. The proposed framework unifies the capabilities of current approaches in capturing distinct motor features while providing novel insights and research opportunities through a nuanced perspective to the muscle synergy.

Article activity feed

  1. Version published to 10.7554/elife.87651.4 on eLife
  2. Version published to 10.7554/elife.87651 on eLife
  3. Version published to 10.7554/elife.87651.3 on eLife
  4. eLife

    Author Response

    The following is the authors’ response to the previous reviews.

    Public Reviews:

    Reviewer #1 (Public Review):

    The proposed study provides an innovative framework for the identification of muscle synergies taking into account their task relevance. State-of-the-art techniques for extracting muscle interactions use unsupervised machine-learning algorithms applied to the envelopes of the electromyographic signals without taking into account the information related to the task being performed. In this work, the authors suggest including the task parameters in extracting muscle synergies using a network information framework previously proposed. This allows the identification of muscle interactions that are relevant, irrelevant, or redundant to the parameters of the task executed.

    The proposed framework is a powerful tool to understand and identify muscle interactions for specific task parameters and it may be used to improve man-machine interfaces for the control of prostheses and robotic exoskeletons.

    With respect to the network information framework recently published, this work added an important part to estimate the relevance of specific muscle interactions to the parameters of the task executed. However, the authors should better explain what is the added value of this contribution with respect to the previous one, also in terms of computational methods.

    It is not clear how the well-known phenomenon of cross-talk during the recording of electromyographic muscle activity may affect the performance of the proposed technique and how it may bias the overall outcomes of the framework.

    We thank reviewer 1 for their useful commentary on this manuscript.

    Reviewer #2 (Public Review):

    This paper is an attempt to extend or augment muscle synergy and motor primitive ideas with task measures. The authors idea is to use information metrics (mutual information, co-information) in 'synergy' constraint creation that includes task information directly. By using task related information and muscle information sources and then sparsification, the methods construct task relevant network communities among muscles, together with task redundant communities, and task irrelevant communities. This process of creating network communities may then constrain and help to guide subsequent synergy identification using the authors published sNM3F algorithm to detect spatial and temporal synergies.

    The revised paper is much clearer and examples are helpful in various ways. However, figure 2 as presented does not convincingly show why task muscle mutual information helps in separating synergies, though it is helpful in defining the various network communities used in the toy example.

    The impact of the information theoretic constraints developed as network communities on subsequent synergy separation are posited to be benign and to improve over other methods (e.g., NNMF). However, not fully addressed are the possible impacts of the methods on compositionality links with physiological bases, and the possibility remains of the methods sometimes instead leading to modules that represent more descriptive ML frameworks that may not support physiological work easily. Accordingly, there is a caveat. This is recognized and acknowledged by the authors in their rebuttal of the prior review. It will remain for other work to explore this issue, likely through testing on detailed high degree of freedom artificial neuromechanical models and tasks. This possible issue with the strategy here likely needs to be fully acknowledged in the paper.

    The approach of the methods seeks to identify task relevant coordinative couplings. This is a meta problem for more classical synergy analyses. Classical analyses seek compositional elements stable across tasks. These elements may then be explored in causal experiments and generative simulations of coupling and control strategies. However, task-based understanding of synergy roles and functional uses is significant and is clearly likely to be aided by methods in this study.

    Information based separation has been used in muscle synergy analyses using infomax ICA, which is information based at core. Though linear mixing of sources is assumed in ICA, minimized mutual information among source (synergy) drives is the basis of the separation and detects low variance synergy contributions (e.g., see Yang, Logan, Giszter, 2019). In the work in this paper, instead, mutual information approaches are used to cluster muscles and task features into network communities preceding the SNM3F algorithm use for separation, rather than using minimized information in separation. This contrast of an accretive or agglomerative mutual information strategy here used to cluster into networks, versus a minimizing mutual information source separation used in infomax ICA epitomizes a key difference in approach here.

    Physiological causal testing of synergy ideas is neglected in the literature reviews in the paper. Although these are only in animal work (Hart and Giszter, 2010; Takei and Seki, 2017), the clear connection of muscle synergy analysis choices to physiology is important, and eventually these issues need to be better managed and understood in relation to the new methods proposed here, even if not in this paper.

    Analyses of synergies using the methods the paper has proposed will likely be very much dependent on the number and quality of task variables included and how these are managed, and the impacts of these on the ensuing sparsification and network communities used prior to SNM3F. The authors acknowledge this in their response. This caveat should likely be made very explicit in the paper.

    It would be useful in the future to explore the approach described with a range of simulated data to better understand the caveats, and optimizations for best practices in this approach.

    A key component of the reviewers’ arguments here is their reductionist view of muscle synergies vs the emergentist view presented in our work here. In the reductionist lens, muscle groupings are the units (‘building blocks’) of coordinated movement and thus the space of intermuscular interactions is of particular interest for understanding movement construction. On the other hand, the emergentist view suggests that muscle groupings emerge from interactions between constituent parts (as quantified here using information theory, synergistic information is the information found when both activities are observed together). This is in line with recent work in the field showing modular control at the intramuscular level, exemplifying a scale-free phenomena. Nonetheless, we consider these approaches to muscle synergy research as complementary and beneficial for the field overall going forward.

    Reviewer #3 (Public Review):

    In this study, the authors developed and tested a novel framework for extracting muscle synergies. The approach aims at removing some limitations and constraints typical of previous approaches used in the field. In particular, the authors propose a mathematical formulation that removes constraints of linearity and couples the synergies to their motor outcome, supporting the concept of functional synergies and distinguishing the task-related performance related to each synergy. While some concepts behind this work were already introduced in recent work in the field, the methodology provided here encapsulates all these features in an original formulation providing a step forward with respect to the currently available algorithms. The authors also successfully demonstrated the applicability of their method to previously available datasets of multi-joint movements.

    Preliminary results positively support the scientific soundness of the presented approach and its potential. The added values of the method should be documented more in future work to understand how the presented formulation relates to previous approaches and what novel insights can be achieved in practical scenarios and confirm/exploit the potential of the theoretical findings.

    In their revision, the authors have implemented major revisions and improved their paper. The work was already of good quality and now it has improved further. The authors were able to successfully:

    • improve the clarity of the writing (e.g.: better explaining the rationale and the aims of the paper);
    • extend the clarification of some of the key novel concepts introduced in their work, like the redundant synergies;
    • show a scenario in which their approach might be useful for increasing the understanding of motor control in patients with respect to traditional algorithms such as NMF. In particular, their example illustrates why considering the task space is a fundamental step forward when extracting muscle synergies, improving the practical and physiological interpretation of the results.

    We thank reviewer 3 for their constructive commentary on this manuscript.

    Recommendations for the authors:

    Reviewer #1 (Recommendations For The Authors):

    Figure 3 should report the distances between reaching points in panel A and the actual length distances of the walking paths in panel C.

    The caption of fig.3 concerning the experimental setup of the datasets analysed has been updated with the following for dataset 1: “(A) Dataset 1 consisted of participants executing table-top point-to-point reaching movements (40cm distance from starting point P0) across four targets in forward (P1-P4) and backwards (P5-P8) directions at both fast and slow speeds (40 repetitions per task) [25]. The muscles recorded included the finger extensors (FE), brachioradialis (BR), biceps brachii (BI), medial-triceps (TM), lateral-triceps (TL), anterior deltoid (AD), posterior deltoid (PD), pectoralis major (PE), latissimus dorsi (LD) of the right, reaching arm.”. For dataset 3, to the best of the authors knowledge, this information was not given in the original paper.

    Figure 4, what is the unit of the data shown?

    The unit of bits is now mentioned in the toy example figure caption and in the caption of fig.5

    Figure 4, the characteristics of the interactions are not fully clear, and the graphical representation should be improved.

    We have made steps to improve the clarity of the figures presented.

    For dataset 3, τ was the movement kinematics, but it is not specified how the task parameters were formulated. Did the authors use the data from all 32 kinematic markers, 4 IMUs, and force plates? If yes, it should be specified why all these signals were used. For sure, there will be signals included that are not relevant to the specific task. Did the authors select specific signals based on their relevance to the task (e.g., ankle kinematics)?

    We have now clarified this in the text as follows: “For datasets 1 and 2, we determine the MI between vectors with respect to several discrete task parameters representing specific task attributes (e.g. reaching direction, speed etc.), while for dataset 3 we determined the task-relevant and -irrelevant muscles couplings in an unassuming way by quantifying them with respect to all available kinematic, dynamic and inertial motion unit (IMU) features.”

    How did the authors endure that crosstalk did not affect their analysis, particularly between, e.g., finger extensors and brachioradialis and posterior deltoid and anterior deltoid (dataset 1)?

    We have addressed this point in the previous round of reviews and made an explicit statement regarding cross-talk in the discussion section: “Although distinguishing task-irrelevant muscle couplings may capture artifacts such as EMG crosstalk, our results convey several physiological objectives of muscles including gross motor functions [66], the maintenance of internal joint mechanics and reciprocal inhibition of contralateral limbs [19,51].”

    It would be informative to add some examples of not trivial/obvious task-related synergistic muscle combinations that have been extracted in the three datasets. Most of the examples reported in the manuscript are well-known biomechanically and quite intuitive, so they do not improve our understanding of synergistic muscle control in humans.

    Our framework improves our understanding of synergistic motor control by enabling the formal quantification of synergistic muscle interactions, a capability not present among current approaches. Regarding the implications of this advance in terms of concrete examples, we have further clarified our examples presented in the results section, for example:

    “Across datasets, many the muscle networks could be characterised by the transmission of complementary task information between functionally specialised muscle groups, many of which identified among the task-redundant representations (Fig.9-10 and Supp. Fig.2). The most obvious example of this is the S3 synergist muscle network of dataset 2 (Fig.11), which captures the complementary interaction between task-redundant submodules identified previously (S3 (Fig.9)).”

    The description shows how our framework can extract the cross-module interactions that align with the higher-level objectives of the system, here the synergistic connectivity between the upper and lower body modules. Current approaches can only capture redundant and task-irrelevant interactions. Thus our framework provides additional insight into movement control.

    The number of participations in dataset 2 is very limited and should be increased. We appreciate the reviewer's comment and would like to point out that for dataset 2 our aim was to increase the number of muscles (30), tasks (72) and trials for each task (30) which produced a very large dataset for each participant. This came at the expense of low number of participants, however all our statistical analyses here can be performed at the single-participant level. Furthermore, dataset 3 includes 25 participants and it enables us to demonstrate the reliability of the findings across participants.

    Reviewer #2 (Recommendations For The Authors):

    I believe it is important in the future to explore the approach proposed with a range of simulation data and neuromechanical models, to explore the issues I have raised and that you have acknowledged, though I agree it is likely out of scope for the paper here.

    We agree with the reviewer that this would be valuable future work and indeed plan to do this in our future research.

    The Github code for this paper should likely include the various data sets used in the paper and figures, appropriately anonymized, in order to allow the data to be explored and analyses replicated and package demonstrated to be exercised fully by a new user.

    We thank the reviewer for this suggestion. Dataset3 is already available online at https://doi.org/10.1016/j.jbiomech.2021.110320. We will also make the other 2 datasets publicly available on our lab website very soon. Until then, as stated in the manuscript, we will make them available to anyone upon reasonable request.

    Reviewer #3 (Recommendations For The Authors):

    I have the following open points to suggest to the authors:

    First, I recommend improving the quality of the figures: in the pdf version I downloaded, some writings are impossible to read.

    We fully agree with the reviewer and note that in the pdf version of the paper, the figures are a lot worse than in the submitted word document submitted. Nevertheless, we will make further improvements on the figures as requested.

    Even though the manuscript has improved, I still feel that some points were not addressed or were only partially addressed. In particular:

    • The proposed comparison with NMF helps understanding why incorporating the task space is useful (and I fully agree with the authors about this point as the main reason to propose their contribution). However, the comparison does not help the reader to understand whether the synergies incorporating the task space are biased by the introduction of the task variables.

    This question can be also reformulated as: are muscle synergies modified when task space variables are incorporated? Is the "weight" on task coefficients affecting the composition of muscle synergies? If so, the added interpretational power is achieved at the cost of losing the information regarding the neural substrate of synergies? I understand this point is not immediate to show, but it would increase the quality of the work.

    • Reference to previous approaches that aimed at including task variables into synergy extraction are still missing in the paper. Even though it is not required to provide quantitative comparisons with other available approaches, there are at most 2-3 available algorithms in the literature (kinematics-EMG; force-EMG), that should not be neglected in this work. What did previous approaches achieve? What was improved with this approach? What was not improved?

    Previous attempts of extracting synergies with non-linear approaches could also be described more.

    In the latest version of the manuscript, we have referenced both the mixed NMF and autoencoders based algorithms. In both the introduction and discussion section of the manuscript, we also specify that our framework quantifies and decomposes muscle interactions in a novel way that cannot be done by other current approaches. In the results section we use examples from 3 different datasets to make this point clear, providing intuition on the use cases of our framework.

  5. eLife

    Author Response

    The following is the authors’ response to the previous reviews.

    Public Reviews:

    Reviewer #1 (Public Review):

    The proposed study provides an innovative framework for the identification of muscle synergies taking into account their task relevance. State-of-the-art techniques for extracting muscle interactions use unsupervised machine-learning algorithms applied to the envelopes of the electromyographic signals without taking into account the information related to the task being performed. In this work, the authors suggest including the task parameters in extracting muscle synergies using a network information framework previously proposed. This allows the identification of muscle interactions that are relevant, irrelevant, or redundant to the parameters of the task executed.

    The proposed framework is a powerful tool to understand and identify muscle interactions for specific task parameters and it may be used to improve man-machine interfaces for the control of prostheses and robotic exoskeletons.

    With respect to the network information framework recently published, this work added an important part to estimate the relevance of specific muscle interactions to the parameters of the task executed. However, the authors should better explain what is the added value of this contribution with respect to the previous one, also in terms of computational methods.

    It is not clear how the well-known phenomenon of cross-talk during the recording of electromyographic muscle activity may affect the performance of the proposed technique and how it may bias the overall outcomes of the framework.

    We thank reviewer 1 for their useful commentary on this manuscript.

    Reviewer #2 (Public Review):

    This paper is an attempt to extend or augment muscle synergy and motor primitive ideas with task measures. The authors idea is to use information metrics (mutual information, co-information) in 'synergy' constraint creation that includes task information directly. By using task related information and muscle information sources and then sparsification, the methods construct task relevant network communities among muscles, together with task redundant communities, and task irrelevant communities. This process of creating network communities may then constrain and help to guide subsequent synergy identification using the authors published sNM3F algorithm to detect spatial and temporal synergies.

    The revised paper is much clearer and examples are helpful in various ways. However, figure 2 as presented does not convincingly show why task muscle mutual information helps in separating synergies, though it is helpful in defining the various network communities used in the toy example.

    The impact of the information theoretic constraints developed as network communities on subsequent synergy separation are posited to be benign and to improve over other methods (e.g., NNMF). However, not fully addressed are the possible impacts of the methods on compositionality links with physiological bases, and the possibility remains of the methods sometimes instead leading to modules that represent more descriptive ML frameworks that may not support physiological work easily. Accordingly, there is a caveat. This is recognized and acknowledged by the authors in their rebuttal of the prior review. It will remain for other work to explore this issue, likely through testing on detailed high degree of freedom artificial neuromechanical models and tasks. This possible issue with the strategy here likely needs to be fully acknowledged in the paper.

    The approach of the methods seeks to identify task relevant coordinative couplings. This is a meta problem for more classical synergy analyses. Classical analyses seek compositional elements stable across tasks. These elements may then be explored in causal experiments and generative simulations of coupling and control strategies. However, task-based understanding of synergy roles and functional uses is significant and is clearly likely to be aided by methods in this study.

    Information based separation has been used in muscle synergy analyses using infomax ICA, which is information based at core. Though linear mixing of sources is assumed in ICA, minimized mutual information among source (synergy) drives is the basis of the separation and detects low variance synergy contributions (e.g., see Yang, Logan, Giszter, 2019). In the work in this paper, instead, mutual information approaches are used to cluster muscles and task features into network communities preceding the SNM3F algorithm use for separation, rather than using minimized information in separation. This contrast of an accretive or agglomerative mutual information strategy here used to cluster into networks, versus a minimizing mutual information source separation used in infomax ICA epitomizes a key difference in approach here.

    Physiological causal testing of synergy ideas is neglected in the literature reviews in the paper. Although these are only in animal work (Hart and Giszter, 2010; Takei and Seki, 2017), the clear connection of muscle synergy analysis choices to physiology is important, and eventually these issues need to be better managed and understood in relation to the new methods proposed here, even if not in this paper.

    Analyses of synergies using the methods the paper has proposed will likely be very much dependent on the number and quality of task variables included and how these are managed, and the impacts of these on the ensuing sparsification and network communities used prior to SNM3F. The authors acknowledge this in their response. This caveat should likely be made very explicit in the paper.

    It would be useful in the future to explore the approach described with a range of simulated data to better understand the caveats, and optimizations for best practices in this approach.

    A key component of the reviewers’ arguments here is their reductionist view of muscle synergies vs the emergentist view presented in our work here. In the reductionist lens, muscle groupings are the units (‘building blocks’) of coordinated movement and thus the space of intermuscular interactions is of particular interest for understanding movement construction. On the other hand, the emergentist view suggests that muscle groupings emerge from interactions between constituent parts (as quantified here using information theory, synergistic information is the information found when both activities are observed together). This is in line with recent work in the field showing modular control at the intramuscular level, exemplifying a scale-free phenomena. Nonetheless, we consider these approaches to muscle synergy research as complementary and beneficial for the field overall going forward.

    Reviewer #3 (Public Review):

    In this study, the authors developed and tested a novel framework for extracting muscle synergies. The approach aims at removing some limitations and constraints typical of previous approaches used in the field. In particular, the authors propose a mathematical formulation that removes constraints of linearity and couples the synergies to their motor outcome, supporting the concept of functional synergies and distinguishing the task-related performance related to each synergy. While some concepts behind this work were already introduced in recent work in the field, the methodology provided here encapsulates all these features in an original formulation providing a step forward with respect to the currently available algorithms. The authors also successfully demonstrated the applicability of their method to previously available datasets of multi-joint movements.

    Preliminary results positively support the scientific soundness of the presented approach and its potential. The added values of the method should be documented more in future work to understand how the presented formulation relates to previous approaches and what novel insights can be achieved in practical scenarios and confirm/exploit the potential of the theoretical findings.

    In their revision, the authors have implemented major revisions and improved their paper. The work was already of good quality and now it has improved further. The authors were able to successfully:

    • improve the clarity of the writing (e.g.: better explaining the rationale and the aims of the paper);
    • extend the clarification of some of the key novel concepts introduced in their work, like the redundant synergies;
    • show a scenario in which their approach might be useful for increasing the understanding of motor control in patients with respect to traditional algorithms such as NMF. In particular, their example illustrates why considering the task space is a fundamental step forward when extracting muscle synergies, improving the practical and physiological interpretation of the results.

    We thank reviewer 3 for their constructive commentary on this manuscript.

    Recommendations for the authors:

    Reviewer #1 (Recommendations For The Authors):

    Figure 3 should report the distances between reaching points in panel A and the actual length distances of the walking paths in panel C.

    The caption of fig.3 concerning the experimental setup of the datasets analysed has been updated with the following for dataset 1: “(A) Dataset 1 consisted of participants executing table-top point-to-point reaching movements (40cm distance from starting point P0) across four targets in forward (P1-P4) and backwards (P5-P8) directions at both fast and slow speeds (40 repetitions per task) [25]. The muscles recorded included the finger extensors (FE), brachioradialis (BR), biceps brachii (BI), medial-triceps (TM), lateral-triceps (TL), anterior deltoid (AD), posterior deltoid (PD), pectoralis major (PE), latissimus dorsi (LD) of the right, reaching arm.”. For dataset 3, to the best of the authors knowledge, this information was not given in the original paper.

    Figure 4, what is the unit of the data shown?

    The unit of bits is now mentioned in the toy example figure caption and in the caption of fig.5

    Figure 4, the characteristics of the interactions are not fully clear, and the graphical representation should be improved.

    We have made steps to improve the clarity of the figures presented.

    For dataset 3, τ was the movement kinematics, but it is not specified how the task parameters were formulated. Did the authors use the data from all 32 kinematic markers, 4 IMUs, and force plates? If yes, it should be specified why all these signals were used. For sure, there will be signals included that are not relevant to the specific task. Did the authors select specific signals based on their relevance to the task (e.g., ankle kinematics)?

    We have now clarified this in the text as follows: “For datasets 1 and 2, we determine the MI between vectors with respect to several discrete task parameters representing specific task attributes (e.g. reaching direction, speed etc.), while for dataset 3 we determined the task-relevant and -irrelevant muscles couplings in an unassuming way by quantifying them with respect to all available kinematic, dynamic and inertial motion unit (IMU) features.”

    How did the authors endure that crosstalk did not affect their analysis, particularly between, e.g., finger extensors and brachioradialis and posterior deltoid and anterior deltoid (dataset 1)?

    We have addressed this point in the previous round of reviews and made an explicit statement regarding cross-talk in the discussion section: “Although distinguishing task-irrelevant muscle couplings may capture artifacts such as EMG crosstalk, our results convey several physiological objectives of muscles including gross motor functions [66], the maintenance of internal joint mechanics and reciprocal inhibition of contralateral limbs [19,51].”

    It would be informative to add some examples of not trivial/obvious task-related synergistic muscle combinations that have been extracted in the three datasets. Most of the examples reported in the manuscript are well-known biomechanically and quite intuitive, so they do not improve our understanding of synergistic muscle control in humans.

    Our framework improves our understanding of synergistic motor control by enabling the formal quantification of synergistic muscle interactions, a capability not present among current approaches. Regarding the implications of this advance in terms of concrete examples, we have further clarified our examples presented in the results section, for example:

    “Across datasets, many the muscle networks could be characterised by the transmission of complementary task information between functionally specialised muscle groups, many of which identified among the task-redundant representations (Fig.9-10 and Supp. Fig.2). The most obvious example of this is the S3 synergist muscle network of dataset 2 (Fig.11), which captures the complementary interaction between task-redundant submodules identified previously (S3 (Fig.9)).”

    The description shows how our framework can extract the cross-module interactions that align with the higher-level objectives of the system, here the synergistic connectivity between the upper and lower body modules. Current approaches can only capture redundant and task-irrelevant interactions. Thus our framework provides additional insight into movement control.

    The number of participations in dataset 2 is very limited and should be increased. We appreciate the reviewer's comment and would like to point out that for dataset 2 our aim was to increase the number of muscles (30), tasks (72) and trials for each task (30) which produced a very large dataset for each participant. This came at the expense of low number of participants, however all our statistical analyses here can be performed at the single-participant level. Furthermore, dataset 3 includes 25 participants and it enables us to demonstrate the reliability of the findings across participants.

    Reviewer #2 (Recommendations For The Authors):

    I believe it is important in the future to explore the approach proposed with a range of simulation data and neuromechanical models, to explore the issues I have raised and that you have acknowledged, though I agree it is likely out of scope for the paper here.

    We agree with the reviewer that this would be valuable future work and indeed plan to do this in our future research.

    The Github code for this paper should likely include the various data sets used in the paper and figures, appropriately anonymized, in order to allow the data to be explored and analyses replicated and package demonstrated to be exercised fully by a new user.

    We thank the reviewer for this suggestion. Dataset3 is already available online at https://doi.org/10.1016/j.jbiomech.2021.110320. We will also make the other 2 datasets publicly available on our lab website very soon. Until then, as stated in the manuscript, we will make them available to anyone upon reasonable request.

    Reviewer #3 (Recommendations For The Authors):

    I have the following open points to suggest to the authors:

    First, I recommend improving the quality of the figures: in the pdf version I downloaded, some writings are impossible to read.

    We fully agree with the reviewer and note that in the pdf version of the paper, the figures are a lot worse than in the submitted word document submitted. Nevertheless, we will make further improvements on the figures as requested.

    Even though the manuscript has improved, I still feel that some points were not addressed or were only partially addressed. In particular:

    • The proposed comparison with NMF helps understanding why incorporating the task space is useful (and I fully agree with the authors about this point as the main reason to propose their contribution). However, the comparison does not help the reader to understand whether the synergies incorporating the task space are biased by the introduction of the task variables.

    This question can be also reformulated as: are muscle synergies modified when task space variables are incorporated? Is the "weight" on task coefficients affecting the composition of muscle synergies? If so, the added interpretational power is achieved at the cost of losing the information regarding the neural substrate of synergies? I understand this point is not immediate to show, but it would increase the quality of the work.

    • Reference to previous approaches that aimed at including task variables into synergy extraction are still missing in the paper. Even though it is not required to provide quantitative comparisons with other available approaches, there are at most 2-3 available algorithms in the literature (kinematics-EMG; force-EMG), that should not be neglected in this work. What did previous approaches achieve? What was improved with this approach? What was not improved?

    Previous attempts of extracting synergies with non-linear approaches could also be described more.

    In the latest version of the manuscript, we have referenced both the mixed NMF and autoencoders based algorithms. In both the introduction and discussion section of the manuscript, we also specify that our framework quantifies and decomposes muscle interactions in a novel way that cannot be done by other current approaches. In the results section we use examples from 3 different datasets to make this point clear, providing intuition on the use cases of our framework.

  6. eLife

    eLife assessment

    The work by O'Reilly and Delis is important to extend the synergy ideas using methods from signal processing and information theory to cluster muscles and task parameters, thereby advancing our understanding of the modular architecture of motor control. The method is innovative, and the findings are compelling from theoretical and practical perspectives. The work will be of broad interest to motor control and neural engineering researchers.

  7. eLife

    Reviewer #1 (Public Review):

    The proposed study provides an innovative framework for the identification of muscle synergies taking into account their task relevance. State-of-the-art techniques for extracting muscle interactions use unsupervised machine-learning algorithms applied to the envelopes of the electromyographic signals without taking into account the information related to the task being performed. In this work, the authors suggest to include the task parameters in extracting muscle synergies using a network information framework previously proposed. This allows the identification of muscle interactions that are relevant, irrelevant, or redundant to the parameters of the task executed.

    The proposed framework is a powerful tool to understand and identify muscle interactions for specific task parameters and it may be used to improve man-machine interfaces for the control of prostheses and robotic exoskeletons.

    With respect to the network information framework recently published, this work added an important part to estimate the relevance of specific muscle interactions to the parameters of the task executed.

    It is not clear how the well-known phenomenon of cross-talk during the recording of electromyographic muscle activity may affect the performance of the proposed technique and how it may bias the overall outcomes of the framework.

  8. eLife

    Reviewer #2 (Public Review):

    This paper is an attempt to extend or augment muscle synergy and motor primitive analyses and ideas with addition of task-driven measures. The authors' idea is to use information metrics (mutual information, co-information) in 'synergy' constraint creation that includes task information directly. By using task related information and muscle information sources and then sparsification, the methods construct task relevant network communities among muscles, together with task redundant communities, and task irrelevant communities. This process of creating network communities may then constrain and help to guide subsequent synergy identification using the authors published sNM3F algorithm to detect spatial and temporal synergies. The revised paper is now much clearer and examples are helpful in various ways.

    The impact of the information theoretic constraints developed as network communities on subsequent synergy separation are posited to be benign and to improve separation and identification of synergies over other methods (e.g., NNMF). However, not fully addressed are the possible impacts of the methods on the resulting compositionality and its links with physiological bases: the possibility remains that the methods here sometimes will instead lead to modules that represent more descriptive ML frameworks for task description, and resulting 'synergies' that may not support physiological work easily. Accordingly, there is a caveat for users of this framework. This is recognized and acknowledged by the authors in their rebuttal letters responding to prior reviews. It will remain for other work to explore this issue, likely through testing on detailed high degree of freedom artificial neuromechanical models and tasks. This possible issue and caveat with the strategy proposed by the authors likely should be more fully acknowledged in the paper.

    The approach of the methods seeks to identify task relevant coordinative couplings. This identification is a meta problem for more classical synergy analyses. Classical/prior analyses seek compositional elements stable across tasks. These elements may then be explored in causal experiments and in generative simulations of coupling and control strategies. However, task-based understanding of synergy roles and functional uses as captured in the proposed methods are significant, and the field is clearly likely to be aided by methods in this study.
    Information based separation has been used in muscle synergy analyses previously, by using infomax ICA, to discover physiological primitives. Though linear mixing of sources is assumed in ICA, minimized mutual information among source (synergy) drives is the basis of the separation and can detect low variance synergy contributions (e.g., see Yang, Logan, Giszter, 2019). In the work in the current paper, instead, mutual information approaches are used to cluster muscles and task features into network communities preceding the SNM3F algorithm use for separation, rather than using minimized information in the separation process directly. This contrast of an accretive or agglomerative mutual information strategy in the paper here, which is used to cluster into networks, versus a minimizing mutual information source separation used in infomax ICA epitomizes a key difference in approach. Indeed, physiological causal testing of synergy ideas is neglected in the literature reviews presented in the paper. Although these are only in animal work (e.g., Hart and Giszter, 2010; Takei and Seki, 2017), the clear connection of muscle synergy analysis choices to physiology is important, and eventually these issues need to be better managed and understood in relation to the new methods proposed here, even if not in this paper. Analyses of synergies using the methods the paper has proposed will likely be very much dependent on the number and quality of task variables included and how these are chosen, and the impacts of these on the ensuing sparsification and network communities used prior to SNM3F has already been noted. The authors acknowledge this in their responses. It would be useful in the future to explore the approach described with a range of simulated data to better understand the caveats, and optimizations for best practices in applications of this approach.

    A key component of the authors' arguments here is their 'emergentist' view presented in the work, but perhaps not made fully explicit. Through the reductionist lens, which was used in the other physiological work noted above, muscle groupings are the units (primitives or 'building blocks' with informational separations) of coordinated movement and thus the space of these intermuscular unit interactions and controls is of particular interest for understanding movement construction and underlying physiology. This may allow representation of a hierarchy or heterarchy of neural control elements with clear physiological bases at spinal, brainstem and cortical levels. On the other hand, the emergentist view utilized by the authors here suggests that muscle groupings emerge from interactions between many constituent parts in a more freeform fashion with potentially larger task synergy assemblies (also quantified here using information tools). Information methods are applied differently using the two different lenses. The emergentist lens may potentially obscure fundamental neural controls and make them harder to explore in the descriptions resulting. Nonetheless, the different approaches to muscle synergy research, seeking different sorts of explanation and description of 'synergy', can be complementary and beneficial for the field overall going forward, so long as the caveats and concerns noted here are employed by readers in the interpretation of this new method.

  9. eLife

    Reviewer #3 (Public Review):

    In this study, the authors developed and tested a novel framework for extracting muscle synergies. The approach aims at removing some limitations and constrains typical of previous approaches used in the field. In particular, the authors propose a mathematical formulation that removes constrains of linearity and couple the synergies to their motor outcome, supporting the concept of functional synergies and distinguishing the task-related performance related to each synergy. While some concepts behind this work were already introduced in recent work in the field, the methodology provided here encapsulates all these features in an original formulation providing a step forward with respect to the currently available algorithms. The authors also successfully demonstrated the applicability of their method to previously available datasets of multi-joint movements.

    Preliminary results positively support the scientific soundness of the presented approach and its potential. The added values of the method should be documented more in future work to understand how the presented formulation relates to previous approaches and what novel insights can be achieved in practical scenarios and confirm/exploit the potential of the theoretical findings.

  10. Version published to 10.7554/elife.87651.2 on eLife
  11. eLife

    Author Response

    The following is the authors’ response to the original reviews.

    eLife assessment

    The study by O'Reilly and Delis provides a valuable data-driven framework for extracting task-related muscle synergies in a step towards the understanding and practical use of synergies in real scenarios (e.g., evaluation of patients in a clinical environment). The approach is incomplete since the authors did not compare their method with classical physiologically grounded approaches for assessing muscle synergies. In this sense, the comparisons with classical approaches would clarify if physiological assemblies were preserved and were not altered to incorporate task space variables. Despite limitations, the proposed framework would interest motor control and neural engineering researchers.

    We thank the editors for the positive assessment of our work and appreciate their constructive feedback. In our revised manuscript, we believe we have sufficiently addressed the identified limitations by a) comparing our approach to existing physiologically-based methods, providing thorough comparisons of their respective outputs, b) applying it to a dataset of post-stroke participants to demonstrate that it can identify physiologically-interpretable markers of motor recovery and c) providing examples to demonstrate how readers can interpret the novel perspective introduced.

    Reviewer #1 (Public Review):

    The proposed study provides an innovative framework for the identification of muscle synergies taking into account their task relevance. State-of-the-art techniques for extracting muscle interactions use unsupervised machine-learning algorithms applied to the envelopes of the electromyographic signals without taking into account the information related to the task being performed. In this work, the authors suggest including the task parameters in extracting muscle synergies using a network information framework previously proposed. This allows the identification of muscle interactions that are relevant, irrelevant, or redundant to the parameters of the task executed.

    The proposed framework is a powerful tool to understand and identify muscle interactions for specific task parameters and it may be used to improve man-machine interfaces for the control of prostheses and robotic exoskeletons.

    With respect to the network information framework recently published, this work added an important part to estimate the relevance of specific muscle interactions to the parameters of the task executed. However, the authors should better explain what is the added value of this contribution with respect to the previous one, also in terms of computational methods.

    We thank the reviewer for their constructive comments. We have adjusted the introduction section of the manuscript to better explain the added value of this framework over previous work. Specifically, we draw the reviewer’s attention to the following updated section of the introduction:

    “In [11], we considered, key limitations among current approaches to muscle synergy analysis in extracting functionally relevant and interpretable patterns of muscle activity [12]. We proposed a combinatorial approach based on information- and network-theory and dimensionality reduction (the network-information framework (NIF)) that significantly improved the generalisability of the extraction process by, among others, removing restrictive model assumptions (e.g. linearity, same mixing coefficients) and the reliance on variance-accounted-for (VAF) metrics [12]. By determining the pairwise mutual information between muscles, this innovation paved the way for the appropriate mapping of muscular interactions to the task space. To elaborate on the significance of this development, the extraction of motor patterns in isolation of the task space comes at the expense of both functional and physiological relevance [12,13]. Furthermore, effective methods for mapping large-scale physiological dynamics to behaviour is a current gap across the neurosciences [14]. Thus, here we build on this work by, for the first time, directly including task space parameters during muscle synergy extraction. In doing so, we address these current research gaps, progressing muscle synergy research and successful engineering applications in a fruitful direction [12,15,16]. This enables us, in a novel way, to dissect the concept of the muscle synergy and therefore quantify interactions between muscle activations with shared or complementary functional roles. “

    In general, the method proposed relies on several hyperparameters and cost functions that have been optimized for the specific datasets. A sensitivity analysis should be performed, varying these parameters and reporting the performance of the framework.

    We thank the reviewer for this comment which enabled us to clarify a potential misunderstanding. Our proposed framework does not require setting or varying hyperparameters to optimise cost functions.

    For model-rank specification, a modularity maximising cost-function is used which determines what partitioning of the networks results in maximal modularity. We have offered two alternative approaches using this cost-function which consistently converge on the same solution. To further ensure the representativeness of this solution, we also offer a consensus-based approach where we apply these alternative approaches to individual participant or task data, then group the collective partitions together and re-apply the approaches. One of these approaches (Equation 2.2) requires two hyperparameters, γ and ω, which adjust the intra- and inter- network layer resolutions. As stated in the manuscript, we set both of these parameters to 1, thus nullifying their presence in the cost-function and aligning our work with the classical notion of modularity. Across the two alternative approaches to model-rank specification, the solution is unique and data-driven and has a demonstratable generalisability across datasets.

    The only other cost-function present in the framework is during dimensionality reduction, which is a standard loss function used across the muscle synergy analysis literature. Thus, the approach is essentially parameter-free and we now have mentioned this more explicitly in the manuscript:

    “To empirically determine the number of components to extract in a parameter-free way, we then concatenated these adjacency matrices into a multiplex network and employed network community-detection protocols to identify modules across spatial and temporal scales (fig.3(D)) [29–32,44].”

    “In its generalised multilayer form, the Q-statistic is given an additional term to consider couplings between layers l and r with intra- and inter-layer resolution parameters γ and ω (Equation 2.2). Here, μ is the total edge weight across the network and γ and ω were set to 1 in the current study for classical modularity [30], thus removing the need for any hyperparameter tuning.”

    It is not clear how the well-known phenomenon of cross-talk during the recording of electromyographic muscle activity may affect the performance of the proposed technique and how it may bias the overall outcomes of the framework.

    Indeed artifacts such as crosstalk are a standard issue across the EMG literature and may impact the performance of subsequent analyses where prevalent in the dataset. Crosstalk is expected to be present irrespective of the task and so should not affect redundant and synergistic muscle representations, however it could be present in the task-irrelevant muscle interactions extracted. Due to the prominence of long-range functional connections with the task-irrelevant representations extracted, we suggest that such artifacts are unlikely to have played a prominent role in the extracted patterns. Nonetheless, we have recognised this possibility with the following updated sentence in the Discussion section:

    “Although distinguishing task-irrelevant muscle couplings may capture artifacts such as EMG crosstalk, our results convey several physiological objectives of muscles including gross motor functions [65], the maintenance of internal joint mechanics and reciprocal inhibition of contralateral limbs [20,50].”

    Reviewer #2 (Public Review):

    This paper is an attempt to extend or augment muscle synergy and motor primitive ideas with task measures. The authors idea is to use information metrics (mutual information, co-information) in 'synergy' creation including task information directly. My reading of the paper is that the framework proposed radically moves from attempts to be analytic in terms of physiology and compositionality with physiological bases, instead into more descriptive ML frameworks that may not support physiological work easily.

    We thank the reviewer for taking the time to provide a thorough commentary on this manuscript. An overall aim in developing this framework is to build on other recent developments in providing a more fine-grained functional architecture underlying movement control [1,2]. It is a requirement for the successful communication and introduction of this toolbox to the field to provide readers with an understanding of how to use the framework and an intuition on how to interpret the results. Thus, we agree with the reviewer that functional interpretations are of crucial use.

    We also agree with the reviewer that maintaining a physiological underpinning is a desirable direction for the field and should not be made secondary to functional descriptions. In our updated version of this manuscript, we have therefore included direct comparisons with the gold-standard in the field for muscle synergy extraction, namely non-negative matrix factorisation based muscle synergy extraction (see ‘Building on current approaches to muscle synergy analysis’ and fig.5-6 of revised manuscript) [3,4]. In these comparison, we show how our framework goes beyond this current approach in terms of functional insight while still maintaining physiological relevance. Indeed, in the revised manuscript we also include a fourth dataset comprising post-stroke participants and healthy controls (Fig.6). We demonstrate, through a simple example application to this dataset, how our proposed framework can produce more predictive representations of motor impairment than the gold-standard approach. The representations we identified were discriminative of motor impairment measured via the Fugl-Meyer assessment using just one trial per participant. This improves considerably upon the sensitivity of the current approach to altered motor patterns which have predominantly required many trials and participants to gain significance [5,6]. Thus, the patterns we extract are a more comprehensive representation of the actual underlying physiological state of the participants.

    This approach is very different from the notions of physiological compositional elements as muscle synergies and motor primitives, and to me seems to really be striving to identify task relevant coordinative couplings. This is a meta problem for more classical analyses. Classical analyses seek compositional elements stable across tasks. These elements may then be explored in causal experiments and generative simulations of coupling and control strategies. The present work does not convince me that the joint 'meta' analysis proposed with task information added is not unmoored from physiology and causal modeling in some important ways. It also neglects publications and methods that might be inconvenient to the new framework.

    We would be very interested in receiving the reviewer’s suggestions of existing approaches that we have not incorporated here and would be happy to discuss these in the revised manuscript.

    Information based separation has been used in muscle synergy analyses using infomax ICA, which is information not variance based at core. Though linear mixing of sources is assumed, minimized mutual information is the basis.

    We agree with the reviewer that ICA relies on information measures, however it does not incorporate task-space information. The novelty of our approach lies in the characterisation of muscle interactions with respect to the task at hand. If the reviewer could provide references to this statement, we would be able to consider this further.

    Physiological causal testing of synergy ideas is neglected in the literature reviews in the paper. Although these are in animal work, the clear connection of muscle synergy choices and analyses to physiology is important and needs to be managed in the new methods proposed. Is any correspondence assumed? Possible?

    We agree with τhe reviewer that this a crucial element of muscle synergy research and will aim to address it in our future work. However, we would like to point out that the current manuscript is a “tools and resources” article aiming to introduce a new framework. In our revised manuscript, we have incorporated an application of the framework to a dataset from post-stroke patients to demonstrate the use of the framework in clinical settings to identify biomarkers and use them to make predictions of motor recovery (see Fig.6 of updated manuscript).

    Questions and concerns with the framework as an overall tool:

    First, muscle based motor information sources have influences on different time scales in the task mechanics. Analyses of synergies in the methods proposed will be very much dependent on the number and quality of task variables included and how these are managed. Standardizing and comparing among labs, tasks sets and instrumentation differences is not well enough considered as a problem in this new proposed method toolset, at least in my reading. Will replication, and testing across groups ever be truly feasible in this framework?

    We agree with the reviewer that this important point can be a limitation of the applicability of the framework. For this reason, we chose a “holistic” approach, applying the framework to several datasets collected in different settings, and selecting different kinds of task variables to extract muscle networks from. Crucially, we used a leave-one-task-out and leave-one-participant-out cross validation procedure to specifically address this point. Our results showed that the extracted couplings are robust irrespective of the task variable and/or participant excluded and this lends credit to the generalisability of the framework.

    Muscle based motor information sources have influences on different time scales in the task mechanics. Kinematic analyses, dynamic analyses and force plate analyses of the same task may provide task variables that alter the results in the proposed framework it seems.

    As we have mentioned above, here we used all the above types of task variables together to illustrate the range of measures that can be included in the proposed framework and showed that the outputs are robust to the exclusion of any task/participant. This point is especially evident for dataset 3 results, where high levels of generalisability were found despite the inclusion of kinematic, dynamic and IMU data (see Table 1. of original submission and updated manuscript). We believe that this is an advantage of the approach as it allows researchers to apply the method to different kinds of measurements they may have collected and gain insights into the relationships of muscle couplings with kinematic/dynamic/force parameters. This will also enable scientists to attribute different functional roles to the identified couplings and it is something we plan to do in future applications of the framework.

    Second, there is a sampling problem in all synergy analyses. We cannot record all muscles or all task parameters. Examining synergies across multiple tasks seeks 'stationary' compositionality. Including task specific elements may or may not reinforce or give increased coordinative precision to the stationary compositionality.

    We fully agree that this is a limitation of all synergy analyses and aimed to consider this study a step in the direction of addressing this limitation by providing the research community with a toolbox that can be used to quantify muscle couplings that can have different levels of task specificity.

    To me the new methods proposed seem partly orthogonal to the ideas of stable compositionality. The 'synergies' obtained will likely differ, and are more likely to be coordinative control groupings of recurrent task and muscle motifs (based on instrumentation) which may or may not relate to core compositionality in physiology. Is there any expectation that the framework should relate to core compositionality and physiology. This is not clear in the paper as written.

    In our new analysis, we have compared the proposed approach to existing physiologically-based methodologies and showed that the new framework can capture several salient physiological features of movement that the current NMF-based approach cannot. For example, as we have moved away from optimising variance accounted for metrics, our framework can identify subtle muscle couplings that have important functional roles. These subtle couplings are often not captured in current muscle synergy analysis as, against physiological relevance, higher amplitude muscles often take prominence. Further, by directly including task parameters during extraction, we can determine the muscles that have a functional role concerning the included task parameter rather than inferring this relationship indirectly using knowledge about the task executed. In our updated manuscript, by applying the framework to post-stroke participants (see Fig.6), we were also able to demonstrate that the extracted couplings are associated with functional parameters of motor recovery and have a clear link with the physiological state of individual participants.

    It would be useful to explore the approach with a range of neuromechanical models and controllers and simulated data to explore the issues I am raising and convince readers that this analysis framework adds clarity rather than dissolving the generalizability and interpretability of analyses in terms of underlying causal mechanisms.

    The authors need to better frame their work in relation to causal analyses if they are claiming links to muscle synergies analyses and claim extension/refinement. Alternatively, these may not be linked, and instead parallel approaches exploring different hypotheses and goals using different organizational data descriptors.

    To address the reviewers concerns here, we have included in the updated manuscript a toy example simulating situations in which pairs of muscles would have a redundant or synergistic functional relationship (see Fig.2). This simulation gives clear intuition on situations where two muscles (e.g. an antagonist-agonist pair) may share functionally similar or complementary information about task direction (left vs right). In particular, within the main text describing this figure, we state how current NMF based approaches consider muscles functionally equivalent when they share similar magnitude activations, whereas our framework captures muscles with identical task information. Thus, our work is an extension of current approaches towards understanding causal mechanisms. The suggestion to use neuromechanical models is valuable, however we consider it beyond the scope of this work. This “Tools and Resources” paper is aimed at introducing the computational framework for the analysis of large-scale muscle couplings in task space. Our future work will use this framework to address unanswered questions in the field and we hope that it will be helpful for other scientists in testing their hypotheses.

    To me this appears a data science tool that may not help any reductionist efforts and leads into less interpretable descriptions of motor control. Not invalid, but sufficiently different that common term use muddies the water.

    We believe that the novel evidence we provided both on simulated and real data have contributed to a better interpretability of the approach outcomes. Specifically, we have introduced examples showing the functional roles of the different types of interactions as well as the predictive power of the outputs. Concerning the use of the term synergy, we have provided a clear description throughout the manuscript regarding the interpretation of synergy vs redundancy in the novel perspective we propose. For example in the discussion section:

    “ We thus sought to provide greater nuance to the notion of ‘working together’ by defining motor redundancy and synergy in information-theoretic terms [6,56]. In our framework, redundancy and synergy are terms describing functionally similar and complementary motor signals respectively, introducing a new perspective that is conceptually distinct from the traditional view of muscle synergies as a solution to the motor redundancy problem [3,6,7]. In this new definition of muscle interactions in the task space, a group of muscles can ‘work together’ either synergistically or redundantly towards the same task. In doing so, the perspective instantiated by our approach provides novel coverage to the partitioning of task-relevant and -irrelevant variability implemented by the motor system along with an improved specificity regarding the functional roles of muscle couplings [20–22]. Our framework emphasises not only the role of functionally redundant muscle couplings that result from the underlying degeneracy of the motor system, but also of complementary, synergistic dependencies that are important for communication and integration across specialised neural circuitry [57,58]. Thus, the present study aligns the muscle synergy concept with the current mechanistic understanding of the nervous system whilst offering an analytical approach amenable to the continued advances in large-scale data capture [14,59].”

    Reviewer #3 (Public Review):

    In this study, the authors developed and tested a novel framework for extracting muscle synergies. The approach aims at removing some limitations and constrains typical of previous approaches used in the field. In particular, the authors propose a mathematical formulation that removes constrains of linearity and couple the synergies to their motor outcome, supporting the concept of functional synergies and distinguishing the task-related performance related to each synergy. While some concepts behind this work were already introduced in recent work in the field, the methodology provided here encapsulates all these features in an original formulation providing a step forward with respect to the currently available algorithms. The authors also successfully demonstrated the applicability of their method to previously available datasets of multi-joint movements.

    Preliminary results positively support the scientific soundness of the presented approach and its potential. The added values of the method should be documented more in future work to understand how the presented formulation relates to previous approaches and what novel insights can be achieved in practical scenarios and confirm/exploit the potential of the theoretical findings.

    Strengths:

    This work proposes a novel framework that addresses physiologically non-verified hypothesis of standard muscle synergy methods: it removes restrictive model assumptions (e.g. linearity, same mixing coefficients) and the reliance on variance-accounted-for (VAF) metrics.

    The method is solid and achieves the prescribed objectives at a computational level and in preliminary laboratory data.

    A toolbox is available for testing the methods on a larger scale.

    The paper is well written and shows a high level of innovation, original content and analysis

    Weaknesses:

    Task performance variables could be specified in more quantitative definition in future work (e.g.: articular angles rather than a generic starting point- end point).

    We agree with this point and will incorporate it in future work. Our aim here was to show that the framework would work with any task variable and that scientists can use it to identify the relevance of muscle interactions to different types of task parameters.

    The paper does not show a comparison with previous approaches (e.g.: NMF) or recently developed approaches (such as MMF).

    We have now illustrated such a comparison on two datasets and explained more how the new framework can dissect the different types of muscle groupings (see ‘Building on current approaches to muscle synergy analysis’ section and Fig.5-6 of revised manuscript).

    A discussion of the likely impact of the work on the field, and the utility of the methods and data to the community.

    In our revised manuscript, we have introduced 2 new applications of the framework to real data to exemplify its use for a) functional interpretability and b) identification of biomarkers (see ‘Building on current approaches to muscle synergy analysis’ section and Fig.5-6 of revised manuscript). We also point towards its use in movement restoration and augmentation devices and in the clinical setting in the discussion section:

    “The separate quantification of these muscle interaction types opens up novel opportunities in the practical application of muscle synergy analysis, as demonstrated in the current study through the identification of a significant predictor of motor impairment post-stroke from single-trials [5,12,65]. For instance, these distinct representations may encapsulate different neural substrates that can be specifically assessed at the muscle-level for the purpose of bodily restoration and augmentation [66]. Uncovering their neural underpinnings is an interesting topic for future research.”

    In this work, the effort of the authors aimed at developing the field is clear. It is fundamental to develop novel frameworks for synergy extraction and use them to make them more interpretable and applicable to real scenarios, as well as more adherent to recent findings achieved in motor control and neuroscience that are not reflected in the standard models. At the same time, muscle synergies are being used more and more in research but their impact in practical scenarios is still limited, probably because synergies have rarely been analyzed in a functional context. This paper shows a very in-depth analysis and a novel framework to interpret data that links to the task space from a functional perspective. I also found that the results on the datasets are very well commented but could expand more to show why using this framework is advantageous.

    There are some key points for discussion that follow from this paper which can be described more, maybe in future work, and that might contribute to major developments in the field, including:

    The understanding of how the separation between relevant (redundant and synergistic) and irrelevant synergies impact on synergy analysis in practical works;

    We have now introduced new figures (Fig. 5 and 6) to the revised manuscript, demonstrating simple applications of the framework and providing intuition regarding the outputs. We have also added points to the Discussion commenting on the differences between types of couplings and how they can be interpreted in future works:

    “Our framework emphasises not only the role of functionally redundant muscle couplings that result from the underlying degeneracy of the motor system, but also of complementary, synergistic dependencies that are important for communication and integration across specialised neural circuitry [57,58]. Thus, the present study aligns the muscle synergy concept with the current mechanistic understanding of the nervous system whilst offering an analytical approach amenable to the continued advances in large-scale data capture [14,59].”

    “Although distinguishing task-irrelevant muscle couplings may capture artifacts such as EMG crosstalk, our results convey several physiological objectives of muscles including gross motor functions [64], the maintenance of internal joint mechanics and reciprocal inhibition of contralateral limbs [19,49]. Thus, task-irrelevant muscle interactions reflect both biomechanical- and task-level constraints that provide a structural foundation for task-specific couplings. The separate quantification of these muscle interaction types opens up novel opportunities in the practical application of muscle synergy analysis, as demonstrated in the current study through the identification of a significant predictor of motor impairment post-stroke from single-trials [5,12,65]. For instance, these distinct representations may encapsulate different neural substrates that can be specifically assessed at the muscle-level for the purpose of bodily restoration and augmentation [66]. Uncovering their neural underpinnings is an interesting topic for future research.”

    Interpreting how different synergistic organizations described in this work allows to better describe data from real scenarios (e.g.: motor recovery of patients after neurological diseases);

    We have now added an example application of the framework to a dataset of stroke patients (Fig.6) and identified a redundant muscle patterns that are predictive of functional measures.

    Discussing in detail how the presented findings compare with standard algorithms such as NMF to determine the added value provided with this approach;

    As indicated above, we have now shown such a comparison on two new datasets (see Fig.5-6 of revised manuscript).

    Describe how redundant synergies reflect real neural organization and - if their "existence" is confirmed - how they contribute to redesign the concept of muscle synergies and of modular/synergistic control in general.

    This is an important point that we have now addressed more in our Discussion by relating redundant muscle couplings to degeneracy in the motor system and synergistic couplings to integrative dynamics by higher-level processes. We have also added a simple simulation illustrating how synergistic and redundant interactions co-exist and represent different contributions to task performance (see Fig.2 of revised manuscript).

  12. eLife

    eLife assessment

    The work by O'Reilly and Delis is important to extend the synergy ideas using methods from signal processing and information theory to cluster muscles and task parameters, thereby advancing our understanding of the modular architecture of motor control. The method is innovative, and the findings are compelling from theoretical and practical perspectives. The work will be of broad interest to motor control and neural engineering researchers.

  13. eLife

    Reviewer #1 (Public Review):

    The proposed study provides an innovative framework for the identification of muscle synergies taking into account their task relevance. State-of-the-art techniques for extracting muscle interactions use unsupervised machine-learning algorithms applied to the envelopes of the electromyographic signals without taking into account the information related to the task being performed. In this work, the authors suggest including the task parameters in extracting muscle synergies using a network information framework previously proposed. This allows the identification of muscle interactions that are relevant, irrelevant, or redundant to the parameters of the task executed.

    The proposed framework is a powerful tool to understand and identify muscle interactions for specific task parameters and it may be used to improve man-machine interfaces for the control of prostheses and robotic exoskeletons.

    With respect to the network information framework recently published, this work added an important part to estimate the relevance of specific muscle interactions to the parameters of the task executed. However, the authors should better explain what is the added value of this contribution with respect to the previous one, also in terms of computational methods.

    It is not clear how the well-known phenomenon of cross-talk during the recording of electromyographic muscle activity may affect the performance of the proposed technique and how it may bias the overall outcomes of the framework.

  14. eLife

    Reviewer #2 (Public Review):

    This paper is an attempt to extend or augment muscle synergy and motor primitive ideas with task measures. The authors idea is to use information metrics (mutual information, co-information) in 'synergy' constraint creation that includes task information directly. By using task related information and muscle information sources and then sparsification, the methods construct task relevant network communities among muscles, together with task redundant communities, and task irrelevant communities. This process of creating network communities may then constrain and help to guide subsequent synergy identification using the authors published sNM3F algorithm to detect spatial and temporal synergies.

    The revised paper is much clearer and examples are helpful in various ways. However, figure 2 as presented does not convincingly show why task muscle mutual information helps in separating synergies, though it is helpful in defining the various network communities used in the toy example.

    The impact of the information theoretic constraints developed as network communities on subsequent synergy separation are posited to be benign and to improve over other methods (e.g., NNMF). However, not fully addressed are the possible impacts of the methods on compositionality links with physiological bases, and the possibility remains of the methods sometimes instead leading to modules that represent more descriptive ML frameworks that may not support physiological work easily. Accordingly, there is a caveat. This is recognized and acknowledged by the authors in their rebuttal of the prior review. It will remain for other work to explore this issue, likely through testing on detailed high degree of freedom artificial neuromechanical models and tasks. This possible issue with the strategy here likely needs to be fully acknowledged in the paper.

    The approach of the methods seeks to identify task relevant coordinative couplings. This is a meta problem for more classical synergy analyses. Classical analyses seek compositional elements stable across tasks. These elements may then be explored in causal experiments and generative simulations of coupling and control strategies. However, task-based understanding of synergy roles and functional uses is significant and is clearly likely to be aided by methods in this study.

    Information based separation has been used in muscle synergy analyses using infomax ICA, which is information based at core. Though linear mixing of sources is assumed in ICA, minimized mutual information among source (synergy) drives is the basis of the separation and detects low variance synergy contributions (e.g., see Yang, Logan, Giszter, 2019). In the work in this paper, instead, mutual information approaches are used to cluster muscles and task features into network communities preceding the SNM3F algorithm use for separation, rather than using minimized information in separation. This contrast of an accretive or agglomerative mutual information strategy here used to cluster into networks, versus a minimizing mutual information source separation used in infomax ICA epitomizes a key difference in approach here.

    Physiological causal testing of synergy ideas is neglected in the literature reviews in the paper. Although these are only in animal work (Hart and Giszter, 2010; Takei and Seki, 2017), the clear connection of muscle synergy analysis choices to physiology is important, and eventually these issues need to be better managed and understood in relation to the new methods proposed here, even if not in this paper.

    Analyses of synergies using the methods the paper has proposed will likely be very much dependent on the number and quality of task variables included and how these are managed, and the impacts of these on the ensuing sparsification and network communities used prior to SNM3F. The authors acknowledge this in their response. This caveat should likely be made very explicit in the paper.

    It would be useful in the future to explore the approach described with a range of simulated data to better understand the caveats, and optimizations for best practices in this approach.

  15. eLife

    Reviewer #3 (Public Review):

    In this study, the authors developed and tested a novel framework for extracting muscle synergies. The approach aims at removing some limitations and constraints typical of previous approaches used in the field. In particular, the authors propose a mathematical formulation that removes constraints of linearity and couples the synergies to their motor outcome, supporting the concept of functional synergies and distinguishing the task-related performance related to each synergy. While some concepts behind this work were already introduced in recent work in the field, the methodology provided here encapsulates all these features in an original formulation providing a step forward with respect to the currently available algorithms. The authors also successfully demonstrated the applicability of their method to previously available datasets of multi-joint movements.

    Preliminary results positively support the scientific soundness of the presented approach and its potential. The added values of the method should be documented more in future work to understand how the presented formulation relates to previous approaches and what novel insights can be achieved in practical scenarios and confirm/exploit the potential of the theoretical findings.

    In their revision, the authors have implemented major revisions and improved their paper. The work was already of good quality and now it has improved further. The authors were able to successfully:
    - improve the clarity of the writing (e.g.: better explaining the rationale and the aims of the paper);
    - extend the clarification of some of the key novel concepts introduced in their work, like the redundant synergies;
    - show a scenario in which their approach might be useful for increasing the understanding of motor control in patients with respect to traditional algorithms such as NMF. In particular, their example illustrates why considering the task space is a fundamental step forward when extracting muscle synergies, improving the practical and physiological interpretation of the results.

  16. Version published to 10.7554/elife.87651.1 on eLife
  17. eLife

    eLife assessment

    The study by O'Reilly and Delis provides a valuable data-driven framework for extracting task-related muscle synergies in a step towards the understanding and practical use of synergies in real scenarios (e.g., evaluation of patients in a clinical environment). The approach is incomplete since the authors did not compare their method with classical physiologically grounded approaches for assessing muscle synergies. In this sense, the comparisons with classical approaches would clarify if physiological assemblies were preserved and were not altered to incorporate task space variables. Despite limitations, the proposed framework would interest motor control and neural engineering researchers.

  18. eLife

    Reviewer #1 (Public Review):

    The proposed study provides an innovative framework for the identification of muscle synergies taking into account their task relevance. State-of-the-art techniques for extracting muscle interactions use unsupervised machine-learning algorithms applied to the envelopes of the electromyographic signals without taking into account the information related to the task being performed. In this work, the authors suggest including the task parameters in extracting muscle synergies using a network information framework previously proposed. This allows the identification of muscle interactions that are relevant, irrelevant, or redundant to the parameters of the task executed.

    The proposed framework is a powerful tool to understand and identify muscle interactions for specific task parameters and it may be used to improve man-machine interfaces for the control of prostheses and robotic exoskeletons.

    With respect to the network information framework recently published, this work added an important part to estimate the relevance of specific muscle interactions to the parameters of the task executed. However, the authors should better explain what is the added value of this contribution with respect to the previous one, also in terms of computational methods.

    In general, the method proposed relies on several hyperparameters and cost functions that have been optimized for the specific datasets. A sensitivity analysis should be performed, varying these parameters and reporting the performance of the framework.

    It is not clear how the well-known phenomenon of cross-talk during the recording of electromyographic muscle activity may affect the performance of the proposed technique and how it may bias the overall outcomes of the framework.

  19. eLife

    Reviewer #2 (Public Review):

    This paper is an attempt to extend or augment muscle synergy and motor primitive ideas with task measures. The authors idea is to use information metrics (mutual information, co-information) in 'synergy' creation including task information directly. My reading of the paper is that the framework proposed radically moves from attempts to be analytic in terms of physiology and compositionality with physiological bases, instead into more descriptive ML frameworks that may not support physiological work easily.

    This approach is very different from the notions of physiological compositional elements as muscle synergies and motor primitives, and to me seems to really be striving to identify task relevant coordinative couplings. This is a meta problem for more classical analyses. Classical analyses seek compositional elements stable across tasks. These elements may then be explored in causal experiments and generative simulations of coupling and control strategies. The present work does not convince me that the joint 'meta' analysis proposed with task information added is not unmoored from physiology and causal modeling in some important ways. It also neglects publications and methods that might be inconvenient to the new framework.

    Information based separation has been used in muscle synergy analyses using infomax ICA, which is information not variance based at core. Though linear mixing of sources is assumed, minimized mutual information is the basis.

    Physiological causal testing of synergy ideas is neglected in the literature reviews in the paper. Although these are in animal work, the clear connection of muscle synergy choices and analyses to physiology is important, and needs to be managed in the new methods proposed. Is any correspondence assumed? Possible?

    Questions and concerns with the framework as an overall tool:

    First, muscle based motor information sources have influences on different time scales in the task mechanics. Analyses of synergies in the methods proposed will be very much dependent on the number and quality of task variables included and how these are managed. Standardizing and comparing among labs, tasks sets and instrumentation differences is not well enough considered as a problem in this new proposed method toolset, at least in my reading. Will replication, and testing across groups ever be truly feasible in this framework? Muscle based motor information sources have influences on different time scales in the task mechanics. Kinematic analyses, dynamic analyses and force plate analyses of the same task may provide task variables that alter the results in the proposed framework it seems.

    Second, there is a sampling problem in all synergy analyses. We cannot record all muscles or all task parameters. Examining synergies across multiple tasks seeks 'stationary' compositionality. Including task specific elements may or may not reinforce or give increased coordinative precision to the stationary compositionality.
    To me the new methods proposed seem partly orthogonal to the ideas of stable compositionality. The 'synergies' obtained will likely differ, and are more likely to be coordinative control groupings of recurrent task and muscle motifs (based on instrumentation) which may or may not relate to core compositionality in physiology. Is there any expectation that the framework should relate to core compositionality and physiology. This is not clear in the paper as written.

    It would be useful to explore the approach with a range of neuromechanical models and controllers and simulated data to explore the issues I am raising and convince readers that this analysis framework adds clarity rather than dissolving the generalizability and interpretability of analyses in terms of underlying causal mechanisms.

    The authors need to better frame their work in relation to causal analyses if they are claiming links to muscle synergies analyses and claim extension/refinement. Alternatively, these may not be linked, and instead parallel approaches exploring different hypotheses and goals using different organizational data descriptors.
    To me this appears a data science tool that may not help any reductionist efforts and leads into less interpretable descriptions of motor control. Not invalid, but sufficiently different that common term use muddies the water.

  20. eLife

    Reviewer #3 (Public Review):

    In this study, the authors developed and tested a novel framework for extracting muscle synergies. The approach aims at removing some limitations and constrains typical of previous approaches used in the field. In particular, the authors propose a mathematical formulation that removes constrains of linearity and couple the synergies to their motor outcome, supporting the concept of functional synergies and distinguishing the task-related performance related to each synergy. While some concepts behind this work were already introduced in recent work in the field, the methodology provided here encapsulates all these features in an original formulation providing a step forward with respect to the currently available algorithms. The authors also successfully demonstrated the applicability of their method to previously available datasets of multi-joint movements.

    Preliminary results positively support the scientific soundness of the presented approach and its potential. The added values of the method should be documented more in future work to understand how the presented formulation relates to previous approaches and what novel insights can be achieved in practical scenarios and confirm/exploit the potential of the theoretical findings.

    Strengths:

    This work proposes a novel framework that addresses physiologically non-verified hypothesis of standard muscle synergy methods: it removes restrictive model assumptions (e.g. linearity, same mixing coefficients) and the reliance on variance-accounted-for (VAF) metrics.

    The method is solid and achieves the prescribed objectives at a computational level and in preliminary laboratory data.

    A toolbox is available for testing the methods on a larger scale.

    The paper is well written and shows a high level of innovation, original content and analysis

    Weaknesses:

    Task performance variables could be specified in more quantitative definition in future work (e.g.: articular angles rather than a generic starting point- end point).

    The paper does not show a comparison with previous approaches (e.g.: NMF) or recently developed approaches (such as MMF).

    A discussion of the likely impact of the work on the field, and the utility of the methods and data to the community.

    In this work, the effort of the authors aimed at developing the field is clear. It is fundamental to develop novel frameworks for synergy extraction and use them to make them more interpretable and applicable to real scenarios, as well as more adherent to recent findings achieved in motor control and neuroscience that are not reflected in the standard models. At the same time, muscle synergies are being used more and more in research but their impact in practical scenarios is still limited, probably because synergies have rarely been analyzed in a functional context. This paper shows a very in-depth analysis and a novel framework to interpret data that links to the task space from a functional perspective. I also found that the results on the datasets are very well commented but could expand more to show why using this framework is advantageous.

    There are some key points for discussion that follow from this paper which can be described more, maybe in future work, and that might contribute to major developments in the field, including:

    The understanding of how the separation between relevant (redundant and synergistic) and irrelevant synergies impact on synergy analysis in practical works;

    Interpreting how different synergistic organizations described in this work allows to better describe data from real scenarios (e.g.: motor recovery of patients after neurological diseases);

    Discussing in detail how the presented findings compare with standard algorithms such as NMF to determine the added value provided with this approach;

    Describe how redundant synergies reflect real neural organization and - if their "existence" is confirmed - how they contribute to redesign the concept of muscle synergies and of modular/synergistic control in general.

  21. Version published to 10.1101/2023.03.17.533096 on bioRxiv