Motor memories of object dynamics are categorically organized

  1. Evan Cesanek
  2. Zhaoran Zhang
  3. James N Ingram
  4. Daniel M Wolpert
  5. J Randall Flanagan

  • Curated by eLife

    eLife logo

    Evaluation Summary:

    This paper is of interest to scientists within the field of motor learning. Converging evidence from several behavioural experiments support key claims of the paper. However, it is unclear to what degree the reported effects can be strongly linked to motor versus cognitive systems, and to what degree they novel demand revision of existing theoretical frameworks.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

The ability to predict the dynamics of objects, linking applied force to motion, underlies our capacity to perform many of the tasks we carry out on a daily basis. Thus, a fundamental question is how the dynamics of the myriad objects we interact with are organized in memory. Using a custom-built three-dimensional robotic interface that allowed us to simulate objects of varying appearance and weight, we examined how participants learned the weights of sets of objects that they repeatedly lifted. We find strong support for the novel hypothesis that motor memories of object dynamics are organized categorically, in terms of families, based on covariation in their visual and mechanical properties. A striking prediction of this hypothesis, supported by our findings and not predicted by standard associative map models, is that outlier objects with weights that deviate from the family-predicted weight will never be learned despite causing repeated lifting errors.

Article activity feed

  1. Version published to 10.7554/elife.71627 on eLife
  2. eLife

    Author Response:

    Reviewer #1:

    Cesanek et al. performed a series of experiments designed to reveal whether or not the encoding of motor memories for novel object weight carries categorical structure. That is, given a set of objects of varying sizes and with weights that must be learned through experience, are objects grouped into categories such that the learned weight of one object generalises to objects within the same category, but not to objects outside of that category. Their results convincingly demonstrate the presence of such a categorical encoding. They show the following:

    1. The weight of an outlier object is not learned if its weight is near the weight predicted by category membership.
    1. The weight of an outlier object is learned if its weight is far from the value predicted by category membership.
    1. The weight of an outlier object is learned if there is no category structure binding the remaining objects (i.e., there is no category against which an outlier can be defined).
    1. If an outlier object is learned, then it influences the estimated weight of the other category members.
    1. If the weight of an outlier object is learned first in isolation, it is unlearned when the remaining objects are introduced if and only if its weight is near the value predicted by category membership.
    1. The threshold that constitutes "near" or "far" from the category boundary depends on recent sensorimotor experience.
    1. Learning of the outlier is all-or-nothing on a per participant basis.

    The major strength of the paper is the persuasiveness of the behavioural experiments, which were designed soundly and yielded clear results. There is little doubt that the some motor memories carry the type of categorical encoding detailed by the authors.

    Thank you for this clear summary of our main findings and the positive evaluation of the persuasiveness of our study.

    The major weakness of the paper is that it does not make strong contact with the relevant existing literature to clearly show that categorical encoding is (1) a truly novel behavioural observation in the motor learning literature, and (2) that it is inconsistent with the predictions of common motor learning theories and models. In particular, the authors own prior work frames motor learning as being governed by multiple internal models that can be switched between depending on contextual cues and environmental demands (see reference below). Insofar as contextual effects can drive similar results as categorical memory encoding, it is unclear how and why this and related models would fail to account for the present data.

    Wolpert, D. M., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11(7-8), 1317-1329.

    With respect to the first issue, we do not know of any literature that discusses categorical effects in motor control. In our revision, we have added text emphasizing that existing models of motor control cannot account for categorical effects because they do not contain any form of categorical encoding. With respect to the second issue, please see our response to Essential Revision #2, where we explain that the MOSAIC model of Wolpert & Kawato (1998) is effectively an associative learning model, not a categorical learning model, and hence could not explain our data.

    Reviewer #2:

    Using a novel 3D robotic device, the authors had participants learn to lift four training similar-looking objects whose weights were linearly correlated with the sizes and then tested how this training influenced the later memory formation for the middle-sized object with different densities. When the difference between the actual weight and the weight estimated from the linear relationship was not so large (i.e., within a family boundary), surprisingly, the training for lifting the test object was ineffective: The estimation of the object weight was constrained by the linear relationship of the family between size and the weight. The memory specific to the new object could be developed only when the difference was large enough.

    The results were unexpected from the conventional idea that object properties are encoded in an "associative map." The authors interpreted these results as evidence that the motor memory for lifting objects with different sizes and weights could be formed according to the "object family effect." All results of other control experiments were consistent with this interpretation.

    I was intrigued by the counter-intuitive results that the motor system sticks to estimate the object weight based on the family property even though this estimation is incorrect and induces the error. Although it remains unclear how such a memory for multiple objects is integrated from memory for each object, it is sure that this study has demonstrated a new aspect of motor memory while manipulating the objects with different sizes and weights.

    Thank you for this positive summary of the impact of our study.

    Reviewer #3:

    In this paper, Cesanek et al. use a novel object lifting task to investigate the "format" of memories for object dynamics. Namely, they ask if those memories are organized according to a smooth, local map, or discrete categories ('families'). They pit these competing models against one another across several experiments, asking if subjects' predicted weights of objects follow the family model or a smooth map. This was tested by having people train on objects of varying volumes/masses that were either consistent with a linear mapping between volume/mass, or where those dimensions were uncorrelated. This training phase was either preceded by, or preceded, a testing phase where a novel object with a deviant mass (but a medium-size volume) was introduced. As the authors expected, individuals trained on the linear mappings treated novel objects that were relatively close to the "family" average mass as a member of the family, and thus obligatorily interpolated to compute the expected mass of that object (i.e., under-predicting its true mass); conversely, when a novel object's mass was a substantial outlier w/r/t the training items, it was treated as a singleton and thus lifted with close to the correct force. Additional variations of this experiment provided further evidence that people tend to treat an object's dynamical features as a category label, rather than simply forming local associative representations. These findings offer a novel perspective on how people learn and remember the dynamics of objects in the world.

    Overall, I found this study to be both rigorous and creative. The experimental logic is refreshingly clear, and the results, which are replicated and extended several times in follow-up experiments, are rather convincing. I do think some additional analyses could be done, and data presentation could be improved. I also thought the generalization analysis, as I interpreted it, was difficult to align with the initial predictions.

    Thank you for this assessment of our work. In particular, please note that we have modified the generalization analysis based on some of your recommendations, and we feel that it is now more convincing and easier to understand.

  3. eLife

    Evaluation Summary:

    This paper is of interest to scientists within the field of motor learning. Converging evidence from several behavioural experiments support key claims of the paper. However, it is unclear to what degree the reported effects can be strongly linked to motor versus cognitive systems, and to what degree they novel demand revision of existing theoretical frameworks.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. Reviewer #1 agreed to share their name with the authors.)

  4. eLife

    Reviewer #1 (Public Review):

    Cesanek et al. performed a series of experiments designed to reveal whether or not the encoding of motor memories for novel object weight carries categorical structure. That is, given a set of objects of varying sizes and with weights that must be learned through experience, are objects grouped into categories such that the learned weight of one object generalises to objects within the same category, but not to objects outside of that category. Their results convincingly demonstrate the presence of such a categorical encoding. They show the following:

    1. The weight of an outlier object is not learned if its weight is near the weight predicted by category membership.

    2. The weight of an outlier object is learned if its weight is far from the value predicted by category membership.

    3. The weight of an outlier object is learned if there is no category structure binding the remaining objects (i.e., there is no category against which an outlier can be defined).

    4. If an outlier object is learned, then it influences the estimated weight of the other category members.

    5. If the weight of an outlier object is learned first in isolation, it is unlearned when the remaining objects are introduced if and only if its weight is near the value predicted by category membership.

    6. The threshold that constitutes "near" or "far" from the category boundary depends on recent sensorimotor experience.

    7. Learning of the outlier is all-or-nothing on a per participant basis.

    The major strength of the paper is the persuasiveness of the behavioural experiments, which were designed soundly and yielded clear results. There is little doubt that the some motor memories carry the type of categorical encoding detailed by the authors.

    The major weakness of the paper is that it does not make strong contact with the relevant existing literature to clearly show that categorical encoding is (1) a truly novel behavioural observation in the motor learning literature, and (2) that it is inconsistent with the predictions of common motor learning theories and models. In particular, the authors own prior work frames motor learning as being governed by multiple internal models that can be switched between depending on contextual cues and environmental demands (see reference below). Insofar as contextual effects can drive similar results as categorical memory encoding, it is unclear how and why this and related models would fail to account for the present data.

    Wolpert, D. M., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11(7-8), 1317-1329.

  5. eLife

    Reviewer #2 (Public Review):

    Using a novel 3D robotic device, the authors had participants learn to lift four training similar-looking objects whose weights were linearly correlated with the sizes and then tested how this training influenced the later memory formation for the middle-sized object with different densities. When the difference between the actual weight and the weight estimated from the linear relationship was not so large (i.e., within a family boundary), surprisingly, the training for lifting the test object was ineffective: The estimation of the object weight was constrained by the linear relationship of the family between size and the weight. The memory specific to the new object could be developed only when the difference was large enough.

    The results were unexpected from the conventional idea that object properties are encoded in an "associative map." The authors interpreted these results as evidence that the motor memory for lifting objects with different sizes and weights could be formed according to the "object family effect." All results of other control experiments were consistent with this interpretation.

    I was intrigued by the counter-intuitive results that the motor system sticks to estimate the object weight based on the family property even though this estimation is incorrect and induces the error. Although it remains unclear how such a memory for multiple objects is integrated from memory for each object, it is sure that this study has demonstrated a new aspect of motor memory while manipulating the objects with different sizes and weights.

  6. eLife

    Reviewer #3 (Public Review):

    In this paper, Cesanek et al. use a novel object lifting task to investigate the "format" of memories for object dynamics. Namely, they ask if those memories are organized according to a smooth, local map, or discrete categories ('families'). They pit these competing models against one another across several experiments, asking if subjects' predicted weights of objects follow the family model or a smooth map. This was tested by having people train on objects of varying volumes/masses that were either consistent with a linear mapping between volume/mass, or where those dimensions were uncorrelated. This training phase was either preceded by, or preceded, a testing phase where a novel object with a deviant mass (but a medium-size volume) was introduced. As the authors expected, individuals trained on the linear mappings treated novel objects that were relatively close to the "family" average mass as a member of the family, and thus obligatorily interpolated to compute the expected mass of that object (i.e., under-predicting its true mass); conversely, when a novel object's mass was a substantial outlier w/r/t the training items, it was treated as a singleton and thus lifted with close to the correct force. Additional variations of this experiment provided further evidence that people tend to treat an object's dynamical features as a category label, rather than simply forming local associative representations. These findings offer a novel perspective on how people learn and remember the dynamics of objects in the world.

    Overall, I found this study to be both rigorous and creative. The experimental logic is refreshingly clear, and the results, which are replicated and extended several times in follow-up experiments, are rather convincing. I do think some additional analyses could be done, and data presentation could be improved. I also thought the generalization analysis, as I interpreted it, was difficult to align with the initial predictions.

  7. Version published to 10.1101/2021.07.13.452183v1 on bioRxiv