Digital wearable insole-based identification of knee arthropathies and gait signatures using machine learning
Curation statements for this article:-
Curated by eLife
eLife assessment
This study presents a valuable dataset and tool that can aid in arthropathies' assessment, potentially enabling such evaluation to be done outside the lab. There is solid evidence supporting the comparison between the force plate and insole data, which can be strengthened by improvements in cross-validation, but the evidence for distinguishing disease signatures and elimination of walking speed as a factor is inconclusive and would need further analysis. This work will be of interest to physical therapists, clinicians, and researchers in the field of ankle/knee/hip osteoporosis and other lower limb joint diseases.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Gait is impaired in musculoskeletal conditions, such as knee arthropathy. Gait analysis is used in clinical practice to inform diagnosis and monitor disease progression or intervention response. However, clinical gait analysis relies on subjective visual observation of walking as objective gait analysis has not been possible within clinical settings due to the expensive equipment, large-scale facilities, and highly trained staff required. Relatively low-cost wearable digital insoles may offer a solution to these challenges. In this work, we demonstrate how a digital insole measuring osteoarthritis-specific gait signatures yields similar results to the clinical gait-lab standard. To achieve this, we constructed a machine learning model, trained on force plate data collected in participants with knee arthropathy and controls. This model was highly predictive of force plate data from a validation set (area under the receiver operating characteristics curve [auROC] = 0.86; area under the precision-recall curve [auPR] = 0.90) and of a separate, independent digital insole dataset containing control and knee osteoarthritis subjects (auROC = 0.83; auPR = 0.86). After showing that digital insole-derived gait characteristics are comparable to traditional gait measurements, we next showed that a single stride of raw sensor time-series data could be accurately assigned to each subject, highlighting that individuals using digital insoles can be identified by their gait characteristics. This work provides a framework for a promising alternative to traditional clinical gait analysis methods, adds to the growing body of knowledge regarding wearable technology analytical pipelines, and supports clinical development of at-home gait assessments, with the potential to improve the ease, frequency, and depth of patient monitoring.
Article activity feed
-
-
Author Response
Reviewer #1 (Public Review):
This work aims to evaluate the use of pressure insoles for measurements that are traditionally done using force platforms in the assessment of people with knee osteoarthritis and other arthropathies. This is vital for providing an affordable assessment that does not require a fully equipped gait lab as well as utilizing wearable technology for personalized healthcare.
Towards these aims, the authors were able to demonstrate that individual subjects can be identified with high precision using raw sensor data from the insoles and a convolutional neural network model. The authors have done a great job creating the models and combining an already available public dataset of force platform signals and utilizing them for training models with transferable ability to be used with data from …
Author Response
Reviewer #1 (Public Review):
This work aims to evaluate the use of pressure insoles for measurements that are traditionally done using force platforms in the assessment of people with knee osteoarthritis and other arthropathies. This is vital for providing an affordable assessment that does not require a fully equipped gait lab as well as utilizing wearable technology for personalized healthcare.
Towards these aims, the authors were able to demonstrate that individual subjects can be identified with high precision using raw sensor data from the insoles and a convolutional neural network model. The authors have done a great job creating the models and combining an already available public dataset of force platform signals and utilizing them for training models with transferable ability to be used with data from pressure insoles. However, there are a few concerns, regarding substantiating some of the goals that this manuscript is trying to achieve.
In addressing these concerns, if the results are further corroborated using the suggestions provided to the authors, this provides an exciting tool for identifying an individual's gait patterns out of a cluster of data, which is extremely useful for providing identifiable labels for personalized healthcare using wearable technologies.
Thank you for this enthusiasm for our work, and we hope that our responses are adequate to address what we can of these comments. Please note that we have made every effort to address comments that we can and appreciated the detailed feedback you provided.
Reviewer #2 (Public Review):
The authors aimed to investigate whether digital insoles are an appropriate alternative to laboratory assessment with force plates when attempting to identify the knee injury status. The methods are rigorous and appropriate in the context of this research area. The results are impressive, and the figures are exceptional. The findings of this study can have a great impact on the field, showing that digital insoles can be accurately used for clinical purposes. The authors successfully achieved their aims.
We thank the reviewer for this enthusiasm and hope our edits adequately address the points the reviewer made to strengthen the manuscript.
Reviewer #3 (Public Review):
In this manuscript, the authors describe the development of a machine-learning model to be used for gait assessment using insole data. They first developed a machine learning model using an existing, large data set of ground reaction forces collected during walking with force plates in a lab, from healthy adults and a group of people with knee injuries. Subsequently, they tested this model on ground reaction forces derived from insoles worn by a group of 19 healthy adults and a group of n=44 people with knee osteoarthritis (OA). The model was able to accurately identify individuals belonging to the knee OA group or the healthy group using the ground reaction forces during walking. Note: I do not have expertise on machine learning and will therefore refrain from reviewing the ML methods that were applied in this paper.
Strengths: The authors successfully externally validated the trained model for GRF on insole data. Insole data carries potentially rich information, including the path of the CoP during the stance phase. The additional value of insoles over force plates in itself is clear, as insoles can be used independently of laboratory facilities. Moreover, insoles provide information on the COP path, which can have added value over other mobile assessment methods such as inertial sensors.
Limitations: The second ML model, using only insole data to identify knee arthropathy from healthy subjects, was trained on a small sample of subjects. Although I have no background in ML, I can imagine that external validation in an independent and larger sample is needed to support the current findings.
Gait speed has a major influence on the majority of gait-related outcomes. Slow or more cautious gait, due to pain or other causes, is reflected in vertical GRF's with less pronounced peaks. A difference in gait speed between people with pain in their knee (due to injury) and healthy subjects can be expected. This raises the question of what the added value of a model to estimate vertical GRF is over a simpler output (e.g. gait speed itself). Moreover, the paper does not elucidate what the added value of machine learning is over a simpler statistical model.
This is a good point, however, clinically we are interested in weight bearing and difference in pressure related metrics in this musculoskeletal group, which speed will simply not provide. So we are looking at additional metrics.
There are numerous publications suggesting that non-speed related metrics are important to predict disease progression in a variety of conditions (e.g., D’Lima DD, Fregly BJ, Pail S, Steklov N, Colwell CW. Knee joint forces: prediction, measurement and significance. Proc Inst Mech Eng H. 2012:226:95–102. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3324308/). In OA, the vector on ground force in medial knee OA (not vertical) creates torque and that is correlated with disease progression. We have modified the text throughout to address these points.
In line with this issue, the current analyses are not strongly convincing me that the model described resulted in an identification of knee arthropathy-specific signature. Only knee arthropathy vs healthy (relatively young) subjects was compared, and we cannot rule out that this group only reflects general cautious, slow, or antalgic gait. As such, the data does not provide any evidence that the tool might be valuable to identify people with more or less severity of symptoms, or that the tool can be used to discriminate knee osteoarthritis from hip, or ankle osteoarthritis, or even to discriminate between people with musculoskeletal diseases and people with neurological gait disorders. This substantially limits the relevance for clinical (research) practice. In short, the output of the model seems to be restricted to "something is going on here", without further specification. Further development towards more specific aims using the insole data may substantially amplify clinical relevance.
While no dataset (or model) is perfect, we feel that this is the first time that this model has been developed and applied in this cohort/clinical context, and of course acknowledge that future work is needed to further validate and examine how clinically meaningful this model is.
We have broken out and added to a Study limitations section within the manuscript to reflect these caveats more clearly.
-
eLife assessment
This study presents a valuable dataset and tool that can aid in arthropathies' assessment, potentially enabling such evaluation to be done outside the lab. There is solid evidence supporting the comparison between the force plate and insole data, which can be strengthened by improvements in cross-validation, but the evidence for distinguishing disease signatures and elimination of walking speed as a factor is inconclusive and would need further analysis. This work will be of interest to physical therapists, clinicians, and researchers in the field of ankle/knee/hip osteoporosis and other lower limb joint diseases.
-
Reviewer #1 (Public Review):
This work aims to evaluate the use of pressure insoles for measurements that are traditionally done using force platforms in the assessment of people with knee osteoarthritis and other arthropathies. This is vital for providing an affordable assessment that does not require a fully equipped gait lab as well as utilizing wearable technology for personalized healthcare.
Towards these aims, the authors were able to demonstrate that individual subjects can be identified with high precision using raw sensor data from the insoles and a convolutional neural network model. The authors have done a great job creating the models and combining an already available public dataset of force platform signals and utilizing them for training models with transferable ability to be used with data from pressure insoles. However, …
Reviewer #1 (Public Review):
This work aims to evaluate the use of pressure insoles for measurements that are traditionally done using force platforms in the assessment of people with knee osteoarthritis and other arthropathies. This is vital for providing an affordable assessment that does not require a fully equipped gait lab as well as utilizing wearable technology for personalized healthcare.
Towards these aims, the authors were able to demonstrate that individual subjects can be identified with high precision using raw sensor data from the insoles and a convolutional neural network model. The authors have done a great job creating the models and combining an already available public dataset of force platform signals and utilizing them for training models with transferable ability to be used with data from pressure insoles. However, there are a few concerns, regarding substantiating some of the goals that this manuscript is trying to achieve.
In addressing these concerns, if the results are further corroborated using the suggestions provided to the authors, this provides an exciting tool for identifying an individual's gait patterns out of a cluster of data, which is extremely useful for providing identifiable labels for personalized healthcare using wearable technologies.
-
Reviewer #2 (Public Review):
The authors aimed to investigate whether digital insoles are an appropriate alternative to laboratory assessment with force plates when attempting to identify the knee injury status. The methods are rigorous and appropriate in the context of this research area. The results are impressive, and the figures are exceptional. The findings of this study can have a great impact on the field, showing that digital insoles can be accurately used for clinical purposes. The authors successfully achieved their aims.
-
Reviewer #3 (Public Review):
In this manuscript, the authors describe the development of a machine-learning model to be used for gait assessment using insole data. They first developed a machine learning model using an existing, large data set of ground reaction forces collected during walking with force plates in a lab, from healthy adults and a group of people with knee injuries. Subsequently, they tested this model on ground reaction forces derived from insoles worn by a group of 19 healthy adults and a group of n=44 people with knee osteoarthritis (OA). The model was able to accurately identify individuals belonging to the knee OA group or the healthy group using the ground reaction forces during walking. Note: I do not have expertise on machine learning and will therefore refrain from reviewing the ML methods that were applied in …
Reviewer #3 (Public Review):
In this manuscript, the authors describe the development of a machine-learning model to be used for gait assessment using insole data. They first developed a machine learning model using an existing, large data set of ground reaction forces collected during walking with force plates in a lab, from healthy adults and a group of people with knee injuries. Subsequently, they tested this model on ground reaction forces derived from insoles worn by a group of 19 healthy adults and a group of n=44 people with knee osteoarthritis (OA). The model was able to accurately identify individuals belonging to the knee OA group or the healthy group using the ground reaction forces during walking. Note: I do not have expertise on machine learning and will therefore refrain from reviewing the ML methods that were applied in this paper.
Strengths: The authors successfully externally validated the trained model for GRF on insole data. Insole data carries potentially rich information, including the path of the CoP during the stance phase. The additional value of insoles over force plates in itself is clear, as insoles can be used independently of laboratory facilities. Moreover, insoles provide information on the COP path, which can have added value over other mobile assessment methods such as inertial sensors.
Limitations: The second ML model, using only insole data to identify knee arthropathy from healthy subjects, was trained on a small sample of subjects. Although I have no background in ML, I can imagine that external validation in an independent and larger sample is needed to support the current findings.
Gait speed has a major influence on the majority of gait-related outcomes. Slow or more cautious gait, due to pain or other causes, is reflected in vertical GRF's with less pronounced peaks. A difference in gait speed between people with pain in their knee (due to injury) and healthy subjects can be expected. This raises the question of what the added value of a model to estimate vertical GRF is over a simpler output (e.g. gait speed itself). Moreover, the paper does not elucidate what the added value of machine learning is over a simpler statistical model.
In line with this issue, the current analyses are not strongly convincing me that the model described resulted in an identification of knee arthropathy-specific signature. Only knee arthropathy vs healthy (relatively young) subjects was compared, and we cannot rule out that this group only reflects general cautious, slow, or antalgic gait. As such, the data does not provide any evidence that the tool might be valuable to identify people with more or less severity of symptoms, or that the tool can be used to discriminate knee osteoarthritis from hip, or ankle osteoarthritis, or even to discriminate between people with musculoskeletal diseases and people with neurological gait disorders. This substantially limits the relevance for clinical (research) practice. In short, the output of the model seems to be restricted to "something is going on here", without further specification. Further development towards more specific aims using the insole data may substantially amplify clinical relevance.
-
