Search for Medical Information and Treatment Options for Musculoskeletal Disorders through an Artificial Intelligence Chatbot: Focusing on Shoulder Impingement Syndrome
This article has been Reviewed by the following groups
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
- Evaluated articles (PREreview)
Abstract
Background
The ChatGPT is an artificial intelligence chatbot that processes natural language text learned through reinforcement learning based on the GPT-3.5 architecture, a large-scale language model. Natural language processing models are being used in various fields and are gradually expanding their use in the medical field.
Purpose
This study aimed to investigate the medical information and treatment options that ChatGPT can provide for SIS.
Method
Using ChatGPT, which is provided as a free beta test, messages related to SIS were entered, and responses to medical information and treatment options were received and analyzed.
Result
ChatGPT not only provided answers to the definition, prevalence, and risk factors of SIS, but also symptoms, diseases with similar symptoms, and orthopedic tests according to the messages input. Additionally, a list of treatment options and exercises were provided.
Conclusion
ChatGPT will be able to provide overall useful medical information and treatment options to patients unfamiliar with SIS. However, caution is required as it contains content that may be biased or inappropriate information for patients with SIS. Nevertheless, if natural language processing technology develops further, it is expected to be able to express more detailed medical information and treatment options.
Article activity feed
-
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/19198800.
Short Summary of Main Findings In this December 2022 medRxiv preprint (v2), the authors evaluated the performance of an early AI chatbot (primarily ChatGPT) in providing medical information and treatment options for shoulder impingement syndrome. They submitted a series of structured queries on symptoms, diagnosis, conservative treatments, exercises, and surgical options. The chatbot generated generally coherent, readable responses that covered common knowledge on the topic, but often lacked depth, cited no sources, occasionally included inaccuracies or hallucinations, and provided overly generic or incomplete advice (e.g., on exercise prescription or when to seek specialist care). The …
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/19198800.
Short Summary of Main Findings In this December 2022 medRxiv preprint (v2), the authors evaluated the performance of an early AI chatbot (primarily ChatGPT) in providing medical information and treatment options for shoulder impingement syndrome. They submitted a series of structured queries on symptoms, diagnosis, conservative treatments, exercises, and surgical options. The chatbot generated generally coherent, readable responses that covered common knowledge on the topic, but often lacked depth, cited no sources, occasionally included inaccuracies or hallucinations, and provided overly generic or incomplete advice (e.g., on exercise prescription or when to seek specialist care). The study highlighted both the potential accessibility of AI for patient education and its current limitations in reliability for musculoskeletal disorders.
How This Work Has Moved the Field Forward It represents one of the earliest documented evaluations of ChatGPT in a specific orthopedic/physiotherapy context (shoulder impingement), shortly after the tool's public release. This helped spark the subsequent wave of research on large language models (LLMs) in patient education, clinical decision support, and medical information dissemination, contributing to growing awareness of both opportunities and risks of AI chatbots in healthcare.
Major Issues
Remains an unreviewed preprint with no identified peer-reviewed journal publication.
Very early evaluation of a rapidly evolving tool (ChatGPT-3.5 era); findings are now largely outdated given major model improvements.
Subjective and non-standardized evaluation methods (no clear scoring rubric, inter-rater reliability, or comparison with gold-standard sources like clinical guidelines).
Small scope (single condition, limited queries) and lack of clinical validation or real-patient outcomes.
Minor Issues
Title is long and somewhat wordy.
Limited discussion of ethical, liability, or misinformation risks.
No quantitative metrics (e.g., accuracy percentages, readability scores) in some sections; results rely heavily on qualitative description.
Competing interests
The author declares that they have no competing interests.
Use of Artificial Intelligence (AI)
The author declares that they did not use generative AI to come up with new ideas for their review.
-
