Authors self-disclosed use of artificial intelligence in research submissions to 49 biomedical journals: A cross-sectional study
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
OBJECTIVE
To analyze the frequency of self-disclosed use of AI in research manuscripts submitted to 49 biomedical journals and to identify types of AI tools used, the tasks they assisted with, and factors associated with disclosure.
DESIGN
Cross-sectional study.
SETTING
49 biomedical journals published by BMJ Group.
PARTICIPANTS
Submitting authors of 25,114 empirical research manuscripts including systematic reviews and meta-analyses, submitted between 8 April 2024 and 6 November 2024.
MAIN OUTCOME MEASURES
Prevalence of manuscripts with self-disclosed use of AI, types of AI tools used, the tasks they assisted with and factors associated with disclosure.
RESULTS
There was a total of 25,114 eligible submissions: Asia (13,505; 53.8%), Europe (6,523; 26.0%), North America (2,795; 11.1%), Africa (1,196; 4.8%), Oceania (708; 2.8%), and South America (387; 1.5%). A total of 1,431 submissions (5.7%) disclosed the use of AI. The most frequently reported AI tools used were Generative AI Chatbots (812/1431; 56.7%) and writing assistants (182; 12.7%). The majority of authors who disclosed AI use reported using it to improve the quality of their writing (1,248/1,431; 87%). Additionally, translation (107; 7.5%), generating data and output (87, 6.1%), literature searches (49; 3.4%), analyzing data (31; 2.2%), image processing or analysis (49; 3.4%), code writing (15; 1.0%), and managing references (14; 0.9%) were mentioned as tasks AI assisted with.
Authors from South America (OR=1.75; 95%-CI: 1.22-2.49) and Europe (OR=1.28; 95%-CI: 1.14-1.45) were significantly more likely to disclose AI use than those from Asia. Conversely, each additional author reduced disclosure odds by 1% (OR=0.99, 95%-CI: 0.97-0.99). Acceptance rate, impact factor, type of journal, and peer review model) were not associated with AI use disclosure.
CONCLUSIONS
We found that only a small proportion (5.7%) of submitting authors disclosed AI use, which is substantially lower than proportions reported in surveys. Improving the quality of writing was the primarily task AI assisted with and AI Chatbots were the most commonly disclosed tool. Authors may be uncertain about what AI use requires disclosure or may be hesitant to declare it.
What is already known on this topic
Artificial intelligence is increasingly used in the conduct of scientific research and writing of publications and the way it is used is changing rapidly.
Leading publishers and editorial organizations dedicated to promoting integrity and best practices in academic publishing have issued policies requiring authors to disclose all AI use in their content.
Previous surveys show that a substantial proportion of researchers (28% to 76%) say they have used AI in their research.
What this study adds
This study of a process for mandatory declaration of AI use in research manuscripts submitted to 49 biomedical journals found a prevalence of only 5.7%, which is substantially lower than use reported in surveys.
Improving the quality of writing was the most common reported task performed by AI, and g enerative AI Chatbots are the most frequently disclosed AI tool used.
How this study might affect research, practice or policy
Current mandatory questions on AI use may have only limited value and other measures are needed to improve declaration and to support detection of undisclosed use.
