Describing the landscape of medical education preprints on medRxiv

Abstract

Introduction

A preprint is a version of a research manuscript posted to a preprint server prior to peer review. Preprints enable authors to quickly and openly share research, afford opportunities for expedient feedback, and enable immediate listing of research on grant and promotion applications. In medical education, most journals welcome preprints, suggesting they play a role in the field’s discourse. Yet, little is known about medical education preprints, including author characteristics, use, and ultimate publication status. This study provides an overview of preprints in medical education in an effort to better understand their role in the field’s discourse.

Methods

The authors queried medRxiv, a preprint repository, to identify preprints categorized as Medical Education and downloaded the related metadata. CrossRef was queried to gather information on preprints later published in journals.

Results

Between 2019-2022, 204 preprints were classified in medRxiv as Medical Education with most deposited in 2021 (n=76, 37.3%). On average, preprint full-texts were downloaded 1875.2 times, and all were promoted on social media. Preprints were authored, on average, by 5.9 authors. Corresponding authors were based in 41 countries with nearly half (45.6%) in the United States, United Kingdom, and Canada. Almost half (n=101, 49.5%) of preprints became published articles in predominantly peer-reviewed journals. Preprints appeared in 65 peer-reviewed journals with BMC Medical Education (n=9, 8.9%) most represented.

Discussion

Medical education research is being deposited as preprints, which are promoted, heavily accessed, and subsequently published in peer-reviewed journals, including those specific to medical education. Considering the benefits of preprints and slowness of medical education publishing, it is likely that preprint deposition will increase and preprints will be integrated into the field’s discourse. Based on these findings, we propose next steps to facilitate the responsible and effective creation and use of preprints in medical education.

This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/10210743.

This review reflects comments and contributions from Melissa Chim, Stephen Gabrielson, Allie Tatarian, Jessica Polka, and Dibyendu Roy Chowdhury. Review synthesized by Stephen Gabrielson.

This study investigates the landscape of medical education preprints on medRxiv. It aims to provide an overview of these preprints, focusing on their characteristic, impact, dissemination, and transition within the field of medical education.

Major comments:

The study highlights the growing importance of preprints in medical education. While the conclusion in the paper provides a good summary of directions going forward for preprinting in this field, a more critical examination of the implications of these trends for educational policy and practice, especially given the lack of peer review in preprints, would be beneficial.

Minor comments:

In the third paragraph of the Methods section, the authors note the limitations of using Genderize.io. A significant limitation of the program is missing, however: this program has trouble analyzing non-Western first names, as described in this article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8485937. This Western bias should be pointed out. Also in the same paragraph, if the authors are acknowledging limitations to their approaches in this section, it may be helpful to note that Crossref matches are incomplete, eg: https://link.springer.com/article/10.1007/s11192-021-03900-7.
I appreciated the section on Creative Commons licenses, as they're an important part of any discussion on open sources. Would the authors be able to comment on why more than half of preprints had no CC license? If preprinting becomes a larger part of scholarly communication training in the medical education field, an understanding of CC licenses will be an important aspect for researchers to learn.
In the Authorship section, I would be interested in knowing the career stage of authors, although I recognize that this would be difficult to infer from preprint metadata alone. Is preprinting helping postdocs and junior faculty to get their work out there?
In the last paragraph of the Discussion section, the authors discuss how preprints can be published in peer reviewed journals, and state that "Reports of this practice from outside of medical education suggest that between 14%⁴³ and 42%⁴⁴ of those papers deposited as preprints eventually get published in academic journals." These values seem low to me, see https://link.springer.com/article/10.1007/s11192-021-03900-7 again and also https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0281659.
When explaining how preprints are not peer reviewed, it would be worth mentioning how medRxiv screens preprints before posting them, even if it's not as rigorous as peer review. https://connect.medrxiv.org/news/2022/06/13/screening_procedures.

Comments on reporting:

The statistical analysis in this paper (as explained in the Results > Preprint Use section) does not acknowledge the existence of extreme outliers in their data - it appears that there are 2 or 3 medical education papers on medRxiv that were extremely popular, and their popularity has skewed the averages reported in this paper. I would like to see what the averages look like with those outliers removed, including the Altmetric and Twitter data. The upper portions of these ranges are all 12-14 standard deviations from the mean.
In regards to the above, in the second paragraph in the Discussion section the authors describe page views from a 2018 study and compare it to the findings of this preprint, which report higher numbers. This would be a more convincing argument without the outliers that are skewing the averages. The 2018 study that is cited here does not include a range, but their standard deviation is much smaller than the standard deviation in this paper (580.7 vs 15,863.44), which makes me feel like their data is less likely to be strongly affected by extreme outliers. Specifically for PDF downloads, without the top two extreme outliers, the average number of PDF downloads goes down to 124.84--well under the average in the paper the authors have cited here.

Suggestions for future studies:

A qualitative analysis of social media interactions around these preprints could provide interesting and deeper insights into public perception.
A future study might focus on medical education authors who reside outside the United States, United Kingdom, and Canada.

Competing interests

The author declares that they have no competing interests.

Read the original source