Engaging doctoral students in peer review: a pre-post study evaluating the effectiveness of the “Peerspectives” course on review quality, knowledge and skills
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
To assess the effectiveness of the Peerspectives course on editor-judged quality of peer review
Design
Single arm, pre-post interventional study
Setting
Peerspectives is a peer review training course developed for doctoral students at the Charité – Universitätsmedizin Berlin, Germany
Participants
Doctoral students in health research domains who enrolled in the Peerspectives course between October 2020 and August 2022 and consented to participate ( N = 80). Intervention: A semester-long course (approx. 18 weeks) providing training about the structure, purpose, and conduct of peer review and editing processes in biomedical journals. The course consisted of 12 hours of interactive lectures, weekly homework assignments, and 12 hours of hands-on, small-group workshops under guidance of mentors with reviewing and editing experience. Students peer-reviewed real scientific manuscripts under review at a partnering journal, The BMJ .
Main outcome measures
The overall quality of the peer review reports as judged by two independent BMJ editors using the global score of the Review Quality Instrument (RQI) pre- and post-intervention. Additionally, participants’ self-assessed knowledge and skills related to scholarly peer review (1-5 Likert scale).
Results
After course completion, participants’ RQI scores were higher than before the course (median increase of 0.5 points, p<0.001; mean increase of 0.36 points, p<0.001). The RQI scores of participants’ post-course reports were not non-inferior to those of actual BMJ reviewers for the same manuscripts. Self-assessed peer review-related knowledge skills increased across all questionnaire items after course completion. Largest improvements were seen in understanding what is expected from a reviewer (increase in means from 2.9 to 4.5), confidence in own ability to review (2.5 to 3.9), and knowing what to look for in a manuscript while reviewing (2.8 to 4.2).
Conclusions
Providing doctoral students with comprehensive training resulted in an editorially significant increase in review report quality and improved understanding of the role and expectations of peer reviewers in the scholarly publishing processes and confidence in giving constructive feedback.
Study pre-registration: https://osf.io/vndcx
