Impact of a Mentored Peer-Review Program on the Sponsoring Journal: Program evaluation at 5 Years

Background Growth in the number of academic medical journals over the past 25 years has increased the demand for quality peer reviewers. Shortages of qualified peer reviewers are partially attributable to a lack of formal training. We developed a formal mentored peer review (MPR) program to recruit and train peer reviewers. The objective of this paper is to evaluate the impact of this program on the sponsoring journal. Methods We compared the peer-reviews generated by mentored medical education fellows from emergency medicine fellowships throughout the United States to peer-reviews generated by our traditional peer reviewers on two key metrics: quality of the peer review and time to accept and then submit the review. Additionally, we report the number of participating education fellowships and their graduates who continued as peer reviewers once they completed their fellowships. Results Over 100 fellows were mentored by 43 fellowship directors who submitted 147 MPRs over a five-year period. We retained 21 MPR graduates and counted them as peer reviewers once they had completed their fellowships. Mentored peer reviews received significantly higher editorial ratings on review quality than traditional reviewers and independent reviews by previously participating fellowship graduates (p < .001). In addition, compared with traditional reviewers, MPRs reduced the time needed to obtain two reviewers by 6 weeks (p < .001). Discussion The MPR program contributed to improving the quality of peer reviews and increasing the size of the journal’s reviewer pool. Although the time to obtain two accepting reviewers decreased, there was no difference between MPRs and controls in submitting the accepted reviews. Peer reviews by graduates of the MPR program were found to be comparable in quality to our journal’s traditional peer reviewers. By contributing to the formal education of academic medical faculty related to peer review, such an MPR program has the potential for other journals, particularly those related to medical specialties, to meet the demand of academic medical journals for quality peer reviewers.