Gaming the peer review system: a sophisticated review mill in medicine highlights the need to ensure reviewer integrity
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
A review mill is a network of researchers who game the peer review system to apparently boost their citations. Members write generic review reports containing suggestions for citations to the work of those in the review mill. Here we report compelling evidence for a review mill in the field of gynecologic oncology.
Methods
The first example of a peer-review report from a review mill was observed by chance. It contained ‘boilerplate’ comments, such as: “Methodology is accurate and conclusions are supported by the data analysis” as well as suggestions that specific PubMed IDs be cited. We searched the internet using Google for review reports using the same boilerplate. We coded all review text to quantify similarities between review reports and compiled a list of citations suggested by reviewers. For 59 of 119 nonanonymous reviews in this target set, we identified a second review of the same article to act as a comparison.
Findings
We identified a set of 195 review mill reports that shared verbatim or highly similar boilerplate text from 170 targeted articles. 186 reports suggested citing at least one article which was co-authored by the reviewer or another member of the mill. Authors of 142 articles complied with some or all suggestions for citation. Nine of the reviewers in this review mill contributed 5 or more reviews; four had acted as editors for articles in the target set, and five had prolific peer review histories on Web of Science. Boilerplate text and self-citation recommendations were rare in the comparison reports.
Interpretation
Review mills threaten both the scientific record and patient safety when clinically relevant articles are improperly scrutinized during peer-review. We recommend that publishers adopt open peer review and transparently report the editors responsible for handling papers, to make it easier to detect review mills.
