FAST principles for preprint feedback

  1. Sandra Franco Iborra
  2. Jessica Polka
  3. Sara Monaco
  4. Sharon Ahmad
  5. Maryrose Franko
  6. Shriyaa Mittal
  7. Samantha Hindle
  8. R. Dyche Mullins
  9. Timothy E.J. Behrens
  10. Gautam Dey
  11. Iratxe Puebla

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

There has been strong interest in preprint commenting and review activities in recent years. Public preprint feedback can bring benefits to authors, readers and others in scholarly communication, however, the level of public commenting on preprints is still low. This is likely due to cultural barriers, such as fear by authors that criticisms on their paper will bias readers, editors and evaluators, and concerns by commenters that posting a public critique on a preprint by a more senior colleague may lead to retribution. In order to help address these cultural barriers and foster positive and constructive participation in public preprint feedback, we have developed a set of 14 principles for creating, responding to, and interpreting preprint feedback. The principles are clustered around four broad themes: Focused, Appropriate, Specific, Transparent (FAST). We describe each of the FAST principles and designate which actors (authors, reviewers and the community) each of the principles applies to. We discuss the possible implementation of the FAST principles by different stakeholders in science communication, and explore what opportunities and challenges lie ahead in the path towards a thriving preprint feedback ecosystem.

Article activity feed

  1. PREreview

    This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/7095730.

    Summary of the work:

    In this work authors have provided a list of principles for a better preprint feedback system, which not just include the author, reviewer or journals rather everyone in a scientific/non scientific community.  The problem they asses here is the flooding of preprints in the preprint servers as seen during covid pandemic leading to issues in management of information by traditional peer review process. Also its said that in the process of reviewing a paper, there are many of the comments are hidden from the public. Here they list our a set of principles called FAST for the community to comment accordingly in preprint , helping journals to review the preprints faster and in unbiased manner.

    I appreciate that this process will enhance the rigor of the scientific work carried out by the authors and may provide insightful findings or ideas from the people not from the same scientific background and will also improve collaborations.

    Comments to implement the system

    -Increasing awareness of preprint feedback system: As authors have pointed out these FAST principles are applied into many journal clubs but in order to implement it as a basic norm. We can put preprint commenting using FAST principles in  various coursework of the graduate students. Which will improve their participation and understanding the nature of scientific commenting. Also, this group of people are more socially active than the other members of the community hence leading to better spread of these principles.

    -In many places authors have mentioned the feedback process in various preprint servers as simple comments. I guess here we need to have a dedicated server where the preprints are retrieved from the preprint servers and are actively commented by various users in the community.

    -Lastly , I find preprint feedback also needs a community of science communication so that the general public can take an interest and comment.

     

  2. PREreview

    This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/6456279.

    This manuscript provides guiding principles for addressing cultural barriers to participation in preprint peer review, clustering these principles around the four themes of Focused, Appropriate, Specific and Transparent (FAST) reviews. The principles are described and the relevant actors identified. The principles are clearly laid out and provide an excellent model for carrying out peer review broadly, as well as for preprint review specifically.

    The manuscript centers on perceived cultural barriers to participation in preprint review as a likely reason for lack of participation. Many of the reasons suggested are possible reasons for people not participating, although it will require further work to identify how much of a factor these barriers are in preventing peer review by those potentially interested in the reviewing process. Significant barriers also exist in understanding and navigating the processes by which commenting can occur, especially given that many preprint servers have their own, different formats for providing commenting. That said, some of these processes are quite simple, and so this work is important for providing guiding principles in which to engage in competent, respectful and useful feedback on preprints. 

    The authors point out the overlap in some principles of peer review for journal curation vs peer review of preprints but also highlight the differences including the gatekeeping function of selecting manuscripts for publication in a publication which limits the number of papers it can publish, focusing therefore on selectivity and not simply validity and improvement of the work. This gatekeeping also extends to who can participate in the review process and this paper highlights that preprints allow for the possibility of all members of our community to provide feedback on work, self-selecting whether they have the expertise to provide feedback on the manuscript (or a specific domain of its content), rather than being controlled by subjective judgements of someone else's expertise by journal editors who provide invitations to reviewers. On this basis the authors go beyond discussing the unidirectional process of someone providing feedback on a manuscript, and include the ability for a conversation to occur between reviewers and authors, as well as the role of the wider community in interacting with this conversation and upholding community norms. This is an innovative part of this manuscript and an important new piece of the review conversation, and the FAST principles are based on this, which is a very great strength of the work.

    Another strength that the authors could highlight, especially for early-career researchers, is that preprint review avoids the unfortunate scenario in journal review where many ECR reviewers struggle to be recognized for their participation in the process as a co-reviewer, either by the invited reviewer that they work for, or the journal that solicits the review. There is a trend towards minimizing or "othering" the participation of early career researchers in journal review processes, including the creation of training programs that ECRs must pass through to be added to the reviewer database, which faculty (who may not have received any training) do not have to pass through. Preprint review does not include this barrier based on subjective assumptions about ability to carry out peer review based on career stage, and instead allows the work to speak for itself; it could be highlighted that this is a key strength of preprints for early career researchers to explore.

    The authors admit that their collection of resources related to norms and behaviors in various kinds of review is based on the group's experience and knowledge, and is not necessarily comprehensive. This leaves the door open for others to take this work and attempt to compile such a comprehensive collection, and indeed this work identifies key resources that will be helpful for others in doing so.

    Related to this issue, there is the interesting question of how far the concerns raised in the manuscript represent concerns across the whole research community, and what other concerns have not yet been uncovered. While this is not the scope of this work, it is possible that the piece could discuss how far preprinting is still a relatively niche activity in some disciplines and also concentrated within certain institutional types (biology preprinting has been shown previously to be very concentrated amongst highly research-intensive institutions in the U.S.). It would also be helpful to describe whether these concerns are representative of specific populations, or have been attributed as likely concerns to those populations, especially if done so in their absence. This is especially common for early career researchers, for example, and so it could be helpful for future work for authors to further clarify which are clear evidence-based causes for concern, and which require more work for the community to focus on in future survey-based work, for example.

    One thing that it might be interesting to expand upon is perhaps the role of preprint authors in soliciting feedback. A major cultural barrier that does not seem to be addressed here is that many authors may not be posting preprints in anticipation of feedback, and therefore there may be the heightened concern that not only has an author received critique (sometimes unpleasant itself when directly solicited as part of the journal curation process) but that it is an unexpected critique. If there are suggestions for author-initiated actions to take place, or ways to more clearly identify which preprints may be interested in reviews (e.g. should authors clearly indicate somewhere specific in a preprint that they welcome feedback based on the FAST principles?), those could be helpful additions. Currently, I am experimenting with emailing the authors first to let them know a review will appear, when I commit to starting a review. Similarly, as an author looking for preprint review explicitly myself later this year, any guidance on this would be very useful for those looking to use preprints for solicitation of feedback and improvement of research products.

    Finally, this review was itself carried out in an attempt to also follow the FAST principles, and is very useful for doing so, and I will definitely keep them as a guiding resource in upcoming review work, and encourage other individual reviewers to do so also. Much of the material has also proved useful for education to a range of audiences in my own work, including for undergraduate classes on peer review, and should prove useful for educators and a wide range of audiences. The utility of this manuscript for a wide range of audiences who may be new to the concepts of preprints and preprint review is another of its strengths.

  3. Version published to 10.31219/osf.io/9wdcq on OSF Preprints