A Post-Publication Observatory to Enable Meta-Reviews of Covid-19 Science

The Covid-19 pandemic has led to an unprecedented surge in scientific publications, many of which have subsequently faced criticism regarding their analytical integrity. Aiming to foster collaborative scientific discourse, we introduce the prototype of an observatory with over 40 Covid-19 studies and a selection of published responses for each study. Through several case studies from the observatory, we examine specific instances of methodological flaws, data integrity issues, and challenges of maintaining scientific rigor that passed through peer review but were uncovered in the responses. The prototype demonstrates the potential for creating a crowd-sourced comprehensive repository of post-publication reviews and supports the creation of “meta-reviews” that integrate multiple critiques of a study. It also reveals limitations such as resource constraints and the complexities involved in identifying influential articles and valid responses. Future developments include refining the selection and ordering criteria, while encouraging greater engagement from the scientific community and informed public with the ultimate goal to enhance the reliability of published research.