Adversarial reanalysis and the challenge of open data in regulatory science

Over the last decade, the transparency of the scientific data supporting environmental regulations has been the subject of partisan conflict at the highest levels of the United States’ government. Recent attempts by conservatives, industry groups, and dissident scientists to promulgate transparency requirements for such data have been described by partisan opponents and critical data studies scholars alike as ‘Trojan Horses’ designed to exclude nonreplicable studies from environmental decision making. However, if we distinguish between replication (performing a study de novo) and reanalysis (re-examining data from an existing study to confirm or challenge its conclusions), several things become clear. First, these recent clashes are part of a long story, stretching back at least to the 1990s, that is centered on data access for reanalysis: while some actors may be interested in preventing the consideration of certain data, others are interested in reanalyzing data to re-examine the scientific consensus, and in fact have done so. Second, using procedure or methodological quality to adjudicate reanalyses often fails, leading to a recourse to reputational markers that amplifies rather than resolves the lack of epistemic closure. Because the transparency of regulatory data has been a bipartisan priority over the last few decades, we anticipate that adversarial reanalyses will only become more common. The aim of transparent, open scientific data is a laudable one, but as data become more open, advocates, scientists, and policymakers alike will have to be mindful of how to make decisions in the absence of scientific consensus.