Code-sharing policies are associated with increased reproducibility potential of ecological findings

  1. Alfredo Sánchez-Tójar
  2. Aya Bezine
  3. Marija Purgar
  4. Antica Culina

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

Software code (e.g., analytical code) is increasingly recognised as an important research output because it improves transparency, collaboration, and research credibility. Many scientific journals have introduced code-sharing policies; however, surveys have shown alarmingly low compliance with these policies. In this study, we expanded on a recent survey of ecological journals with code-sharing policies by investigating sharing practices in a comparable set of ecological journals without code-sharing policies. Our aims were to estimate code- and data-sharing rates, assess key reproducibility-boosting features, such as the reporting of software versioning, and compare reproducibility potential between journals with and without a code-sharing policy. We reviewed a random sample of 314 articles published between 2015 and 2019 in 12 ecological journals without a code-sharing policy. Only 15 articles (4.8%) provided analytical code, with the percentage nearly tripling over time (2015-2016:2.5%, 2018-2019:7.0%). Data-sharing was higher than code-sharing (2015-2016:31.0%, 2018-2019:43.3%), yet only eight articles (2.5%) shared both code and data. Compared to a comparative sample of 346 articles from 14 ecological journals with a code-sharing policy, journals without a code-sharing policy showed 5.6 times lower code-sharing, 2.1 times lower data-sharing, and 8.1 times lower reproducibility potential. Despite these differences, the key reproducibility-boosting features of the two journal types were similar. Approximately 90% of all articles reported the analytical software used; however, for journals with and without a code-sharing policy, the software version was often missing (49.8% and 36.1% of articles, respectively), and exclusively proprietary (i.e., non-free) software was used in 16.7% and 23.5% of articles, respectively. Our study suggests that journals with a code-sharing policy have greater reproducibility potential than those without. Code-sharing policies are likely to be a necessary but insufficient step towards increasing reproducibility. Journals should prioritize adopting explicit, easy-to-find, and strict code-sharing policies to facilitate researchers' compliance and should implement mechanisms such as checklists to ensure adherence.

Article activity feed

  1. Version published to 10.24072/pcjournal.541
  2. Peer Community in Ecology

    Researchers do not live in a vacuum, and the social context we live in affects how we do science. On one hand, increased competition for scarce funding creates the wrong incentives to do fast analysis, leading sometimes to poorly checked results that accumulate errors (Fraser et al. 2018). On the other hand, the actual challenges the world faces require more than ever robust scientific evidence that can be used to tackle the current rapid human-induced environmental change. Moreover, scientists' credibility is at stake at this moment where the global flow of information can be politically manipulated, and accessing reliable sources of information is paramount for society. At the crossroads of these challenges is scientific reproducibility. Making our results transparent and reproducible ensures that no perverse incentives can compromise our findings, that results can be reliably applied to solve relevant problems, and that we regain societal credibility in the scientific process. Unfortunately, in ecology and evolution, we are still far from publishing open, transparent, and reproducible papers (Maitner et al. 2024). Understanding which factors promote increased use of good practices regarding reproducibility is hence very welcome.

    Sanchez-Tojar and colleagues (2025) conducted a (reproducible) analysis of code and data-sharing practices (a cornerstone of scientific reproducibility) in journals with and without explicit policies regarding data and code deposition. The gist is that having policies in place increases data and code sharing. Doing science about how we do science (meta-science) is important to understand which actions drive our behavior as scientists. This paper highlights that in the absence of strong societal or personal incentives to share code and data, clear policies can catalyze this process. However, in my opinion, policies are a needed first step to consolidate a more permanent change in researchers' behavior regarding reproducible science, but policies alone will not be enough to fix the problem if we do not change also the cultural values around how we publish science. Appealing to inner values, and recognizing science needs to be reproducible to ensure potential errors are easily spotted and corrected requires a deep cultural change. 

    References

    Fraser, Hannah, Tim Parker, Shinichi Nakagawa, Ashley Barnett, and Fiona Fidler. "Questionable research practices in ecology and evolution." PloS one 13, no. 7 (2018): e0200303. https://doi.org/10.1371/journal.pone.0200303

    Maitner, Brian, Paul Efren Santos Andrade, Luna Lei, Jamie Kass, Hannah L. Owens, George CG Barbosa, Brad Boyle et al. "Code sharing in ecology and evolution increases citation rates but remains uncommon." Ecology and Evolution 14, no. 8 (2024): e70030. https://doi.org/10.1002/ece3.70030

    Alfredo Sánchez-Tójar, Aya Bezine, Marija Purgar, Antica Culina (2025) Code-sharing policies are associated with increased reproducibility potential of ecological findings. EcoEvoRxiv, ver.4 peer-reviewed and recommended by PCI Ecology. https://doi.org/10.32942/X21S7H

  3. Version published to 10.32942/x21s7h