Investigating the reproducibility of the social and behavioural sciences

  1. Olivia Miske
  2. Anna Lou Abatayo
  3. Macie Daley
  4. Mirka Dirzo
  5. Nicholas William Fox
  6. Noah Haber
  7. Krystal Hahn
  8. Melissa Kline Struhl
  9. Brinna Mawhinney
  10. Priya Silverstein
  11. Theresa Stankov
  12. Andrew Harold Tyner
  13. Matus Adamkovic
  14. Shilaan Alzahawi
  15. Saule Anafinova
  16. Eli C. Awtrey
  17. Erick Axxe
  18. James Bailey
  19. Bert N. Bakker
  20. Akshaya Balaji
  21. Gabriel Baník
  22. František Bartoš
  23. henk berkman
  24. Zachariah Berry
  25. Felix Bethke
  26. Timothy F. Brady
  27. Nate Breznau
  28. Sara Capitan
  29. Tabaré Capitán
  30. Laura Caquelin
  31. Kent Jason Go Cheng
  32. William J. Chopik
  33. Gwen-Jiro Clochard
  34. Tom Coupe
  35. Jamie Cummins
  36. Elif Gizem Demirag Burak
  37. Jianhua Duan
  38. Kevin Esterling
  39. Thomas Rhys Evans
  40. Nathan Fiala
  41. James Field
  42. Victor Gay
  43. Jing Geng
  44. Johanna Gereke
  45. Ilka Helene Gleibs
  46. Amélie Gourdon-Kanhukamwe
  47. Dmitry Grigoryev
  48. Nicholas Gunby
  49. Paul H. P. Hanel
  50. Sanghyun Hong
  51. Sean Dae Houlihan
  52. Nick Huntington-Klein
  53. Kamil Izydorczak
  54. Kristin Jankowsky
  55. Kai Jonas
  56. Pavol Kačmár
  57. Hansika Kapoor
  58. Sebastian Karcher
  59. Marta Kołczyńska
  60. David Kretschmer
  61. Ljiljana Lazarevic
  62. Katelin E. Leahy
  63. Jessica C. Lee
  64. Angelo Christopher Limnios
  65. An-Chiao Liu
  66. John Wills Lloyd
  67. Ruben Lopez-Nicolas
  68. Nigel Mantou Lou
  69. Richard Lucas
  70. Maximilian Maier
  71. Daniel J. Mallinson
  72. Marcel Martončik
  73. Michael C. McCall
  74. Nikita P Mehta
  75. Esteban Méndez-Chacón
  76. Johannes Michalak
  77. Daniel C. Molden
  78. Faisal Mushtaq
  79. Claudia Neuendorf
  80. Austin Lee Nichols
  81. Gustav Nilsonne
  82. Ernest O'Boyle
  83. Jeewon Oh
  84. Thomas Ostermann
  85. Abiola OYEBANJO
  86. Radoslaw Panczak
  87. Yuri G. Pavlov
  88. Zoran Pavlović
  89. Noemi Peter
  90. Kim Olivia Peters
  91. Nathaniel D Porter
  92. Mariah Purol
  93. Arathy Puthillam
  94. Marco Ramljak
  95. Arran T Reader
  96. W. Robert Reed
  97. Jan Philipp Röer
  98. Ivan Ropovik
  99. Alexander Olof Savi
  100. Kathleen Schmidt
  101. Landon Schnabel
  102. Eric L. Sevigny
  103. Samuel Shaki
  104. Shishir Shakya
  105. Andrew Soh
  106. Angela Somo
  107. Fatih Sonmez
  108. Eirik André Strømland
  109. Jordan Suchow
  110. Anna Szabelska
  111. Anirudh Tagat
  112. Melba Tutor
  113. Karolina Urbanska
  114. Pieter Van Dessel
  115. Elisabeth Julie Vargo
  116. Diem Thi Hong Vo
  117. Victor Volkman
  118. Ke Wang
  119. Aaron Lee Wichman
  120. Jamal Rodgers Williams
  121. Fabian Winter
  122. Ferdinand Wintermantel
  123. Nan Zhang
  124. Ignazio Ziano
  125. Cristina Zogmaister
  126. Zorana Zupan
  127. Brian A. Nosek
  128. Timothy M. Errington
Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.

Article activity feed

  1. Version published to 10.31222/osf.io/wqt9u_v2 on OSF Preprints
  2. Version published to 10.31222/osf.io/wqt9u_v1 on OSF Preprints