Investigating the replicability of the social and behavioural sciences

  1. Andrew Harold Tyner
  2. Anna Lou Abatayo
  3. Macie Daley
  4. Samuel Field
  5. Nicholas William Fox
  6. Noah Haber
  7. Krystal Hahn
  8. Melissa Kline Struhl
  9. Brinna Mawhinney
  10. Olivia Miske
  11. Priya Silverstein
  12. Courtney K. Soderberg
  13. Theresa Stankov
  14. Ahmed Abbasi
  15. Chris Aberson
  16. Balazs Aczel
  17. Matus Adamkovic
  18. Nihan Albayrak
  19. Peter Allen
  20. Michael Andreychik
  21. Eli C. Awtrey
  22. Erick Axxe
  23. Flavio Souza Azevedo
  24. Miles Bader
  25. Bence Bago
  26. James Bailey
  27. Marjan Bakker
  28. Gabriel Baník
  29. George Christopher Banks
  30. Ernest Baskin
  31. Sophie M. Behr
  32. Nicholas Berente
  33. Zachariah Berry
  34. Jedrzej Bialkowski
  35. Bojana Bodroža
  36. Laura Boeschoten
  37. Miklos Bognar
  38. Christian Bokhove
  39. Diane B. V. Bonfiglio
  40. Robin B Bouwman
  41. Timothy F. Brady
  42. Scott R. Braithwaite
  43. Gabriel Briceño Jiménez
  44. Cameron Brick
  45. Traci Bricka
  46. Roman Briker
  47. Annette Nicole Brown
  48. Gordon David Alexander Brown
  49. Robbie Cornelis Maria van Aert
  50. Kathryn Caldwell
  51. Sara Capitan
  52. Tabaré Capitán
  53. Jesse J. Chandler
  54. Tessa Charles
  55. Christopher R. Chartier
  56. Rahul Chawdhary
  57. Kent Jason Go Cheng
  58. William J. Chopik
  59. Bruce Clark
  60. Victoria E. Colvin
  61. Christine Cozette Comer
  62. Giulio Costantini
  63. Tom Coupe
  64. Jamie Cummins
  65. Aneta Czernatowicz-Kukuczka
  66. Joshua R de Leeuw
  67. David Dobolyi
  68. James Druckman
  69. Jianhua Duan
  70. Marin Dujmović
  71. Daniel J. Dunleavy
  72. Patrick Durkee
  73. Cécile Emery
  74. Kevin Esterling
  75. Thomas Rhys Evans
  76. Anna Fedor
  77. Belén Fernández
  78. Nathan Fiala
  79. James G. Field
  80. Nathan Fong
  81. Miguel Fonseca
  82. Alexandra LJ Freeman
  83. Jeremy Freese
  84. Sandra Jeanette Geiger
  85. Jing Geng
  86. Laura Marie Getz
  87. Linda Geven
  88. Ilka Helene Gleibs
  89. Donna Pamella Gonzales
  90. Janaki Gooty
  91. Amélie Gourdon-Kanhukamwe
  92. Cristina Greculescu
  93. Siobhan Griffin
  94. Lusine Grigoryan
  95. Martina Grunow
  96. Nicholas Gunby
  97. Braeden Forrester Hall
  98. Paul H. P. Hanel
  99. Erin Hannon
  100. Sam Harper
  101. Marco Juergen Held
  102. Louis Hickman
  103. Nathan C. Higgins
  104. Svenja Hippel
  105. Sven Hoeppner
  106. Sanghyun Hong
  107. Tom Hostler
  108. Michael Inzlicht
  109. Kamil Izydorczak
  110. Bastian Jaeger
  111. Kristin Jankowsky
  112. Johannes Jarke-Neuert
  113. Matthew Jensen
  114. Biljana Jokic
  115. Daniel Jolles
  116. Phillip Jolly
  117. Angela Jones
  118. Marie Juanchich
  119. Pavol Kačmár
  120. Hansika Kapoor
  121. Andjela Keljanovic
  122. Samjhana Koirala
  123. Marta Kołczyńska
  124. Dimitra Kouroupaki
  125. Ulrich Kühnen
  126. Michelangelo Geovanny Landgrave
  127. Michael J. Larson
  128. Lyonel Laulié
  129. Alice Charlotte Emily Lawrence
  130. Joel Michel Le Forestier
  131. Katelin E. Leahy
  132. Sungmok Lee
  133. Jared Leslie
  134. Savannah C. Lewis
  135. Angelo Christopher Limnios
  136. Hause Lin
  137. An-Chiao Liu
  138. John Wills Lloyd
  139. Elliot Andrew Ludvig
  140. Dermot Lynott
  141. Jordan MacDonald
  142. Peter Robert Mallik
  143. Daniel J. Mallinson
  144. Daniele Marinazzo
  145. Corinna Martarelli
  146. Joshua Jason Matacotta
  147. Andrew McBride
  148. Cillian McHugh
  149. Gail McMillan
  150. Esteban Méndez-Chacón
  151. Mitchell Metzger
  152. Michalis Michaelides
  153. Johannes Michalak
  154. Leticia Micheli
  155. Jeremy K. Miller
  156. Marina Milyavskaya
  157. Daniel C. Molden
  158. Ambar Monjaras
  159. David Moreau
  160. Audrey Morrow
  161. Cristóbal Moya
  162. Liad Mudrik
  163. Laetitia B. Mulder
  164. Katie A Munt
  165. Arijit Nandi
  166. Kathryn Nason
  167. Carolin Nast
  168. Gideon Nave
  169. Heinrich H Nax
  170. Florian Neubauer
  171. Phuong Linh Le Nguyen
  172. Austin Lee Nichols
  173. Gustav Nilsonne
  174. Ernest O'Boyle
  175. Jule Oettinghaus
  176. Jeewon Oh
  177. Adoril Oshana
  178. Thomas Ostermann
  179. Rachel P. Ostrowski
  180. Abiola OYEBANJO
  181. Radoslaw Panczak
  182. Jamie Patrianakos
  183. Ignacio Pavez
  184. Yuri Pavlov
  185. Sofia Persson
  186. Marco Perugini
  187. Kim Peters
  188. Constant Pieters
  189. Vladimir Ponizovskiy
  190. Nathaniel D Porter
  191. Danka Purić
  192. Mariah Purol
  193. Arathy Puthillam
  194. Kimberly Quinn
  195. Marco Ramljak
  196. W. Robert Reed
  197. Michaela Ritchie
  198. Margaret Ritzau
  199. Sean Patrick Roche
  200. Romina Rodela
  201. Jan Philipp Röer
  202. Ivan Ropovik
  203. Jacob Rothschild
  204. Justine Saal
  205. Hani Safadi
  206. Jason Samaha
  207. Mary Sanchez
  208. Soorya Sankaran
  209. David Santos
  210. Amanda Sargent
  211. Marian Sauter
  212. Kathleen Schmidt
  213. Landon Schnabel
  214. Amber Schroeder
  215. Sebastian W Schuetz
  216. Michael Schulte-Mecklenbeck
  217. Astrid Schütz
  218. Eric L. Sevigny
  219. Ellie Shackleton
  220. Richard Shafranek
  221. Samuel Shaki
  222. Shishir Shakya
  223. Miroslav Sirota
  224. Matthew Ryan Sisco
  225. Maksim Sitnikov
  226. L. Robert Slevc
  227. Laura Smalarz
  228. Colin Smith
  229. Joel S. Snyder
  230. Nicolas Sommet
  231. Fatih Sonmez
  232. Bobbie Spellman
  233. Natalia Stanulewicz-Buckley
  234. George Stock
  235. Chris N. H. Street
  236. Eirik André Strømland
  237. Tina Sundelin
  238. Moin Syed
  239. Anna Szabelska
  240. Barnabas Szaszi
  241. Ewa Szumowska
  242. Anirudh Tagat
  243. Susanne Täuber
  244. Louis Tay
  245. Stuti Thapa
  246. Jason Thatcher
  247. Domna Tsaklakidou
  248. Lars Tummers
  249. Elise Turkovich
  250. Melba Tutor
  251. Karolina Urbanska
  252. Anna Elisabeth van 't Veer
  253. Marcel A. L. M. van Assen
  254. Niels van de Ven
  255. Ruben Van den Goorbergh
  256. Elisabeth Julie Vargo
  257. Leigh Ann Vaughn
  258. Simine Vazire
  259. Jentien Vermeulen
  260. Diem Thi Hong Vo
  261. Victor Volkman
  262. Eric-Jan Wagenmakers
  263. Deliah Wagner
  264. Lukasz Walasek
  265. Frank Walter
  266. Lara Warmelink
  267. Liuqing Wei
  268. Marie Isabelle Weißflog
  269. Nicholas Weller
  270. Aaron Lee Wichman
  271. Jonathan Wilbiks
  272. Jamal Rodgers Williams
  273. Kelly Wolfe
  274. Finnian Wort
  275. Ryan Wright
  276. Jesper N. Wulff
  277. XINDONG xue
  278. Veronica X. Yan
  279. Yuzhi Yang
  280. Sangsuk Yoon
  281. Iris Zezelj
  282. Yinxian Zhang
  283. Ignazio Ziano
  284. Cristina Zogmaister
  285. Zorana Zupan
  286. Rolf A. Zwaan
  287. Brian A. Nosek
  288. Timothy M. Errington
Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

We attempted replications of 274 claims of positive results from 164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioral sciences. Replications were high-powered on average to detect the original effect size (Median = 99.6%), used original materials when relevant and available, and were peer-reviewed in advance through a standardized internal protocol. Replications showed statistically significant results in the original pattern for 151 of 274 claims (55.1% [95% CI 49.2 - 60.9%]) and for 80.8 of 164 papers (49.3% [95% CI 43.8 - 54.7%]) weighed for replicating multiple claims per paper. We observed modest variation in replication rates across disciplines (42.5% to 63.1%) though some estimates had high uncertainty. For claims with Pearson’s r effect sizes, the median was 0.25 [95% CI 0.21 - 0.27] for original studies and 0.10 [95% CI 0.09 - 0.13] for replication studies, a 82.4% [95% CI 67.8 - 88.2%] reduction in shared variance. Thirteen methods for evaluating replication success provided estimates ranging from 28.6% to 74.8% (median = 49.3%). Some decline in effect size and significance is expected based on power to detect original effects and regression to the mean due to replicating only positive results. The conditions that promote or inhibit replicability are worthy of additional investigation.

Article activity feed

  1. Version published to 10.31222/osf.io/b7x8f_v1 on OSF Preprints