The effects of communicating uncertainty around statistics, on public trust
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Uncertainty around statistics is inevitable. However, communicators of uncertain statistics, particularly in high-stakes and potentially political circumstances, may be concerned that presenting uncertainties could undermine the perceived trustworthiness of the information or its source. In a large survey experiment (Study 1; N = 10 519), we report that communicating uncertainty around present COVID-19 statistics in the form of a numeric range (versus no uncertainty) may lead to slightly lower perceived trustworthiness of the number presented but has no impact on perceived trustworthiness of the source of the information. We also show that this minimal impact of numeric uncertainty on trustworthiness is also present when communicating future, projected COVID-19 statistics (Study 2; N = 2,309). Conversely, we find statements about the mere existence of uncertainty, without quantification, can reduce both perceived trustworthiness of the numbers and of their source. Our findings add to others suggesting that communicators can be transparent about statistical uncertainty without undermining their credibility as a source but should endeavour to provide a quantification, such as a numeric range, where possible.
Article activity feed
-
-
SciScore for 10.1101/2021.09.27.21264202: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics not detected. Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:However, we must acknowledge some limitations. Firstly, the uncertainty ranges communicated in our messages were all drawn from existing reports. While this provides some ecological validity, it did mean we were not able to control for the magnitude of the range presented across some conditions. Previous research suggests people may be less trusting of …
SciScore for 10.1101/2021.09.27.21264202: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Ethics not detected. Sex as a biological variable not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:However, we must acknowledge some limitations. Firstly, the uncertainty ranges communicated in our messages were all drawn from existing reports. While this provides some ecological validity, it did mean we were not able to control for the magnitude of the range presented across some conditions. Previous research suggests people may be less trusting of uncertainty presented as a very large numeric range (Kreps & Kriner, 2020; van der Bles et al., 2020). More research is needed to identify the effect of range magnitude and how this varies across contexts. Secondly, Studies 2 and 3 were only conducted in UK samples. As shown in Study 1, there is between-country variation in how people respond to uncertainty. Therefore, we cannot confidently generalize our UK findings to other national contexts. Future research should seek to replicate these findings with other countries as well as investigating possible explanations for the variation seen in Study 1. Lastly, the types of uncertainty we examined in the current research were narrowly defined. Thus, our findings may not map onto to other forms or formats of uncertainty. For example, statistical uncertainty could be expressed visually—rather than as numerals—and the existence of unquantified uncertainty could be phrased in a multitude of different ways. Uncertainty can also be further attributed to different sources, with potentially differing impacts. For example, claims could be accompanied by statements about uncertainty due to ...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We found bar graphs of continuous data. We recommend replacing bar graphs with more informative graphics, as many different datasets can lead to the same bar graph. The actual data may suggest different conclusions from the summary statistics. For more information, please see Weissgerber et al (2015).
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
