Decisions in the Balance: How Different Representations of Uncertainty Impact Judgements about Risk

IntroductionPast research has shown that different representations of uncertainty can lead to differences in task performance and decision making. However, it can be difficult to determine whether these findings will generalize to different data types, tasks, and domains. Our goal in this study was to determine how different representations of uncertainty for time series data would impact viewers’ perceptions of risk and their subsequent decisions.Data CollectionWe ran four within-subjects behavioral experiments. Participants completed a mock Public Safety Power Shutoff task, in which they used uncertain wind speed forecasts to decide whether or not to take a protective action (shutting down an electrical circuit) to lower the risk of a wildfire. All four experiments manipulated the riskiness of the situation by changing the probability of the wind speed crossing a critical threshold. Experiment 1 compared 15 visual representations of uncertainty and five text representations of uncertainty within the same task. Experiments 2 and 3 focused only on visual representations but added interactivity, allowing participants to select which representations to view (Experiment 2) or to combine multiple representations (Experiment 3). Experiment 4 tested whether the effects observed in Experiments 1-3 would extend to professional electric grid operators who had experience with real PSPS scenarios.Data AnalysisThe data were analyzed using mixed effects models with Tukey corrections for multiple comparisons on estimated marginal means. For each experiment, we assessed how the different representations of uncertainty impacted participants’ decisions at each risk level. ResultsDifferent visualizations of the same sets of wind speed forecasts could lead to very different patterns of decisions. The participants’ decisions were primarily driven by simple visual heuristics, such as the visual width of the uncertainty representation and whether any part of that representation crossed the critical threshold. This effect persisted even when the participants were allowed to interact with the visualizations and view multiple representations of uncertainty. The same effect also appeared for subject matter experts, who made more risk-averse decisions overall, but were still influenced by the visual cues.ConclusionSimple visual cues in plots of time series data can have a substantial impact on viewers’ perceptions of risk. This can be a positive or a negative effect, depending on the likelihood and consequences of different outcomes. These results highlight the need for additional research on how to design representations of uncertainty that strike an appropriate balance for viewers who must reason about different types of risks and outcomes. Materialshttps://osf.io/z39wd/