Assimilation to External Cues: Comparing the Reliability of Anchoring Effects, Advice Taking, and Hindsight Bias and Exploring its Determinants

People’s quantitative estimates, such as judgments about the height of a tower or the distance between two cities, can be influenced by external values, including random numbers in the environment or another person’s estimate. We reviewed three distinct but related phenomena in quantitative estimation: anchoring effects, advice taking, and hindsight bias. These effects are assimilative, meaning estimates are drawn toward the external value. They are also robust and they exhibit medium to large effect sizes. However, previous studies suggest that the reliability of their measurements differs considerably, ranging from low (anchoring) to high (advice taking). In this project, we compared the reliabilities of the three phenomena using open data (Study 1) and a new experiment employing a homogenized paradigm (Study 2). Homogenization involves controlling for differences in the informativeness of external values, computing assimilation scores, and using the same setting and knowledge domain. In Study 1, we found that anchoring effects were largely unreliable, advice taking was reliably measured, and hindsight bias showed moderate reliability, consistent with prior research. This overall pattern was [reproduced] in [the simulation of] Study 2. We conclude that although methodological factors contribute to variability, they alone cannot explain differences in reliability.