Knowing When to Stop: Optimal stopping in sequential decision-making

Making an informed decision often depends on observing sequential outcomes, which may be temporally spaced across a sequence of immediate decisions or decisions made over weeks or months. Making decisions based on multiple observed sequential outcomes is the cornerstone of optimal stopping theory. Here, we review and describe four optimal stopping problems: The Secretary Problem, The Burglar Problem, The Devil’s Task, and the Game of Pigs. We outline their respective optimal policies and draw comparisons between these problems. Simulations of the standard Secretary Problem highlight the generalizability of its optimal policy across several different set sizes and show that with larger set sizes, the optimal policy stabilizes. Conversely, simulating the Burglar Problem illustrates how the outcome of pseudo-optimal policies can be directly tied to assumptions of the choice environment. A brief discussion of the usefulness of optimal stopping policies in the real world is also outlined. Using sequential decision tasks with optimal policies may provide a better understanding of how people make decisions, deviate from optimality, and help inform how to make people more accurately engage in optimality.