A Foraging-Theory Based Model Captures The Full Spectrum of Human Behavioral Diversity in a Classic RL Task
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Decision-making tasks involving multiple, simultaneously presented options are mainstays of cognitive neuroscience and psychology, and are increasingly important to the emerging field of computational psychiatry. Modeling approaches to these tasks overwhelmingly assume that participants make choices based on explicitly comparing the values of the presented options. Contrary to this long-held assumption, we found instead that humans employ a compare-to-threshold decision process, similar to theories of foraging, when making sequential decisions about concurrently available options. We confirmed this result in a large (1000 participant) dataset with multiple converging lines of evidence comparing both model fits and model generative performance.
Value-comparison models were restricted to a reduced area of the potential space of single-trial outcome-dependent behavior, demonstrating an intrinsic limitation in the ability to reproduce strategy diversity. Furthermore, we found that using even the best-fit value-comparison model led to a substantial, systematic bias and a compression of individual differences in reconstructed behavior compared to the foraging-based model, leading to weaker predictions of behavioral health measures. Our results imply that studies using value-comparison models to link behavior with neural activity or psychiatric symptoms may be less sensitive to individual differences than a simple alternative based on ethological foraging.
Author summary
This study challenges the core assumption of reinforcement learning (RL) models in sequential decision-making: that choices are made by comparing option values. We demonstrate that most individuals actually employ a foraging-like mechanism, where decisions to explore or exploit depend on comparing exploitation value against a threshold. The foraging model not only achieves better conventional fitness metrics (AIC, BIC) but also captures the full spectrum of reward responsiveness observed in human data, while traditional RL models represent only a restricted subset. This limitation causes RL models to compress individual differences and introduce systematic bias. Furthermore, the foraging-like model explains mental health symptoms more effectively and parsimoniously than traditional RL approaches. Our findings suggest that sequential decision-making should be reconsidered through the lens of foraging theory, potentially enhancing our understanding of both cognitive mechanisms and their relationship to psychopathology.
