Testing Behavior Dynamic Models to Predict Choice in a Non-Stationary Two-Choice Foraging Environment

Between the 1970s-1990s, researchers proposed several dynamic models of choice to explain how organisms adjust responding under changing contingencies (e.g., melioration, kinetic model, behavioral momentum, hill-climbing). Although widely debated, these models were rarely tested against one another on the same dataset, and alternative models from related sciences similarly predict individual choice based on contact with reinforcement. The present study revisits this literature by fitting 22 models to trial-level choice data from 60 humans completing a four-phase, non-stationary, two-option foraging task with regime shifts, contingency reversals, and perturbation probes. Models included three baseline heuristics, six historical models, three reinforcement-learning (RL) models, three hidden Markov models (HMMs), three foraging models, and four empirical dynamical models (EDM). Using AIC, HMMs with 3-4 latent states performed best (mean AIC = −2,010 to −4,183) followed by RL (≈ 1,040), historical models (≈ 1,052), then foraging models (≈1,216). Next-response predictive accuracy showed a similar pattern: HMMs performed best (≈ .88), followed by EDM (.74), RL (.69), historical (.66), foraging (.66) and baseline models (.53). Overall, state-switching architectures may better describe and predict choice in dynamic contexts where steady-state behavior is unlikely to emerge. Regardless, much work remains to predict response-by-response behavior dynamics across varying environmental contexts.