Deep Learning Improves Parameter Estimation in Reinforcement Learning Models

Reinforcement learning models are widely used in psychology and neuroscience to study cognitive processes underlying decision-making. However, accurately and reliably estimating model parameters is often challenging due to factors ubiquitous in cognitive modeling, such as limited data, measurement noise, and model complexity, hindering the interpretation of these parameters in behavioral and neural data. Here we evaluate whether the deep learning method — combining neural networks with modern optimization techniques — can improve parameter estimation compared to the Nelder–Mead method ( fminsearch ), a de facto baseline in cognitive modeling. We perform a systematic comparison using ten distinct value-based decision-making datasets, in which humans and animals perform various bandit tasks. Although both methods achieve comparable predictive performance, they yield different parameter estimates. Notably, parameters obtained through the deep learning method consistently exhibit smaller gaps between training and testing performance (improved generalizability), greater resilience to parameter perturbations (enhanced robustness), superior performance in recovering ground-truth parameters when data availability is constrained (better identifiability), and higher consistency across repeated measurements from the same subject (improved test-retest reliability). Our study highlights the critical importance of systematically assessing generalizability, robustness, and identifiability to ensure meaningful interpretations of cognitive model parameters and advocates for adopting the deep learning method for parameter estimation in cognitive modeling.