Excessive flexibility? Recurrent neural networks can accommodate individual differences in reinforcement learning by capturing higher-order history dependencies

Cognitive and computational modeling has been used as a method to understand the processes underlying behavior in humans and other animals. A common approach in this field involves the use of theoretically constructed cognitive models, such as reinforcement learning models. However, human and animal decision-making often deviates from the predictions of these theoretical models. To capture characteristics that these cognitive models fail to account for, artificial neural network (ANN) models have recently gained attention. One of the roles of ANNs in cognitive and computational modeling is to provide a (upper) benchmark of predictive accuracy for cognitive models, helping ensure that no important factors are missing in theoretical models. For modeling choice behavior involving reinforcement learning, recurrent neural networks (RNNs), a type of ANN, have been frequently used. RNNs are able to capture how the choice probability changes depending on past experience. In this work, we demonstrate that RNNs can improve future choice predictions by capturing individual differences on the basis of past behavior, even when a common single model is fit to the entire population. We term this property of the RNN the individual difference tracking (IDT) property. While the IDT property might be useful for prediction, it may introduce excessive flexibility when providing an upper bound on the predictive accuracy. Through numerical simulations, we illustrate the properties of RNNs and discuss their implications and issues for further consideration.