Centaur: A model without a theory

Binz et al. (in press; preprint at: arxiv.org/abs/2410.20268) developed a large language model (LLM) called Centaur that better predicts trial-by-trial human responses in 159 of 160 behavioural experiments compared to existing cognitive models. The authors conclude that Centaur provides “…tremendous potential for guiding the development of cognitive theories” and suggest that it makes an important contribution to building a unified theory of cognition. In our view, the curation of the Psych-101 database of psychological experiments is a valuable resource for training and testing LLMs as models of human cognition. However, we highlight three major problems with Centaur that undermine the authors’ conclusions. First, Centaur was not subjected to severe tests and therefore, it is unclear whether Centaur accounts for many phenomena (effects) that are informative regarding psychological theory. Second, the model is not limited by the most basic mechanistic constraints of human cognition. Therefore, it produces wildly implausible behaviour when subjected to simple experimental manipulations and/or absurd instructions. Third, even if the model did behave like a human under such manipulations, it is unclear what theoretical insights would have been gained. Successful prediction does not imply successful explanation, and as cognitive scientists the development of explanatory theories is our main goal.