The recoverability, reliability and generalisability of reward processing parameters and relation to mental health symptoms

Theory-driven computational psychiatry attempts to use cognitive models of computation to understand how mental health problems might relate to (or be caused by) changes in cognitive processes such as learning and decision-making. However, the potential applications and relevance of this approach are contingent on several (often implicit) assumptions, including that computational parameters 1) are recoverable, 2) are reliable over time, 3) reflect conceptually similar processes across different tasks, and 4) relate to symptoms. To illustrate and test these assumptions for a selection of commonly used tasks, we recruited a large online sample of participants (n=548), who completed seven mental health questionnaires and five tasks. A subset of n=115 was re-invited to complete the five tasks 14 days later. For each task, five models (including a null model, or model of no interest) were fit and the winning model was selected through Bayesian model comparison. The parameters from the winning models showed good recovery (mean r=0.853, sd=0.135), and good to excellent levels of test-retest reliability (mean ICC=0.635, sd=0.30). Parameters that were theoretically or mathematically similar to each other were, however, generally not related, with relationships only found within the class of ‘inverse temperature’-like parameters (significant r values ranged from 0.11 to 0.23): indicating that in general, parameters do not generalise across task contexts. Finally, only parameters from two of the five tasks (four-armed bandit and cognitive effort) related to symptoms, and these relationships were weak (maximum absolute r values ~0.10). To conclude, it seems that at least some of the implicit assumptions when advocating for the use of computational models in psychiatry are not met for these popular tasks and computational models. This limits the clinical and translational utility of this approach. Computational psychiatry researchers should carefully assess the assumptions and psychometric properties of their tasks and models as part of their early-phase development to ensure robustness of the field going forward.