Insights into the mind using tools from Explainable AI: A practical introduction to SHAP in cognitive science

Computational models of cognition are becoming increasingly more complex and subsequently harder to interpret. A recent exploratory method from Explainable AI, SHAP, aims at providing a simple and consistent way to explain the decisions such models make. However, to date, while very popular within Explainable AI, this method is hardly used in cognitive science. We here make a case for its utility in cognitive science, provide a non-technical introduction including detailed practical and mathematical considerations and demonstrate how this tool can be applied in cognitive modeling.