Folie à 1 – Artificially induced delusion and trust in LLMs

Abstract: Trust in Large Language Models (LLMs) is common. This trust is explained by LLMs’ highly fluent – fast and coherent – output. A recent spate of reports about psychotic delusions caused by LLMs shows that this trust in LLMs is risky and misplaced. As I will argue, it is not even trust in the robust sense of the term. LLM-induced delusion is a variant of a well-known phenomenon called induced delusion or folie à deux, where delusions are socially transmitted. Drawing on this phenomenon, I argue that when a user takes themselves to trust a LLM, they are actually only trusting themselves, but this self-trust is cloaked by the LLM’s agent-like nature. Given the considerable epistemic and moral hazards of this cloaked self-trust, we should discourage talk of trust in LLMs and avoid acting as if we trusted them.