Language Models and the Missing Qualia in Face Evaluation

Human cognition is grounded in direct sensory experience, whereas recent advances in large language models (LLMs) rely primarily on indirect, language-derived symbolic computation, raising intense debate about the limitation boundary of artificial intelligence. Here, we examine what social cognition based on face evaluation can be achieved from language-derived knowledge with direct visual experience in humans or without it in LLMs. We constructed 2,304 text-described faces by factorially combining ten structural features. Human participants (N=25) were asked to imagine each face and rate 12 traits on 1–9 scales, and five LLMs (Claude 3.5, GPT-4, GPT-3.5, Kimi, and ERNIE) performed the same task with identical instructions by API calls. Representational similarity analyses showed high human–LLM correspondence and strong cross-model homogeneity. Linear mixed-effects models further aligned the human consensus with LLMs’ ratings and captured agreement on facial structural traits. However, despite consensus-level alignment, humans showeddominant participant-by-stimulus variance for social traits, whereas LLMs showed little corresponding idiosyncratic variance. XGBoost attributions indicated shared reliance on expression and skin texture, while humans additionally showed domain-sensitive cue reweighting, a pattern absent in LLMs. Interestingly, higher visual imagery ability in humans predicted greater idiosyncratic variance and weaker residual alignment with LLMs. Together, these results suggested that language statistics can approximate the shared scaffold of face evaluation, whereas experience-linked imagery contributes structured individual-specific variability that current LLMs largely fail to capture, highlighting the key roles of direct sensory experience and visual imagery in human social cognition.