I See, You See: Investigating Theory of Mind using CAVE-VR and EEG

Communication involves understanding the difference between our own perspective (e.g., visual perspective, beliefs and knowledge) and that of an addressee. Previous research found that knowledge about a speaker’s background (e.g., stereotypes, accent, etc.) affects on-line language processing in listeners. However, the role and automaticity of Theory of Mind (ToM) – the ability to attribute beliefs to someone – remains unclear. Typically, we expect individuals to speak according to their beliefs and perspectives. But what if their utterances mismatch with what we expect them to know? We designed a novel perspective-taking paradigm in a virtual environment, which manipulates a virtual agent’s beliefs. We then recorded electroencephalography data from participants, while they listened to the agent making statements either matching or mismatching their false or true beliefs. Specifically, we analyzed the N400: an event-related component, whose amplitude scales with the unexpectedness of each content word. We predicted that statements mismatching an agent’s true belief (e.g., seeing a car and saying “That’s a screwdriver”) would result in more pronounced N400 components, compared to statements matching the true beliefs. We found strong evidence for this prediction. Further, we predicted that knowledge of an agent’s unawareness that the state of affairs has changed (i.e., a false belief) is considered when interpreting their statements. Neither statements matching agents’ false beliefs nor statements mismatching agents’ false beliefs (e.g., agent says something they couldn’t have known) are marked by the N400. This could only be the case if participant’s brains rapidly take the agent’s belief (ToM) into account when processing the semantics of the agent’s utterance.