A tale of three types of internal object representations in the human brain

Both object knowledge retrieval and sensory imagery generation engage perceptual regions in the brain and exhibit representational overlap with perception. However, whether they rely on common or distinct representational mechanisms remains unclear. In this study, we conducted an fMRI visual working memory experiment in individuals with (visualizers) and without (aphantasics) visual imagery, combined with support vector machine decoding to identify regions that contained shared neural representations across the presence and absence of visual input. Critically, we applied representational similarity analysis using a multimodal deep neural network (DNN) model comprising text and image encoders to dissociate language-structured and visual-structured knowledge representations. Comparisons between visualizers and aphantasics allowed us to test the relevance of voluntary imagery experience to these representations. We found that while ventral occipitotemporal regions contained shared representations across visual presence and absence, anatomically distinct regions supported different representational structures with differential associations to imagery experience: 1) the left ventral lateral occipitotemporal cortex (LOTC) encoded visual-structured knowledge representations and was linked to imagery experience; 2) the bilateral fusiform gyrus, left dorsal LOTC, and right ventral LOTC encoded language-structured knowledge representations, irrelevant to imagery experience; 3) the left superior parietal lobule encoded priorless knowledge-absent visual maintenance representation, also irrelevant to imagery experience. Together, these findings suggest that shared neural representations between the presence and absence of sensory input arise from multiple, functionally and computationally distinct mechanisms, which vary in their reliance on prior knowledge and imagery experience.