Spatiotemporal properties of common semantic categories for words and pictures

The timing of semantic processing during object recognition in the brain is a topic of ongoing discussion. One way of addressing this question is by applying multivariate pattern analysis (MVPA) to human electrophysiological responses to object images of different semantic categories. However, while MVPA can reveal whether neuronal activity patterns are distinct for different stimulus categories, concerns remain on whether low-level visual features also contribute to the classification results. To circumvent this issue, we applied a cross-decoding approach to magnetoencephalography (MEG) data from stimuli from two different modalities: images and their corresponding written words. We employed items for three categories and presented them in a randomized order. We show that If the classifier is trained on words, pictures are classified between 150 - 430 ms from after stimulus onset, and when training on pictures, words are classified between 225 - 430 ms. The topographical map identified using a searchlight approach for cross-modal activation in both directions showed left lateralization confirming the involvement of linguistics representations. These results point to semantic activation of pictorial stimuli occurring at ≈150 ms whereas for words the semantic activation occurs at ≈230 ms.