Scene hierarchy structures mental object representations while flexibly adapting to varying task demands

Objects in our environment are hierarchically organized. They are part of scenes and, within them, grouped into different spatial clusters (“phrases”). Some objects serve as “anchors” aiding localization and identification of other objects near them (“local objects”). This organization seems reflected in mental representations of individual objects in memory, but it is not clear how flexibly mental representations adapt to different tasks demands. Therefore, we collected pairwise similarity judgments for 45 objects with three different groups of participants performing the task using different criteria: without explicit definition, based on visual appearance, and based on actions performed with objects. These similarity judgments were modelled using two sets of predictors identifying different levels of the hierarchy (“scene”, “phrase”, “object type”), and a set of visual predictors. Results showed a stable effects of hierarchy on mental representations across tasks. For instance the phrasal predictor was stronger in the “actions” task, while in the “visual” task scene grammar hierarchy was weaker than visual predictors. Crucially, without explicit instructions, judgements were almost identical to the “actions” task. We conclude that scene hierarchy seems to organize mental representations of objects by default, while flexibly adapting to changing goals. In particular, the default mental structure seems to support efficient interactions with objects in the environment.