Characterizing internal models of the visual environment

Despite the complexity of real-world environments, natural vision is seamlessly efficient. To explain this efficiency, researchers often use predictive processing frameworks, in which perceptual efficiency is determined by the match between the visual input and internal models of what the world should look like. In natural scene processing, predictions derived from our internal models of a scene should play a particularly important role, given the highly reliable statistical structure of our environment. Despite their importance for scene perception, we still do not fully understand what is contained in our internal models of the environment. Here, we argue that the current literature on scene perception disproportionately focuses on an experimental approach that tries to infer the contents of internal models from arbitrary, experimenter-driven manipulations in stimulus characteristics. To make progress, additional participant-driven approaches are needed: Rather than solely relying on manipulating the input to the visual system, researchers should adopt a complementary approach focusing on participants' descriptions of what they believe constitutes a typical scene. Such descriptions promise to capture the contents of internal models in more unconstrained ways on the level of individual participants. Critically, the descriptions of internal models can in turn be used to predict the efficiency of scene perception. We highlight recent studies on memory and perception using innovative methodologies like line drawings to characterize internal representations. These emerging methods show that it is now time to also study natural scene perception from a different angle – starting with a characterization of individual’s expectations about the world.