The Curious Mind: Eye Movements to Maximize Scene Understanding

Human eye movements while identifying a face, searching for targets, and executing motor actions are directed to regions contributing to task accuracy. However, what humans look at and do when free-viewing a scene without a specific task is not well understood. We show that observers' free-viewing fixations are similar to fixations of observers instructed to describe the scenes and dissimilar to fixations of observers counting objects or searching for specific objects. Small visual alterations to images that change a scene’s understanding but not the most salient or its meaning map alter where humans most frequently fixated. Free viewing fixations are more frequently directed to objects critical to the understanding of a scene (objects that, when erased from the scene, maximally alter the scene’s description) rather than the most salient, most meaningfully judged scene region (meaning map), or the object perceived to be gazed or grasped. By having observers describe the scene while maintaining fixation on objects relevant or irrelevant to scene understanding, we show that eye movements during free viewing are functionally important to extracting an accurate understanding of scenes. Thus, we conclude that the human brain's default task during free viewing is to understand scenes reflected by frequent eye movements to objects that maximize accurate understanding even when these objects are not gazed at or to be grasped.