The Curious Mind: Eye Movements to Maximize Scene Understanding

Human eye movements while identifying a face, searching for targets, and executingmotor actions are directed to regions contributing to task accuracy. However,what humans look at and do when free-viewing a scene without a specific task isnot well understood. We show that observers’ free-viewing fixations are similar tofixations of observers instructed to describe the scenes and dissimilar to fixationsof observers counting objects or searching for specific objects. Small visual alterationsto images that change a scene’s understanding but not the most salientor its meaning map alter where humans most frequently fixate. Free viewing fixationsare more frequently directed to objects critical to the understanding ofa scene (objects that, when erased from the scene, maximally alter the scene’sdescription) rather than the most salient, most meaningfully judged scene region(meaning map), or the object perceived to be grasped or gazed at. By havingobservers describe the scene while maintaining fixation on objects relevantor irrelevant to scene understanding, we show that eye movements during freeviewing are functionally important to understand scenes accurately. The theoreticalframework also explains the high frequency of fixations on people in scenesbecause, when people are erased, scene descriptions are maximally altered. Thus,we conclude that an important default task during free viewing for the humanbrain is understanding scenes, reflected by frequent eye movements toward peopleand objects that maximize accurate scene understanding.