Delaying the onset of aided target recognition highlights allows for a more dispersed allocation of overt attention
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Visual search is a critical component of many professions such as military operations, baggage screening, and radiology. Aided Target Recognition (AiTR) systems are designed to highlight potential threats across the operator’s visual field in real-time, directing attention and improving accuracy. However, these systems may impact search and, consequently, situational awareness by diverting attentional resources from non-highlighted, yet relevant, locations. Previous work suggests that scene gist is extracted within the first 250 ms of scene onset (Vö & Henderson, 2010). As such, this study examined whether a 250 ms AiTR onset delay could encourage a more even distribution of attention. Participants searched synthetically generated scenes and classified each person in the scene as armed or unarmed. Depending on their condition, participants either saw the scenes unaugmented (No AiTR condition), with AiTR highlights consisting of red bounding boxes around armed people and yellow boxes around unarmed (AiTR condition), or with AiTR highlights presented 250 ms post scene onset (Delayed AiTR condition). A surprise memory test of background objects presented in the search scenes was administered to all participants upon completion of the search task. As predicted and preregistered, results showed less overt attentional deployment to background information (anything other than the people themselves) in the AiTR condition compared to No AiTR, however, decreased overt attentional deployment was not seen in the Delayed AiTR group. A similar pattern was observed in the memory data (with the AiTR condition having a lower score than the No AiTR condition and the Delayed AiTR condition), this difference was not significant.
Significance statement
Aided target recognition (AiTR) systems are designed to augment a user’s field of view with salient highlights at locations containing task-relevant or threatening information in the environment. As computer vision models advance, such technologies will become an even more critical tool for people that perform visual search professionally (e.g. Soldiers, baggage screeners, radiologists). In a military context especially, having a system that potentially improves threat detection accuracy and reduces response times could be lifesaving. However, such benefits can only be realized if AiTR systems are designed to be integrated with the cognitive mechanisms of the human user. Without this, displays run the risk of being distracting, creating attentional blind spots, and generally producing a poorer user experience and performance. The current work, motivated by findings in the literature, tested the introduction of a short delay between scene onset and the onset of AiTR highlights. It was found that this very brief delay (250 ms) was sufficient to allow participants to direct overt attention to areas of the scene not highlighted by the AiTR system. This finding is critical because, while the purpose of AiTR highlights is to encourage the placement of attentional resources across the visual field, it is imperative that the human operator ultimately be able to attend to non-highlighted locations in the event of system failures (e.g. threats missed by the AiTR). Moreover, this work demonstrates how cognitive research can be used to systematically explore complex application spaces for improve human-machine integration.
