Using face averages to measure differential accuracy for demographic groups in facial recognition

Facial recognition Deep Neural Networks (DNNs) and humans both show systematic differences in face recognition accuracy for different demographic groups. Differential accuracy has been shown due to race, age and gender, raising important questions about the impact of facial recognition decisions on the fairness of society. Current methods for measuring bias require curating large databases of face images, which is labor-intensive, producing bespoke tests that cannot be easily shared or standardized due to data privacy. Here we develop a novel solution to this problem inspired by psychological research in face perception. We ask whether face averages can be used to predict face recognition accuracy differentials for DNNs and humans. We generate sets of average images from random samples of exemplar faces from a demographic group to measure the density of these sets in representational space. We find that this approach provides reliable predictions of differential accuracy across demographic groups in both DNNs and human participants. However, we also find evidence that face averages were not represented in the center of face categories. This finding should be addressed in future development of our approach, and also challenges influential cognitive models of face identity representation.