Object categorization based on abstract concepts by monkeys, humans, and vision models

Viewing an object in the external world evokes a rich variety of concepts associated with it in the human mind, such as whether it is animate or not, man-made or not, big or small, and countless other attributes. This concept recognition seems to depend on the human-specific knowledge about these objects. However, studies have shown that neurons in visual areas of non-human primates encode some object concepts, raising questions about the extent to which object concepts are uniquely possessed by humans. Here, we show that macaque monkeys can rapidly learn to classify natural object images according to a surprisingly rich set of rules defined by what humans would call abstract concepts. We tested more than 10 binary classification tasks, including animate versus inanimate, natural versus man-made objects, and mammalian versus non-mammalian animals. The monkeys learned each rule in a few days, generalized the learned rules to new images of objects they had probably never seen before and made error patterns consistent with human judgments. Visual deep neural networks (DNNs) could also perform the tasks that monkeys could learn, whereas both DNNs and monkeys failed to classify some less common abstract concepts. Thus, the monkeys’ successful classification likely depended on whether concepts were embedded in natural images as visual features that could be extracted by high-level visual processing. The observed capacity of primate brains may subserve the formation of rich abstract concepts that the human mind possesses.