Machine Learning Matches Human Performance at Segmenting the Human Visual Cortex

A major problem in human visual neuroscience research is the localization of visual areas on the cortical surface. Currently available methods are capable of making detailed predictions about many areas, but human raters do not agree as well with these methods as they do with each other. Although highly accurate, human raters require substantial time and expertise that many researchers do not have. Additionally, human raters require functional data for drawing visual area boundaries that requires additional scan time, budget, and expertise to collect. Here, we train convolutional neural network (CNN) models to predict the boundaries and iso-eccentric regions of V1, V2, and V3 in both the Human Connectome Project dataset and the NYU Retinotopy dataset. CNNs trained to use the same functional data available to human raters predicted these boundaries with an accuracy similar to humans, while CNNs trained to use only anatomical data had a lower accuracy that was nonetheless higher than that of any currently available method. A comparison of the model accuracies when predicting eccentricity-based boundaries and polar angle-based boundaries suggests that eccentricity is substantially less closely tied to anatomical structure than polar angle and that the cortical magnification function, at least in terms of eccentricity, varies substantially between subjects. We further find that the fraction of V1, V2, and V3 that can be accurately parcellated into function regions using gray matter structural data alone is ∼75% (∼80% of the inter-rater reliability of human experts), implying a much tighter coupling between structure and function in these areas than previously estimated. We conclude that machine learning techniques such as CNNs provide a powerful tool for mapping the brain with human accuracy and predict that such tools will become integral to neuroscience research going forward.