BrainACTIV: Identifying visuo-semantic properties driving cortical selectivity using diffusion-based image manipulation

The human brain is thought to efficiently represent visual inputs through specialized neural populations that selectively respond to specific image categories. Advancements in generative modeling have enabled data-driven discovery of neural selectivity using brain-optimized image synthesis. However, current methods independently generate one sample at a time, making it hard to discern which image features drive neural response selectivity. To address this issue, we introduce Brain Activation Control Through Image Variation (BrainACTIV), a method for manipulating a reference image to enhance or suppress activity in a target cortical region using pretrained diffusion models. Starting from a reference image allows for fine-grained and reliable identification of optimal visuo-semantic properties. We show that our manipulations effectively modulate predicted fMRI responses and agree with hypothesized preferred categories in established regions of interest, while remaining structurally close to the reference image. In addition, we describe how two hyperparameters allow a trade-off between semantic variation and low-level structural control, and how both variations affect predicted brain activity. Finally, we demonstrate how our method accentuates differences between brain regions that are selective to the same category. We propose that BrainACTIV holds the potential to formulate robust hypotheses about brain representation as well as produce controllable naturalistic stimuli for neuroscientific experiments.