Evaluating AI Models for Food and Alcohol Ad Classification Against Human Raters

The growth of food and alcohol marketing on social media creates a need for scalable monitoring methods that go beyond manual processing. This study evaluates whether Large Language Models and Vision-Language Models can recognize advertisements and identify their features in consistence with general public or expert opinion. We collected 1000 Facebook ads from major Belgian brands, and annotated them with 600 crowd workers, three dieticians and four AI models (GPT-4o, Qwen 2.5, Pixtral and Gemma3). Our analysis of the data shows that for single-option advertisement features, like alcohol presence or target group, GPT-4o and Qwen reached agreements with dieticians above 90%, similar to the range of across dieticians. Though agreement was lower for multiple choice features, like premium offers and marketing strategies, it was still within the variability observed in crowd raters. The bias analysis revealed how models interpret certain labels, with some being consistently under- or over-detected. These findings show that AI models can already automate advertisement annotations but still require label modification or expert oversight for some of the features.