PlumageParts: A fine-grained avian plumage segmentation dataset and benchmark for ecological image analysis
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Fine-grained localisation of plumage regions is a prerequisite for computational analyses of avian colouration, patterning and visual traits in ecological and evolutionary research. Progress is limited by the scarcity of image resources with annotations aligned to biologically meaningful anatomical units: existing avian benchmarks provide either landmark points or coarse part categories that do not capture ornithologically defined plumage regions. We present a curated dataset of 4,705 bird images annotated for nine plumage regions: head, throat, breast, belly, vent, back, coverts, remiges and tail. Spanning 39 avian orders and 222 families, the dataset provides a taxonomically broad resource for fine-grained avian image analysis. The dataset was built through an iterative model-assisted annotation workflow, in which model predictions were reviewed and corrected rather than drawn from scratch, improving the efficiency of region-level annotation. We benchmark classical segmentation architectures, SAM-based models and self-supervised foundation-model encoders on this task. A frozen DINOv3 encoder with a lightweight decoder achieved the highest performance on the held-out test set, reaching 84.01% mean Intersection over Union while requiring substantially less memory than end-to-end fine-tuning. The model generalised to external avian benchmarks, including the bird subset of PartImageNet and CUB-200-2011, and achieved competitive performance on the full PartImageNet part-segmentation benchmark, which includes diverse animal taxa. We provide a modular detect-track-segment pipeline as a proof-of-concept extension to video data. Together, these results show that anatomically grounded avian annotations can serve both as a resource for plumage phenotyping and as a benchmark for efficient, transferable biological part segmentation.
Author Summary
Birds vary enormously in colour and pattern, but studying this variation at large scales requires more than identifying the bird in a photograph. Researchers often need to know where each colour or pattern occurs on the body, such as on the head, throat, breast, wing or tail. We created PlumageParts to make this kind of region-level analysis easier. The dataset contains 4,705 bird images annotated into nine biologically meaningful plumage regions, covering a wide range of bird families and orders. To build the dataset efficiently, we used a model-assisted workflow in which computer-generated masks were checked and corrected by researchers rather than drawn entirely by hand. We then tested several image-segmentation approaches and found that a frozen self-supervised vision model, combined with a lightweight decoder, provided accurate plumage-region predictions while requiring relatively modest computing resources. The same approach also performed well on a broader animal part-segmentation benchmark, suggesting that it may be useful beyond birds when suitable annotations are available. By releasing the annotations, code and trained model, we aim to support future studies of bird plumage and biologically meaningful image segmentation.
