The Global Wheat Full Semantic Organ Segmentation (GWFSS) Dataset

Computer vision is increasingly used in farmers’ fields and agricultural experiments to quantify important traits related to crop performance. In particular, imaging setups with a submillimeter ground sampling distance enable the detection and tracking of plant features, including size, shape and color. While today’s AI-driven foundation models segment almost any object in an image, they still fail to perform effectively for complex plant canopies. To improve model performance for wheat, the global wheat dataset consortium assembled a large and diverse set of images from research experiments around the globe. After the success of the global wheat head detection dataset (GWHD), the new dataset targets a full semantic segmentation (GWFSS) of wheat organs (leaves, stems and spikes). Images were collected by 11 institutes and universities using a wide range of imaging setups. Two datasets are provided: i) an annotated set of 1096 diverse images in which all organs were labeled at the pixel level, and (ii) a large dataset of 48,675 images without annotations that is available for additional training. The labeled set was used to train segmentation models based on DeepLabV3Plus and Segformer. Our Segformer base model performed slightly better than DeepLabV3Plus with a mIOU for leaves and spikes of ca. 90%. However, the precision for stems with 54% was rather lower. The major advantages over published models are: i) the exclusion of weeds from the wheat canopy, ii) the detection of all wheat features including necrotic and senescent tissues and its separation from crop residues. This facilitates further use of the dataset in classifying healthy vs unhealthy organs so that the model may have utility in addressing the increasing need for accurate quantification of senescence and diseases in wheat canopies.