Orchestrating segment anything models to accelerate segmentation annotation on agricultural image datasets

Increasingly many applications of machine vision and artificial intelligence (AI) can be observed in agriculture. Yet, high quality training data remains a bottleneck in the development of many AI solutions which is particularly true for image segmentation. Therefore, ARAMSAM (agricultural rapid annotation module based on segment anything models) was developed, a user interface orchestrating the pre-labelling capabilities of both the segment anything models (SAM 1, SAM 2) as well as conventional annotation tools. One in silico experiment on zero-shot performance of SAM 1 and SAM 2 on three unseen agricultural datasets and another experiment on hyperparameter optimization of the automatic mask generators (AMG) were conducted. In a user experiment, 14 agricultural experts applied ARAMSAM to quantify the reduction of annotation times. SAM 2 benefitted greatly from hyperparameter optimization of its AMG . Based on ground truth masks matched with predicted masks, the F ₂ -score of SAM 2 improved from 0.05 to 0.74 while SAM 1 was improved from 0.87 to 0.93. The user interaction time could be reduced to 2.1 s/mask on single images (SAM 1) and to 1.6 s/mask on image sequences (SAM 2) compared to polygon drawing (9.7 s/mask). This study demonstrates the potential of segment anything models as incorporated in ARAMSAM to significantly accelerate the process of segmentation mask annotation in agriculture and other fields. ARAMSAM will be released as open-source software (AGPL-3.0 license) at https://github.com/DerOehmer/ARAMSAM.