BeeSAM2: detecting bees in cherry flowers using timelapse images and foundational models

Bees perform important pollination services in fruit crops such as cherry. Growers will often bring in bees to supplement natural pollinators. Monitoring the performance of these pollinators is important to understand the impact of augmenting pollinators on fruit yield particularly in relation to June drop which is a major cause of yield instability in the cherry industry. Timelapse imaging plus automated image analysis methods is a valuable tool in studying the role of bee pollination in fruit set. Timelapse cameras allow for continuous monitoring of the flowers, but manual analysis of the generated footage is very time consuming.

We have developed a novel method of detecting bees in time lapse images, called BeeSAM2. This exploits both the zero shot detector Grounding Dino and the foundational model Segment Anything 2. Promising results are achieved with the method able to detect the bumblebee Bombus terrestris in images with a recall of 0.959 and precision of 0.991. These results are accurate enough to deploy our method to quantify bee activity in cherry plantations, advancing the ability of researchers to monitor bee interactions with flowers with a significant time saving over manual analysis of timelapse footage.