BirdNET can be as good as experts for acoustic bird monitoring in a European city
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
BirdNET has become a leading tool for recognising bird species in audio recordings. However, its applicability in ecological research has been questioned over the sometimes large number of species falsely identified. Using species-specific confidence thresholds has been identified as a powerful approach to solving this issue. However, determining these thresholds is time and resource-consuming. While optimising the parameter setting of the algorithm could be an alternative strategy, the effect of parameter settings on the algorithm’s performance is not well understood. Here, we compared the species identification of BirdNET against expert identification using an acoustic dataset from a single site in Munich, Germany. The performance of BirdNET was evaluated using three performance metrics: precision, recall, and F1-score, using 24 combinations of the parameters: week, sensitivity, and overlap at four temporal aggregations (pooling of data across time intervals). We found that BirdNET performance varied widely depending on parameter settings (0.46–0.84). When given more data (higher temporal aggregation) and with tuned parameters, BirdNET came close to matching the expert identification (F1 score = 0.84). While BirdNET missed five species of the 23 species identified by the experts, our confirmation test revealed that BirdNET also found one species missed by the experts. To understand how each parameter affects F1 score, we trained linear mixed effects models. Our models showed that the confidence threshold had the strongest effect on the F1 score (p < 0.001) and significantly interacted with temporal aggregation, sensitivity, and overlap. Our results showed that while there are still limitations, using appropriate parameter settings, aggregating results over longer periods and undertaking some basic validation, BirdNET can yield results comparable to experts without the need for time-consuming estimation of species-specific thresholds.
