WeedNet: A Foundation Model-Based Global-to-Local AI Approach for Real-Time Weed Species Identification and Classification

Early identification of weeds is essential for effective management and control, and there is growing interest in automating the process using computer vision techniques coupled with artificial intelligence (AI) methods. However, challenges associated with training AI-based weed identification models, such as limited expert-verified data and complexity and variability in morphological features, have hindered progress. To address these issues, we present WeedNet, a global-scale weed identification model capable of recognizing an extensive set of weed species. WeedNet is an end-to-end real-time weed identification pipeline and uses SSL, fine-tuning, and enhanced trustworthiness strategies. WeedNet achieved 91.02% accuracy across 1,593 weed species, with 41% species achieving 100% accuracy. Using a fine-tuning approach, the local Iowa WeedNet model achieved an overall accuracy of 97.38% for 84 Iowa weeds, most classes exceeded a 90% mean accuracy per class. Testing across intra-species dissimilarity (different developmental stages) and inter-species similarity (look-alike species) suggests that diversity in the images collected, spanning all the growth stages and distinguishable plant characteristics, is crucial in driving model performance. The generalizability and adaptability of the Global WeedNet model enable it to function as a foundational model, with the Global-to-Local strategy allowing fine-tuning for region-specific weed communities. Additional validation of drone- and ground-rover-based images highlights the potential of WeedNet for integration into robotic platforms. Furthermore, integration with AI for conversational use provides intelligent agricultural and ecological conservation consulting tools for farmers, agronomists, researchers, land managers, and government agencies across diverse landscapes.