Green Pepper Harvesting Robot System Based on Multi-Target Tracking with Filtering and Intelligent Scheduling

To address the challenges of unstable target localization and poor multi-module coordination in automated green pepper harvesting—caused by occlusions from branches and leaves, as well as varying lighting conditions—this paper presents the design and implementation of a modular robotic picking system. At the perception level, the system integrates a YOLOv8 detector with a RealSense D435i camera to identify and locate the calyx–ectocarp junctions of green peppers. A multi-target tracking with filtering algorithm is proposed, combining IoU-based association, Mahalanobis-distance-based matching, the Hungarian algorithm, Kalman filtering, and single exponential smoothing. This algorithm suppresses depth noise and trajectory jitter, thereby enhancing the stability and accuracy of 3D localization. At the control and execution level, a depth-first picking sequence strategy with ID freeze-state management is implemented within a multithreaded software–hardware co-design architecture. This approach avoids task conflicts and duplicate operations while supporting continuous multi-fruit harvesting. Field experiments under natural outdoor lighting and varying occlusion levels demon-strate that the proposed system achieves recognition rates of 91.57% and 80.29%, and harvesting success rates of 82.85% and 77.68% for non-occluded and lightly occluded fruits, respectively. The average picking cycle per pepper fruit is 9.8 s. This system provides an effective technical solution for addressing stability control challenges in the automated harvesting process of green peppers.