Enhanced Multi-Layer Steel Bar Intersection Recognition through Depth-Based Ground Plane Fitting and Image Fusion

The development of construction automation has significantly reduced labour intensity, with steel binding robots emerging as a pivotal tool. However, accurately recognizing steel intersections remains a challenging task due to multi-layer steel arrangements, conduit interference, and varying visual angles and lighting conditions. This paper proposes an innovative multi-layer steel bar intersection recognition algorithm leveraging depth cameras and image fusion techniques. By fitting the ground plane using random patches and probability statistics from depth images, we separate multiple steel bar layers. Subsequently, a binary mask generated from the depth image is fused with the RGB image, enhancing the semantic information for training a deep neural network, specifically YOLOv8. A new dataset, MuLSIR-252, is introduced, comprising images from various working conditions. Extensive experiments demonstrate that our algorithm achieves high accuracy and robustness, with an mAP of 98.8\%, validating its practical applicability in construction sites. Our contributions include a novel ground fitting method, a unique image fusion approach, and a comprehensive multi-layer steel bar recognition dataset. The code is available at: https://github.com/xuezhen2018/steel_detection.