Development of Perception and Control Applications for Collaborative Robots Using ROS2

This work explores the integration of perception systems in behavior-based control applications using ROS2, with a focus on safe and effective interactions between humans and collaborative robots. Unlike conventional industrial robots, collaborative robots are designed to interact directly with human operators without physical barriers, making automation more accessible, particularly for small and medium-sized enterprises. A system integrating the UR3e robot and the Intel RealSense D435i camera was developed, enabling the robot to recognize objects and interpret gestures. YOLOv8 was used for object detection and segmentation, MediaPipe for gesture recognition and face detection, and Detectron2 for human detection, empowering the robot to manipulate objects and respond to gesture commands. The system's efficiency was validated through practical tests by moving objects between known points. The study highlights opportunities for improvement, such as replacing Detectron2 with YOLOv8, expanding the dataset, and adopting low-latency kernels. This research contributes to collaborative robotics by demonstrating the feasibility of integrating perception and behavior-based control, laying the foundation for future investigations and applications.