UniROS: ROS-Based Reinforcement Learning Across Simulated and Real-World Robotics

Reinforcement Learning (RL) enables robots to learn and improve from data without being explicitly programmed. It is well-suited for tackling complex and diverse robotic tasks, offering adaptive solutions without relying on traditional, hand-designed approaches. However, RL solutions in robotics have often been confined to simulations, with challenges in transferring the learned knowledge or learning directly in the real world due to latency issues, lack of a standardized structure, and complexity of integration with real robot platforms. While the use of Robot Operating System (ROS) provides an advantage in addressing these challenges, existing ROS-based RL frameworks typically support sequential, turn-based agent-environment interactions, which fail to represent the continuous, dynamic nature of real-time robotics or support robust multi-robot integration. This paper addresses this gap by proposing UniROS, a novel ROS-based RL framework explicitly designed for real-time multi-robot/task applications. UniROS introduces a ROS-centric implementation strategy for creating RL environments that support asynchronous and concurrent processing, which is pivotal in reducing the latency between agent-environment interactions. This study validates UniROS through practical robotic scenarios, including direct real-world learning, sim-to-real policy transfer, and concurrent multi-robot/task learning. The proposed framework, including all examples and supporting packages developed in this study, is publicly available on GitHub, inviting wider use and exploration in the field.