RRH-RL: Recursive Receding Horizon Deep Reinforcement Learning for End-to-End Robot Navigation in Mapless Harsh Environments

This paper presents RRH-RL, a Recursive Receding Horizon Reinforcement Learning approach for end-to-end mobile robot navigation in mapless, harsh environments. Navigation in such conditions poses multiple challenges, including partial observability, short-sighted decision-making, trap avoidance, and energy inefficiency. RRH-RL addresses these by predicting a horizon of future actions at each step, executing the first, and recursively embedding the rest into subsequent observations to maintain temporal consistency and to produce energy-efficient motion. A probabilistic costmap—updated via Bayesian filtering and belief propagation (BP)—serves as spatial memory to avoid revisiting dead ends and being trapped. A projection-based mechanism estimates future sensor observations, enabling accurate foresight-driven reward shaping. Experiments in high-fidelity simulations across diverse harsh environments maps show that RRH-RL significantly outperforms state-of-the-art approaches, achieving up to 7.58\% higher success rate, 9.86\% shorter path length, 22.06\% smoother motion, and 20.67\% lower energy consumption, compared to the next best approach. Additionally, ablation studies are performed to validate the contribution of horizon length, recursive action embedding, and the BP to overall performance. The full source code is available on GitHub.