Transformer-Enhanced Deep Q-Learning for AdaptiveRobot Path Planning in Dynamic Environments

Efficient navigation in dynamic environments remains a critical challenge for autonomous robots in industrial manufacturing, search-and-rescue operations, and automated warehousing. Traditional path-planning algorithms struggle to adapt to real-time obstacle movements, while conventional reinforcement learning (RL) approaches lack the capacity to model long-range spatial dependencies. This paper presents Transformer-Enhanced Deep Q-Learning (Transformer-DQN), a novel framework that integrates transformer architectures with Deep Q-Networks (DQN) to address the limitations. By leveraging multi-head self-attention mechanisms and Cartesian positional encoding, the model dynamically captures obstacle interactions and optimizes navigation in cluttered environments. Hyperparameter tuning via Optuna ensures a balance between exploration and exploitation, while prioritized experience replay enhances training stability. Experimental results demonstrate significant advancements over baseline methods:20×20 Grids: Reduces average pathfinding time by 33.65% (209.5s → 139s) andcollisions by 37.5% compared to vanilla DQN.30×30 Grids: Achieves a 94.6% time reduction (9125s → 492.5s) and 33.3% fewercollisions, showcasing superior scalability.Adaptive Performance: Outperforms PPO (70% vs. 85% success rate) and classicalplanners (RRT*/D*) in dynamic settings, approaching optimal path lengths (28 vs. 25steps).The Transformer-DQN’s ability to generalize across grid sizes and dynamically re-plan in real-time positions it as a robust solution for time-sensitive applications. Theoretical analysis confirms convergence guarantees, while empirical validation highlightsits energy efficiency and reduced computational overhead. This work bridges the gap between simulated and real-world robotic systems, offering a scalable framework forautonomous navigation in unstructured, dynamic environments.