Multi-AGV Path Planning Using Deep Reinforcement Learning with Internal Curiosity

Deep Reinforcement Learning (DRL) is promising for multi-agent path planning problems in which sparse external environmental rewards may cause the agent group to make overly conservative decisions and explore the environment inefficiently. In general, the reward shaping mechanism is used to mitigate the above problems with the additional reward function setting. However, it requires specific domain knowledge, which limits the general applicability, and the added reward functions are not necessarily applicable to all environments. This paper aims to improve the path planning efficiency of single agents and groups of agents with the Internal Curiosity Module (ICM) mechanism to boost the generalization abilities of the agents in different environments. To this end, we incorporate the internal curiosity mechanism into the soft actor-critic model for enhancing exploration strategies, adapting to environmental changes, and improving learning effectiveness. Then, we propose a multi-agent path planning method in which the curiosity mechanism is integrated with the Multi-Agent POsthumous Credit Assignment (MA-POCA) algorithm. The neural networks can automatically calculate the additional intrinsic rewards based on observed information about the environment and the actions taken by the group of agents. Based on our experiments, we make qualitative and quantitative analyses of the performance of the proposed methods and the baseline DRL methods. The experimental results show that our proposed methods can decline the number of learning episodes and the training time of path planning, so the proposed algorithms can accelerate the exploration of single agents or agent groups in the sparse reward environment.