A Hierarchical State Feature-Driven Deep Reinforcement Learning Framework for Semiconductor Fabrication AGV Path Planning

This paper addresses the path-planning problem for automated guided vehicles (AGVs) in semiconductor fabrication facilities, which feature dense layouts, narrow corridors, and dynamic obstacles. We opose a Hierarchical State Feature-Driven Deep Reinforcement Learning (HSF-DRL) framework. Built upon the options framework in hierarchical reinforcement learning (HRL), the proposed method decomposes navigation into a two-tier decision-making process: a high-level meta-controller selects temporally extended options (e.g., global navigation, dynamic avoidance, and precise docking), which are integrated with an online heuristic search using task-specific features; meanwhile, a low-level executor, implemented with a Deep Q-Network (DQN), generates primitive actions. A key innovation is a dynamic feature-fusion mechanism in which the weights of environmental, procedural, and heuristic features are conditioned on the active high-level option, enabling context-aware perception for the low-level policy. Evaluations in a grid-based semiconductor-fab simulation demonstrate that HSF-DRL outperforms traditional DQN and Dyna-Q in path optimality, convergence speed, and stability, particularly in highly dynamic scenarios. Overall, this work provides a theoretically grounded solution with a novel architecture for AGV navigation in complex industrial settings.