Autonomous Driving System based on Dual Process Theory and Deliberate Practice Theory

Autonomous driving (AD), despite significant progress, is still not widely applied in open, unconstrained environments, primarily owing to deficiencies in hazard perception, few-shot generalization, corner case generalization, and evaluation metrics, resulting in reliability concerns. Therefore, we propose CogniDrive, an innovative framework based on dual-process and deliberate practice theories, leveraging contextual reasoning of the large language model (LLM) to enhance AD robustness and generalization. Inspired by dual-process theory, CogniDrive comprises two cognition modes: InstinctNav for rapid, intuitive decision-making and ReflectPlan for reflective reasoning. Enhanced by a thought model and experience embedding for LLM, InstinctNav combines behavioral cloning and retrieval augmented generation to enhance few-shot learning efficiency based on deliberate practice theory. ReflectPlan processes and internalizes reward signals embedded in language tokens within the prompt, derived from a self-reflection mechanism, to enable continuous improvement and generalization. To detect hazards in corner cases precluded by limited training data distribution, a vision language model is integrated for comprehensive environmental understanding through multimodal self-reflection. Furthermore, we propose an evaluation system that addresses the incompleteness of traditional metrics and emphasizes safety, comfort, and energy efficiency. Experiments on nuScenes and CODA datasets using the proposed evaluation system demonstrate CogniDrive's superior robustness and adaptability.