Improving SpikeProp’s Training Efficiency in Spiking Neural Networks for Large Language Models Through Innovative Weight Initialization

Spiking neural networks (SNNs) mimic the functions of biological neurons by leveraging individual temporal spikes for communication and computation. Since SNN was perceived to be complex and analytically challenging, it had long been overlooked. In this study, we explore the enhancement of SpikeProp, a supervised learning model customized for SNNs. Three distinct models are being investigated, including the Proposed Model 1, the Proposed Model 2, and the Proposed Model 3, each providing unique improvements to the SpikeProp algorithm. To accelerate convergence and adapt learning rates, momentum factors are integrated into Proposed Model 1. In Proposed Model 2, a rate dependency is introduced based on Angle Driven Learning. By incorporating particle swarm optimization (PSO), Model 3 combines the strengths of Model 1 and 2. SNNs can be trained and classified more efficiently and accurately using these models. Furthermore, we examine how large language models (LLMs) might inform the design and interpretability of neural architectures and learning methodologies while also enhancing SNN training. Through the use of LLMs, we seek to enhance model transparency and encourage more Responsible AI (RAI) principles. A thorough evaluation and comparison of Proposed Model 1, Proposed Model 2, and Proposed Model 3 with traditional methods confirms that these models consistently outperform them. Consequently, they have a high potential for practical applications in neural network training in real-world settings and LLM-informed development, contributing to the advancement of AI systems.