Fit-DeepOKAN: Enhancing Neural Operator Learning of Soliton Dynamics via FitNets Distillation

The Nonlinear Schrödinger (NLS) equation plays a pivotal role in various physical domains, including optics, plasma physics, and quantum mechanics. Since solitons arise as fundamental localized solutions of the NLS equation, their efficient and accurate computation carries profound theoretical and engineering significance. In this work, we investigate the potential of neural operator frameworks for soliton modeling. As one of the most promising operator learning networks, DeepOKAN provides a powerful baseline for modeling nonlinear wave dynamics. Building upon this foundation, we propose a novel DeepOKAN architecture augmented with the FitNets distillation strategy, and conduct systematic evaluations on a range of NLS-type equations with diverse soliton families as benchmark problems.Furthermore, we explore the impact of different distillation schemes and the incorporation of activation functions on model performance. Quantitative results indicate that the FitNets-based distillation outperforms the logits-only scheme, reducing relative $l_2$ errors by over 16.4\%. In contrast, the incorporation of the tanh activation not only fails to enhance accuracy but, in the worst case, increases the error by an order of magnitude. Results from experiments show that even with limited network sizes, the suggested model has a great expressive potential. This highlights its capability in learning complex functional mappings and affirms the feasibility of embedding structured knowledge into operator learning models. The approach shows promising extensibility and potential for broader applications in scientific computing and engineering contexts. Mathematics Subject Classification (2020) 35Q55 · 37K40 · 68T07