Lighting-Resilient Pedestrian Trajectory Prediction: A Hybrid Vision Transformer and Convolutional LSTMApproach with Dynamic Lighting Augmentation

Pedestrian trajectory prediction in dynamic and variable lighting environments presents significant challenges for traditional models, which often struggle to maintain the accuracy and robustness under such conditions. To address these limitations, we propose a novel hybrid model that integrates Vision Transformers (ViTs) with convolutional Long Short-Term Memory (ConvLSTM) networks. This model leverages the global contextual awareness of ViTs and the spatiotemporal modeling capabilities of the ConvLSTM to enhance trajectory prediction accuracy. The proposed model is further strengthened by incorporating dynamic lighting condition augmentation and contrastive spatiotemporal learning, which improves its generalization across diverse real-world scenarios. Our extensive evaluation using the KAIST Multispectral Pedestrian Dataset demonstrates that the proposed model significantly outperforms existing models, including social-LSTM and S-GAN, across key performance metrics. Specifically, the model achieves a low Mean Squared Error (MSE) of 0.035 and a Root Mean Squared Error (RMSE) of 0.187, along with an Average Displacement Error (ADE) of 0.25 meters and a Final Displacement Error (FDE) of 0.40 meters. Additionally, the model's Trajectory Consistency Score (TCS) of 0.92 and Lighting Variability Robustness (LVR) score of 0.88 underscore its ability to maintain accurate and consistent predictions under varying lighting conditions. Although the proposed model sets a new benchmark for pedestrian trajectory prediction, it requires substantial computational resources for training and may require further optimization for deployment in real-time applications. Future work will focus on enhancing the robustness of the model to extreme weather conditions and occlusions, as well as improving computational efficiency. This study contributes to the advancement of pedestrian trajectory prediction, offering a robust and adaptable solution for complex and dynamic environments.