Real-Time Immersive Rendering Using NeuralRadiance Fields and Lightweight Transformers forEnhanced VR Experiences

Real-time immersive rendering is essential for delivering high-quality virtual reality (VR) experiences, yet it poses significantchallenges in terms of computational efficiency, rendering quality, and real-time responsiveness. Traditional methods oftenstruggle to balance these aspects, leading to either high latency or compromised visual fidelity. This paper introduces a novelframework that integrates Neural Radiance Fields (NeRFs) with lightweight transformers to address these challenges effectively.The proposed method, termed the Sparse Radiance Planner, comprises three innovative modules: the Manifold ConstrainedRadiance Encoder, the Agent-driven Scene Interaction Unit, and the Probabilistic Rendering Regularizer. These modulescollaboratively optimize the representation and rendering of complex 3D scenes, dynamically adapt to user interactions, andensure robustness under uncertainty. A key feature of this framework is the Uncertainty-Aware Rendering strategy, whichmodels and propagates uncertainty throughout the rendering pipeline. This strategy enables adaptive sampling and constrainedoptimization, significantly enhancing computational efficiency. Experimental results indicate that the proposed frameworkachieves high-fidelity rendering with low latency, demonstrating its suitability for next-generation VR applications. The integrationof NeRFs with lightweight transformers not only improves rendering quality but also ensures real-time performance, marking asubstantial advancement in the field of immersive VR experiences.keywords: Neural Radiance Fields, Lightweight Transformers, Sparse Radiance Planner, Uncertainty-Aware Rendering,Computational Efficiency