Emergent Vision Technology: 3D Human Pose Estimation for Single-Pixel Imaging (SPI)

Applying 3D human pose and body shape details from a single monocular image presents a significant challenge in computer vision. Traditional methods that rely on RGB images often face constraints due to varying lighting conditions and occlusions. However, advancements in imaging technologies have introduced new techniques, such as single-pixel imaging (SPI), which can overcome these limitations. SPI is particularly effective in capturing 3D human pose in the Near-Infrared (NIR) spectrum. This wavelength can penetrate clothing and is less affected by lighting variations than visible light, providing a reliable means to accurately capture body shape and pose data, even in challenging environments. In this work, we explore using an SPI camera operating in the NIR range, with Time-of-Flight (TOF) technology at wavelengths of 850-1550 nm, to detect humans in night-time environments. Our proposed system employs SPI for depth estimation and feature extraction in humans. These features generate point clouds integrated into a 3D body model (SMPLX) via 3D body shape regression. This process utilizes deep learning techniques based on self-supervised 3D human mesh methodologies. We constructed a laboratory scenario simulating night-time conditions to evaluate the efficacy of NIR-SPI 3D image reconstruction. This setup allowed us to test the feasibility of using NIR-SPI as a vision sensor in outdoor environments. By assessing the results obtained from this setup, we aim to demonstrate the potential of NIR-SPI as an effective tool for detecting humans in night-time scenarios and accurately capturing their 3D body pose and shape, with future applications in environmental rescue.