From Range-Angle Maps to Poses: Human Skeleton Estimation from mmWave Radar FMCW Signal

Human skeleton estimation technology is indispensable in several fields, including medical monitoring and sports biomechanics analysis. To overcome the drawbacks of traditional optical sensors and wearable devices, which include light sensitivity, privacy concerns, and usability challenges, we introduce a non-contact human skeleton estimation system that uses dual millimeter-wave (mmWave) Frequency-Modulated Continuous-Wave (FMCW) radar, termed as mm-HSE. We start by creating a dual-node mmWave FMCW radar data acquisition platform. In three distinct environments—a hallway, a meeting room, and an open space—data are collected from 12 participants, resulting in 30,000 range-angle maps using a customized signal processing pipeline. We then present a two-stage network for human skeleton estimation and optimization. In Stage 1, multi-scale spatiotemporal features are extracted from two input branches using a depthwise separable convolutional neural network augmented with a self-attention mechanism. Initial estimates of 21 skeletal keypoints are generated via a cross-modal attention fusion module. In Stage 2, we introduce a novel skeletal topology optimizer that leverages graph convolutional networks to refine keypoint positions. Experimental results demonstrate that mm-HSE achieves an average mean absolute error (MAE) of 2.78 cm. In cross-domain evaluations, the MAE remains consistently low at 3.14 cm, underscoring the model’s strong detection accuracy, environmental adaptability, and overall robustness.