MultiScale3D: A Multi-Scale Fusion Algorithm for Action Recognition

Camera-based behavior recognition is an effective means to prevent accidents. However, when a single camera is used to identify distant and close-range people's behaviors in a complex background, it is difficult to extract global and local features with different resolutions simultaneously, resulting in low recognition accuracy and slow recognition speed. In this paper, a multi-scale fusion motion recognition method, MultiScale3D, is proposed. First of all, based on YOLO-Pose technology, we put forward the remote and near synchronous bone key point extraction module, which improves the collaborative extraction speed of remote and near motion features. Then, a three-level ResNet module is designed to realize feature sampling with different resolutions, and the features of each level are splicing with the features of the other two levels respectively to achieve multi-scale feature fusion. The multi-scale information is weighted by the proposed MultiScale3D module, which further enhances the expression ability of key features. Among the experimental results of three public datasets, the model achieved an accuracy of 94.4% (state of arts) on NTU60-Xsub and 95.9% on FineGYM dataset. At the same time, the algorithm shows good robustness and generalization in the recognition of distant and close-range behaviors. Training code are available at https://github.com/Zhai-Mao/MultiScale3D