FLiTrak3D: Improved deep-learning-based 3D insect flight kinematics tracking using spatial and temporal encoding

Quantitative measurements of insect flight behaviour are essential for understanding the biome-chanics, control, and ecology of flight, yet obtaining such measurements under free-flight conditions remains challenging. Small body size, rapid wing motion, visual symmetry, and frequent occlusions complicate three-dimensional pose estimation, often requiring restrictive experimental setups or substantial manual annotation.

We present FLiTrak3D , an open-source Python package for estimating insect flight kinematics from multi-view videography. It combines machine-learning-based markerless tracking with biomechanical modelling to reconstruct and parametrise insect body and wing motion. The workflow integrates image preprocessing, including dynamic image cropping and enhancement, two-dimensional bodypart localisation, three-dimensional reconstruction, and optimisation-based skeletal fitting.

A key innovation is the use of spatio-temporal encoding across synchronised camera views and adjacent frames to improve neural-network awareness of spatial and temporal context during bodypart localisation. Multi-view image stitching allows the network to use cross-view spatial relationships, reducing left–right bodypart misidentifications, while temporal encoding stacks consecutive greyscale frames into RGB images provide short-term motion information.

A species-specific skeleton is then fitted to reconstructed keypoints to enforce kinematic constraints, and estimate body and wing orientations. We demonstrate the approach using free-flying Aedes aegypti mosquitoes, achieving bodypart localisation errors close to human labelling, and realistic flight kinematics.