Long Duration UAV Localization Cross Day and Night by Fusing Dual Vision Geo-Registration with Inertial Measurements

Visual-inertial navigation plays a crucial role in autonomous flight in the Global Navigation Satellite System (GNSS) denied environment. However, most existing visual-aided navigation systems use visible images for localization, which is incapable of compensating the inertial drifts during the night. In this work, we develop a long-term localization system by fusing visi-ble/thermal visual geo-registration with inertial measurements. To achieve cross day and night localization performance, we propose to match both the onboard visible and thermal images to a remote RGB map. To deal with the large differences between visible and thermal images, we inspected various visual features and propose to utilize a pretrained network for cross domain feature extraction and matching. To obtain an accurate position from vision registration, we demonstrate a localization error compensation algorithm with considerations about the camera attitude, flight height, and terrain height. Finally, the inertial and dual vision information is fused with a State Transformation Extended Kalman Filter (ST-EKF) to generate long-term, drift-free localization performance. Finally, we conducted actual long-duration flight experi-ments with altitudes ranging from 700 to 2400 meters and flight distances longer than 344.6 kilometers. Experimental results have demonstrated that the proposed method’s localization error is less than 50 meters in RMSE.