Evaluation of NICT space weather forecast for extreme events in May 2024

In May 2024, multiple X-class solar flares, full-halo coronal mass ejections (CMEs), severe geomagnetic disturbances, and severe ionospheric negative storms were observed, and these space weather events caused some social impacts. In this study, we evaluate performance of space weather forecasts against these extreme space weather events, focusing on maximum forecast levels of X-class solar flares, geomagnetic disturbances with K ≥ 7, and ionospheric storms with I-scale = I3. Firstly, NICT’s forecasts were evaluated using multi verification indices such as proportion correct (accuracy), probability of detection (discrimination), false alarm ratio (reliability), frequency bias (bias), and equitable threat score (skill). As a result. It was found that NICT’s forecasts were characterized as having strong discrimination ability for X-class solar flares with moderate reliability, and high accuracy for ionospheric storms but subject to discrimination ability. While nearly perfect performance for K ≥ 7 geomagnetic disturbances forecast were achieved this time by using solar wind simulation (SUSANOO-CME) incorporating multiple earth-directing CMEs. Comparative analysis with forecast of other countries indicated that RWC Japan was the highest discrimination capability for X-class solar flares, while RWC USA archived highest skill for solar flare forecasts with zero false alarms. Perfect skill for K ≥ 7 geomagnetic disturbances were archived by RWC Japan, together with RWC Australia. Although continued efforts to improve forecast performance are still significant subject, we anticipate that the evaluation presented here will support appropriate implementation of effective measures to mitigate space weather disaster risks to modern social infrastructure.