Select earthquake forecasting models demonstrate consistency with decadal prospective observations in California

Aftershock and earthquake forecasting systems are now operationalized by agencies in several countries, providing situational awareness to decision-makers and the public. To ensure that forecasts follow the best science available, the underlying models require rigorous testing against prospective data and benchmarking. Between August 2007 and August 2018, 27 time-varying forecasting models, including the types used for Operational Earthquake Forecasting (OEF), were automatically operated as part of a multi-institutional project by the Collaboratory for the Study of Earthquake Predictability (CSEP), generating an unprecedented database of over 50,000 next-day M _w ≥ 3.95 seismicity forecasts in California. Here, we comprehensively evaluate their predictive skills against 597 prospective M _w ≥ 3.95 earthquakes using community-vetted statistical methods. We find that approximately one-quarter, two-thirds, and 95% of the models produced forecasts consistent with the number, locations, and magnitudes of observed earthquakes, respectively, marking a significant milestone in earthquake predictability research and building confidence in their use for continuous OEF. Furthermore, newer models and models calibrated on small(er) earthquakes exhibited the best relative performances, demonstrating improved earthquake forecasting over time. We provide detailed recommendations and benchmarking data to support the development of OEF systems worldwide.