Assessing predictive accuracy of species abundance models in dynamic systems

Cascading human pressures and environmental change are affecting the natural dynamics of animal populations. Forecasting population abundances from time-series data provides an important avenue for testing competing ecological theories, and for supporting conservation planning and sustainable use, yet changing system dynamics may lead to erroneous predictions. Predictions from a model fitted and tested on historical system dynamics may become irrelevant if system dynamics change. Here we describe methods to test forecast skill in a rapidly changing system where model parameters are likely to be non-stationary. We presented two ways to split time-series into training and test datasets so that training data were: 1) contemporary to the testing data (‘modern split’), and 2) not contemporary to the testing data (‘legacy split’). As a case-study, we use animal abundance data from a 30-year time-series from a global warming hotspot. We tested our approach on four temperate reef species with different temporal trends. The case-study and simulation tests confirmed larger forecast errors in legacy split when compared to the modern split. We found that the legacy split had errors that could be more than for times larger for a species that had a rapid collapse in abundance and non-stationary population dynamics. As expected for the species with rapid collapse, the legacy split estimated much higher forecast error than the modern split. Our approach is applicable to a large range of species and systems, including fisheries and threatened species population monitoring, where rapidly changing environments present threats to both the species and management efficacy. Accumulated lessons from across species and systems should shed light on critical generalities that precede broader ecosystem change.