“A history of the world imperfectly kept”: Will we ever know how biodiversity has changed over deep time?

The fossil record is our only direct source of evidence for how life on Earth has waxed and waned over its long history. However, the fossil record is also incomplete and biased in many ways, after passing through biological, geological, and socio-economic filters. This means that we only possess snapshots of information, relating to specific places and times in Earth history, from which to try and understand large-scale biodiversity patterns. Over the last fifty years, a wide variety of methods have been developed to try and elucidate macroevolutionary patterns by accounting for fossil record structure or bias, with varying levels of success. Here we review the different approaches that have previously been applied to this problem, and discuss their strengths and weaknesses. We illustrate this by applying a selection of these methods to the global brachiopod fossil record of the Permian and Triassic. Finally, we highlight some avenues for future improvement, including (1) using simulations to investigate method efficacy, (2) designing studies around testable hypotheses, (3) embracing uncertainty, and (4) improving the integration of data from fossil and modern organisms. Although we cannot know exactly how biodiversity has changed over life’s history, it is clear that new innovations in computational palaeontology are helping us to improve the trustworthiness of our estimates of biodiversity through deep time.