Frequency-Domain Evaluation and Estimation of Semi-Markov Disease Natural History Models

Introduction

Natural history models must often be estimated from data that do not directly capture disease transitions. Semi-Markov processes let the risk of progression depend on the time spent in a health state, which Markov models need extra states to represent. Their limitation is computational: discretizing time introduces approximation error, and simulating individuals introduces Monte Carlo error and a non-differentiable calibration objective.

Methods

We show that the model-implied prevalence, incidence and cause-specific mortality of a continuous-time semi-Markov model satisfy convolution equations that become a small linear system when Laplace-transformed. Solving it on a grid of complex frequencies and inverting with an accelerated Fourier series returns the targets without a time grid or Monte Carlo error, and yields exact first- and second-order derivatives for one additional linear solve each. These support Newton-type maximum likelihood estimation, gradient-informed Bayesian sampling, and local identifiability diagnostics. We apply our open-source implementation to a urinary bladder cancer model calibrated to SEER incidence and mortality by age and stage.

Results

The method reproduced an independent implementation to within its accuracy (maximum relative difference 8 × 10 ⁻⁶ ) and evaluated all targets, their gradient, and their Hessian in 1.3, 2.5, and 9.1 milliseconds; a one-year-cycle cohort approximation deviated by up to 42% and a two-million-person microsimulation by up to 811%. Maximum likelihood and Bayesian estimates agreed, and second-order information showed that the aggregate targets constrain where individuals go from preclinical states far better than how long they stay. Early-stage incidence and mortality showed systematic misfit across age.

Conclusions

The framework supplies existing calibration algorithms with fast, exact model evaluations and derivatives for models whose transition hazards depend on time in state rather than on age.

Highlights

• The age-specific prevalence, incidence, and cause-specific mortality implied by a continuous-time semi-Markov model can be computed in the frequency domain without time discretization and without Monte Carlo error.

• The same computation returns exact gradients and Hessians of the calibration objective, which enables Newton-type optimization, gradient-informed Bayesian calibration, and local identifiability diagnostics that derivative-free calibration cannot provide.

• The framework supplies model evaluations and derivatives to existing calibration algorithms rather than replacing them, as we demonstrate using a urinary bladder cancer model fitted to aggregate SEER data.

• Open-source software implements the framework, including competing risks and age-specific background mortality rates.