Summary measures in non-inferiority clinical trials with a time-to-event outcome: an empirical comparison of power

Background : Time-to-event data is commonly used in non-inferiority clinical trials. While the hazard ratio is a popular summary measure in this context, the difference in restricted mean survival time has been theoretically shown to increase power and interpretability. This study aimed to empirically compare the power of the hazard ratio, difference in survival and difference in restricted mean survival time for non-inferiority clinical trials with a time-to-event outcome recently published in key clinical journals. Methods : 65 non-inferiority trials with a time-to-event outcome were included from two literature searches. Individual patient data were reconstructed and reanalysed. The hazard ratio, difference in survival and difference in restricted mean survival time were estimated under proportional hazards, using a Cox model for the hazard ratio and a flexible parametric survival model for the latter two summary measures. The latter measures were additionally estimated non-parametrically. Margin conversion was done using observed data in the control arm. Empirical power was defined as the proportion of trials that rejected the null hypothesis. Results : Difference in restricted mean survival time gave a potential power advantage over the hazard ratio with an empirical power increase of 7.7 (-5.4, 20.7) percentage points, and consistently outperformed difference in survival. Difference in survival was more powerful than the hazard ratio, but while difference in restricted mean survival time showed an empirical power advantage even when estimated non-parametrically, this was not generally true for difference in survival. Sub-group analyses consistently showed similar results. Results were more variable when converting margins under an exponential distribution, highlighting the importance of correct margin conversion. Conclusion : Our results empirically corroborate the theoretical advantage of difference in restricted mean survival time over the hazard ratio and difference in survival in non-inferiority clinical trials. This advantage is most apparent when estimation is done under proportional hazards. Choosing a relevant time point at which to evaluate the survival-based summary measures is an important aspect that should be carefully considered. We recommend incorporation of the difference in restricted mean survival time in the design and analysis of non-inferiority clinical trials when clinically justifiable. If appropriate, estimation under proportional hazards is preferable.