Imprecision in tuberculosis infection outcomes; implications for non-inferiority vaccine trials

Introduction

Randomised trials comparing new vaccines against tuberculosis for use in neonates and infants, for whom Bacille Calmette-Guérin (BCG) vaccination is established practice, are using tuberculosis infection as the primary endpoint in a non-inferiority design. Markers of tuberculosis infection have imperfect sensitivity and specificity. Flaws in the non-inferiority trial design typically bias towards the null, which may result in falsely declaring non-inferiority.

Methods

We conducted a statistical simulation study to assess the impact of imperfect markers of tuberculosis infection on the interpretation of tuberculosis vaccine trials testing a non-inferiority hypothesis of an infection primary outcome in a two-arm randomized comparison. Data were generated in three 2-year cumulative risk of tuberculosis infection scenarios (2%, 5%, and 8%). The specificity of tests of tuberculosis infection was assumed to range from 100% to 85%, while the sensitivity was assumed to range from 100% to 64%. Log-binomial regression was used to estimate the relative risk of tuberculosis infection.

Results

With 100% sensitivity and specificity, type-I and type-II error were both approximately equal to the expected values (2.5% and 80%, respectively) in all three cumulative tuberculosis risk scenarios. With modest deviations from perfect sensitivity and specificity (95% for both), the risk of falsely declaring non-inferiority was 96.8%, 53.2%, and 27.8% in the 2%, 5%, and 8% cumulative tuberculosis risk infection scenarios, respectively.

Discussion

Tuberculosis vaccine non-inferiority trials using an infection primary outcome must be designed and interpreted accounting for the specificity of the tools used to measure infection, otherwise they risk declaring non-inferiority by default.

Key messages

• We conducted a statistical simulation study to assess the impact of imperfect sensitivity and specificity, in the primary outcome definition of tuberculosis infection, in vaccine trials testing a non-inferiority hypothesis.

• With only modest departures from perfect specificity in tuberculosis infection markers, the risk of falsely declaring non-inferiority is substantial.

• Vaccine trials testing a non-inferiority hypothesis with an infection primary outcome must account for the imprecision in the tools used to define the outcome, otherwise vaccines may be falsely declared non-inferior.