Net and gross efficacies; when a single RCT is not enough and multiple RCTs are impracticable

This work takes place at two interacting levels. One is a reflection on how to define therapeutic efficacy. The other one is a series of comparative simulation experiments confronting two definitions of therapeutic efficacy.

In real-world settings, treatment efficacy is typically estimated through randomized controlled trials (RCTs), yielding what we define as gross efficacy (GE). However, biases introduced by the enrollment process and imperfect results of randomization can affect GE estimates. In contrast, in a model informed drug development (MIDD) perspective, an in silico clinical trial (ISCT) is unbiased. It is conducted via computational simulations using a quantitative systems pharmacology (QSP) model of disease and treatments and a virtual population. The two (or more) compared treatments are given in turn to the same virtual patient, with the same environment, at the same time, in an as close as possible representation of the whole population of interest, resulting in the prediction of the net efficacy (NE). Because inter-individual and inter-occasion differences are removed, the NE prediction contains no external bias.

In order to explore the two paradigms, GE and NE, clinical trials were simulated using a disease model of advanced EGFR-mutated lung adenocarcinoma (aLUAD), a subtype of Non Small Cell Lung Cancer (NSCLC) and two treatment models, one for the investigational treatment, osimertinib in monotherapy, (A), and one for the control treatment, chemotherapy agents cisplatin and pemetrexed (B). The model used in the current simulations was a simplified model of a model that has proven its credibility in prospectively and blindly predicting accurately the results of real world phase 3 clinical trials, after a thorough validation procedure. The simplified model was applied in two different settings to a large virtual population. The first setting was designed to compute the net efficacy of treatment A versus B. The second setting was set-up to mimic 1,000 real-life, parallel group design RCTs, based on standard sampling theory to compare A and B. The number of patients per arm was selected to be equal to or greater than the currently published trials. As opposed to the first approach, the approach mimicking the RCT provides measures of the gross efficacy. A single RCT was randomly drawn to simulate a phase 3 trial which concludes a clinical development program. While the mean hazard ratios of clinical outcomes calculated by the GE approach and the hazard ratio of clinical outcome given by the NE approach are not very different, they could nevertheless lead to quite different assessments of the population benefit of the new treatment compared to a comparator and to different regulatory decisions. Further, some values of GE from the distribution of “observed” GE were rather far from the mean GE and the NE. The convergence of the GE estimate takes more than a couple of RCTs, showing that a single phase 3 trial is not enough. However, the number of trials sufficient for convergence is impracticable. This work also shows that a randomized phase 3 trial of a genuinely effective therapeutic can easily fail to demonstrate a difference between the two groups of treatment due solely to sampling fluctuations, incorrectly leading to the discontinuation of the investigational drug because of lack of chance. These findings suggest that while ISCTs cannot replace real-world RCTs, they provide valuable insights for establishing clinical development strategy, trial design, trial monitoring, and trial results interpretation.