Early prediction of clinical response to checkpoint inhibitor therapy in human solid tumors through mathematical modeling

Article activity feed

1. 

   Version published to 10.7554/elife.70130 on eLife

   Nov 9, 2021
2. 

   eLife

   Sep 30, 2021

   Author Response:

   Reviewer #2:

   The study by Butner et al. leverages a previously derived mathematical model (Butner et al. Sci Adv 2020) to predict immunotherapy response using published clinical data from immunotherapy-treated cancer cohorts. The model was fitted to a calibration cohort (meta-analysis, n=189) and then applied to a smaller validation cohort (n=62, Welsh et al., JITC, 2020). The estimated model parameters were tested for their ability to classify responders and non-responders. Using the tumour volume estimated from CT scans as input for the model, the immunotherapy response was predicted with an accuracy of 81.4% (n=62) within 2 months from treatment onset.

   Modelling of the anti-tumour response under immunotherapy is a relevant approach to understand the dynamics behind this process. The results from this study …

   Author Response:

   Reviewer #2:

   The study by Butner et al. leverages a previously derived mathematical model (Butner et al. Sci Adv 2020) to predict immunotherapy response using published clinical data from immunotherapy-treated cancer cohorts. The model was fitted to a calibration cohort (meta-analysis, n=189) and then applied to a smaller validation cohort (n=62, Welsh et al., JITC, 2020). The estimated model parameters were tested for their ability to classify responders and non-responders. Using the tumour volume estimated from CT scans as input for the model, the immunotherapy response was predicted with an accuracy of 81.4% (n=62) within 2 months from treatment onset.

   Modelling of the anti-tumour response under immunotherapy is a relevant approach to understand the dynamics behind this process. The results from this study suggest that the model parameters for the tumour-cell killing rate and the ratio of cancer cells to cytotoxic cells are different on average between patients with objective responses and stable/progressive disease. The main advantage of this approach is that the estimations are derived solely using the CT scans to infer tumour volume. However, given that therapy response is characterized by a large tumour and immune heterogeneity, clonal selection over time, and importantly immune escape mechanisms, which were not considered in the model, a larger validation cohort is needed to confirm that the estimated parameters are robust predictors. Their predictive value also needs to be compared to current biomarkers of response.

   The point is well-taken. We agree that tumor response under immunotherapy is affected by a large range of heterogeneous factors, which to our knowledge to date have not all been included into a (any) single response rubric. This is probably because, from a practical standpoint, characterizing the full heterogeneity of all cancer within a patient is likely not as yet possible (e.g., full characterization of clonal heterogeneity would require removal and analysis of all tumor cells or nodules, probably through highly invasive measures). Different from those modeling efforts attempting to include as many possible factors as possible (in a way that we do not believe could ever been fully informed in a real-world clinic), we have attempted to only focus on the key mechanisms (after extensive literature research and also extensive scientific discussion with our experimental and clinical collaborators) that we believe are essential to understanding the overarching response, and to refine the model into a form that could actually be used in clinical settings.

   The focus of this work is not to validate the predictive ability of the model, which has already been well-established (PMIDs 32426472, 33398132), but rather to demonstrate an entirely new measure that can be used to inform the model: i.e., that IHC measurements are associated with model parameters and may be used to quantify them, instead of imaging-based assessment as done previously. This goal, combined with an already-established predictive model, is why we have shown comparisons of the relationship between model parameters/IHC informed from both methods in here, and focused on the tumor volume, which was the only “marker” available for all patients evaluated in this study.

   Interestingly, PDL1 positive cell percentage and CD8 T cell count were estimated based on the model parameters and compared between the different response groups. The average levels of estimated and observed T cell counts and %PLD1+ cells were comparable between the groups. However, to demonstrate correlation, the estimated and observed values need to be compared on patient level. This has the potential to be the focus and significance of the study, as it could be relevant in the absence of biopsy data.

   We appreciate the astute insight. We agree with the Referee that this would represent a key step moving forward. Unfortunately, we have been unable to obtain any per-patient data that contains both patient response to ICI therapy and also IHC measures of PD-L1 expression or CD8+ tumor-infiltrating lymphocyte counts from the same patients that would enable this direct comparison. The results we have shown are the result of careful consideration as to how we could best address this problem in the absence of such an ideal data set in the real world. That said, we are now working towards in-house collection of these data (we have added discussion of this ongoing work on lines 548-554 of the revised text), and we believe that the present study represents an advance towards this goal.

   Reviewer #3:

   This work proposed a non-linear mathematical model with a particular ordinary differential equation to capture the dynamics of tumor size over time in response to the immune microenvironment with treatment of checkpoint inhibitors. The parameters in the model are initially trained by a time-course dataset from six clinical trials consisting size changes over time of six types of tumors from 189 patients in response to the treatment of PD1 or PDL1, which were validated by an additional dataset from a study of 64 patients with non-small cell lung cancer. The authors further investigated the biological relevance of each parameter and found two of them μ and Λ were capable in classification of patients who are response or not to the treatment. However, the training procedure as well as the validation/testing of the model is not carefully evaluated, which could result in overfitting of the parameters to some datasets.

   The model fitting and validation were done as we did previously (PMID: 32426472). To help clarify, we have now added a reference to this study in the revised manuscript lines 222-224: “A unique fit (and thus set of parameter values) was obtained for the data from each individual patient, and more details on this procedure may also be found in (45).”

   Strengths:

   Instead of training a classification model to directly fitting tumor characters to the drug treatment in bulk level, this study built a non-linear mechanistic model to capture the dynamics of tumor size over time in response to tumor microenvironment and indirectly using its key parameters to classify the drug effects. This approach, integrated more intrinsic information at cell-cell interaction level, is potentially allowing build a more reliable predictive model across different cancers and treatments.

   We thank the Referee for the encouraging comments and also for the detailed assessment of our work.

   Weaknesses:

   1. The prediction power of model is high depended on the robustness of performance in different tumors at different stage under different treatment, however, this study did not provide data on the effects of tumor heterogeneity.

   This is a fair criticism.

   We emphasize that our intent in this work is not to prove the model works across all possible clinical scenarios, but instead is to demonstrate how model parameters correlate with, and thus may be informed by, histological measures. This work represents a key step towards this ultimate goal. By demonstrating the model works across the 6 different tumor types included in the calibration cohort (Table 1), as well as methods to inform the model from IHC, we believe this work represents an important step in this direction. Having said that, we have now added additional discussion to highlight this limitation, and to more accurately represent what or and may not be concluded from our results, lines 454-456, 528-530, and 548-554. We thank the Referee for this constructive comment.

   1. The parameter of proliferation constants (α) defined in the study is coupled and vary with dataset structure of each clinical trial, which should be evaluated independently by patients without the treatments or controlled data from in vitro experiments.

   We understand this concern, as our dataset collected in-house for the validation cohorts confirms that there is variation in tumor growth rates among patients prior to treatment. Unfortunately, per-patient tumor measurement data prior to treatment start was not available in the calibration cohort, so we were forced to make this approximation. Even if these data could be collected from an independent cohort, it would merely represent another, different average value, as there would be no reason to assign relation between individual patients between these independent sets. This different average value could, in principle, shift the absolute value of parameters shown, e.g., in Figure 3, but the absolute value of the separation between PR/CR and stable/progressive values would remain the same. Thus, this would not change the observed trends and conclusions we have drawn. Moreover, we believe that data obtained from in vitro experiments would be vulnerable to the same criticism: that it does not accurately represent per-patient growth rates, as again there would be no reason to assume relation between experiments and individual patients. This would instead just provide a third, different estimate of pre-treatment average growth rate, and the outcome would be the same as described previously.

   1. It is unknown how the parameters of the model were trained or validated in batches and whether parameters were overfitting to the datasets.

   We understand how this could be ambiguous as described, thank you for bringing this to our attention. We have added the necessary details in lines 222-224: “A unique fit (and thus set of parameter values) was obtained for the data from each individual patient, and more details on this procedure may also be found in (45).”

   The model parameters were not trained or validated in batches. Usually, this sort of analysis (e.g., k-fold) is used in the case of small datasets or when independent validation sets are unavailable, and is only valid when the training and validation sets are from the same population. Instead of taking this computational approach, we chose to work with our clinical collaborators to obtain additional, independent validation data for more robust validation. We agree with the assessment in Essential Revisions comment #2 that this is a “large-scale validation cohort”, and offers superior validation to cross-validation. We further note that, in the revised manuscript, we have provided additional analysis to better understand how model parameters may vary between cancer types; please see our reply to Reviewer #2, comment 8 for more details. This new analysis is discussed in the main text, lines 335-341 and shown in the new figure S3 of the revised manuscript.

   Regarding the Referee concern for “overfitting” possibility, it may be observed from Equation 1 that, mathematically, the overall shape of the time-dependent model curve may take one of three shapes, these are: (i) decreasing exponential, asymptoting to a value 1>x>0; (ii) increasing exponential, asymptoting to a value ρ∞>x>1; or (iii) exponential increasing uncontrollably towards (∞,∞). Please note that, although this third case is mathematically possible, it was not observed in our analysis; however, it is included here for completeness. Because of the inability of the functional shape to include many inflection points, in combination with the dimensionally reduced form of the model in equation 1, we are confident that the model is not overfit to each per-patient dataset.

   Read the original source
3. 

   eLife

   Sep 30, 2021

   Evaluation Summary:

   In this manuscript one novel model was constructed to be predictive of cancer immunotherapy based on three parameters proposed to be associated with treatment efficacy. The parameters are easy to fetch under the clinical setting, so the model is simple for application to help predict potential patients who would benefit from immunotherapy.

   (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

   Read the original source
4. 

   eLife

   Sep 30, 2021

   Reviewer #1 (Public Review):

   The authors demonstrate a translational mathematical model dependent on three key parameters for describing efficacy of checkpoint inhibitors in human cancer.The model parameters may serve as early and robust biomarkers of the efficacy of checkpoint inhibitor therapy on an individualized per‐patient basis.

   Read the original source
5. 

   eLife

   Sep 30, 2021

   Reviewer #2 (Public Review):

   The study by Butner et al. leverages a previously derived mathematical model (Butner et al. Sci Adv 2020) to predict immunotherapy response using published clinical data from immunotherapy-treated cancer cohorts. The model was fitted to a calibration cohort (meta-analysis, n=189) and then applied to a smaller validation cohort (n=62, Welsh et al., JITC, 2020). The estimated model parameters were tested for their ability to classify responders and non-responders. Using the tumour volume estimated from CT scans as input for the model, the immunotherapy response was predicted with an accuracy of 81.4% (n=62) within 2 months from treatment onset.

   Modelling of the anti-tumour response under immunotherapy is a relevant approach to understand the dynamics behind this process. The results from this study suggest …

   Reviewer #2 (Public Review):

   The study by Butner et al. leverages a previously derived mathematical model (Butner et al. Sci Adv 2020) to predict immunotherapy response using published clinical data from immunotherapy-treated cancer cohorts. The model was fitted to a calibration cohort (meta-analysis, n=189) and then applied to a smaller validation cohort (n=62, Welsh et al., JITC, 2020). The estimated model parameters were tested for their ability to classify responders and non-responders. Using the tumour volume estimated from CT scans as input for the model, the immunotherapy response was predicted with an accuracy of 81.4% (n=62) within 2 months from treatment onset.

   Modelling of the anti-tumour response under immunotherapy is a relevant approach to understand the dynamics behind this process. The results from this study suggest that the model parameters for the tumour-cell killing rate and the ratio of cancer cells to cytotoxic cells are different on average between patients with objective responses and stable/progressive disease. The main advantage of this approach is that the estimations are derived solely using the CT scans to infer tumour volume. However, given that therapy response is characterized by a large tumour and immune heterogeneity, clonal selection over time, and importantly immune escape mechanisms, which were not considered in the model, a larger validation cohort is needed to confirm that the estimated parameters are robust predictors. Their predictive value also needs to be compared to current biomarkers of response.

   Interestingly, PDL1 positive cell percentage and CD8 T cell count were estimated based on the model parameters and compared between the different response groups. The average levels of estimated and observed T cell counts and %PLD1+ cells were comparable between the groups. However, to demonstrate correlation, the estimated and observed values need to be compared on patient level. This has the potential to be the focus and significance of the study, as it could be relevant in the absence of biopsy data.

   Read the original source
6. 

   eLife

   Sep 30, 2021

   Reviewer #3 (Public Review):

   This work proposed a non-linear mathematical model with a particular ordinary differential equation to capture the dynamics of tumor size over time in response to the immune microenvironment with treatment of checkpoint inhibitors. The parameters in the model are initially trained by a time-course dataset from six clinical trials consisting size changes over time of six types of tumors from 189 patients in response to the treatment of PD1 or PDL1, which were validated by an additional dataset from a study of 64 patients with non-small cell lung cancer. The authors further investigated the biological relevance of each parameter and found two of them μ and Λ were capable in classification of patients who are response or not to the treatment. However, the training procedure as well as the validation/testing of …

   Reviewer #3 (Public Review):

   This work proposed a non-linear mathematical model with a particular ordinary differential equation to capture the dynamics of tumor size over time in response to the immune microenvironment with treatment of checkpoint inhibitors. The parameters in the model are initially trained by a time-course dataset from six clinical trials consisting size changes over time of six types of tumors from 189 patients in response to the treatment of PD1 or PDL1, which were validated by an additional dataset from a study of 64 patients with non-small cell lung cancer. The authors further investigated the biological relevance of each parameter and found two of them μ and Λ were capable in classification of patients who are response or not to the treatment. However, the training procedure as well as the validation/testing of the model is not carefully evaluated, which could result in overfitting of the parameters to some datasets.

   Strengths:

   Instead of training a classification model to directly fitting tumor characters to the drug treatment in bulk level, this study built a non-linear mechanistic model to capture the dynamics of tumor size over time in response to tumor microenvironment and indirectly using its key parameters to classify the drug effects. This approach, integrated more intrinsic information at cell-cell interaction level, is potentially allowing build a more reliable predictive model across different cancers and treatments.

   Weaknesses:

   1. The prediction power of model is high depended on the robustness of performance in different tumors at different stage under different treatment, however, this study did not provide data on the effects of tumor heterogeneity.

   2. The parameter of proliferation constants (α) defined in the study is coupled and vary with dataset structure of each clinical trial, which should be evaluated independently by patients without the treatments or controlled data from in vitro experiments.

   3. It is unknown how the parameters of the model were trained or validated in batches and whether parameters were overfitting to the datasets.

   Read the original source
7. 

   Version published to 10.1101/2021.05.10.21256419v1 on medRxiv

   May 10, 2021