Predicting T Cell Receptor Antigen Specificity From Structural Features Derived From Homology Models of Receptor-Peptide-Major Histocompatibility Complexes

  1. Martina Milighetti
  2. John Shawe-Taylor
  3. Benny Chain

  • Curated by eLife

    eLife logo

    Evaluation Summary:

    This manuscript reports a new approach to the important and challenging problem of predicting T cell receptor:peptide-MHC interactions, one that relies on molecular model building (with previously published tools) followed by feature extraction and machine learning. The strengths of the study are more conceptual than practical: the overall framework and analytical approach; a balanced, critical assessment of the method's performance (which does not shy away from negative results); some observations on TCR:pMHC docking geometry. On the practical side, the classifier does not appear to generalize well to unseen epitopes (neither do the published tools it's compared to), so at the end of the day it's not clear that it will be preferable to simpler sequence-based approaches.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

The physical interaction between the T cell receptor (TCR) and its cognate antigen causes T cells to activate and participate in the immune response. Understanding this physical interaction is important in predicting TCR binding to a target epitope, as well as potential cross-reactivity. Here, we propose a way of collecting informative features of the binding interface from homology models of T cell receptor-peptide-major histocompatibility complex (TCR-pMHC) complexes. The information collected from these structures is sufficient to discriminate binding from non-binding TCR-pMHC pairs in multiple independent datasets. The classifier is limited by the number of crystal structures available for the homology modelling and by the size of the training set. However, the classifier shows comparable performance to sequence-based classifiers requiring much larger training sets.

Article activity feed

  1. eLife

    Evaluation Summary:

    This manuscript reports a new approach to the important and challenging problem of predicting T cell receptor:peptide-MHC interactions, one that relies on molecular model building (with previously published tools) followed by feature extraction and machine learning. The strengths of the study are more conceptual than practical: the overall framework and analytical approach; a balanced, critical assessment of the method's performance (which does not shy away from negative results); some observations on TCR:pMHC docking geometry. On the practical side, the classifier does not appear to generalize well to unseen epitopes (neither do the published tools it's compared to), so at the end of the day it's not clear that it will be preferable to simpler sequence-based approaches.

    (This preprint has been reviewed by eLife. We include the public reviews from the reviewers here; the authors also receive private feedback with suggested changes to the manuscript. The reviewers remained anonymous to the authors.)

  2. eLife

    Reviewer #1 (Public Review):

    The manuscript by Milighetti et al aims to infer interactions between TCRs and peptide-MHC complexes using structural information about the potential interaction partners. They use template matching algorithms to infer the TCR structure of receptors that were not previously crystallized. They develop a classifier for this task with and find a performance comparable to the state-of-the art sequence based methods for TCR-pMHC pairing. Overall, the idea behind the work is good but we are still limited by structural information for TCR-pMHC complexes to make such analysis work.

    Strengths:
    The question of TCR-pMHC pairing is important and of great interest. However with the currently available data we still don't have any generalizable model for this prediction task. Most of the methods are sequence based and given that TCR and pMHC molecules are proteins, it is natural that their function (binding) should depend on their tertiary structures. The authors included structural information to build a classifier for TCR-pMHC pairing. Given the limited number of available co-crystallized TCR-pMHC structures, they used algorithms for TCR structure predictions that leverage sequence homology and use template matching for structure prediction. This method enables them to look at a much larger pool TCRs, which is a strength of this analysis. They also did extensive benchmarking of their method against prior sequence-based approaches and also provided some biophysical insight into TCR-pMHC interactions.

    Weaknesses:
    The method uses homology based algorithm (template matching) to infer the 3D structure of (uncrystallized) TCRs. Template matching can work well very similar TCRs and especially it can infer reliable structures in the framework regions. However, as the authors have pointed out, the CDR3 region is the part of a TCR that is more closely in contact with the presented peptides. From the structure part of the problem, we know that even a single mutation can have a significant impact on function (binding) in CDR3. So I am a bit hesitant as how appropriate these template matching algorithms are when the main goal is to infer changes in CDR3. On the other hand, as the authors show, their structure based algorithm has a comparable performance to sequence-based algorithms. The sequences used for this analysis are the ones that were matched by homology to the (limited) receptors with available 3D structures, which implies that the training set is confined to classes of very similar TCRs. It is then somewhat expected that this classification algorithm primarily picks up common sequence features of TCRs with a given binding preference, i.e. we are back to a sequence-based classification model. In unsupervised part of the analysis, I was hoping to see some coarse grained structural features of TCRs to be informative for TCR-pMHC pairing. I think lack of this result is due to data limitations.

  3. eLife

    Reviewer #2 (Public Review):

    A summary of what the authors were trying to achieve:

    The authors have developed an approach to prediction of T cell receptor:peptide-MHC (TCR:pMHC) interactions that relies on 3D model building (with published tools) followed by feature extraction and machine learning. The goal is to use structural and energetic features extracted from 3D models to discriminate binding from non-binding TCR:pMHC pairs. They are not the first to make such an attempt (e.g., Lanzarotti, Marcotili, Nielsen, Mol. Imm. 2018), but they provide a detailed critical evaluation of the approach that sets the stage for future attempts. The hope is that structure-based approaches may have better power to generalize from limited training data and/or to model unseen pMHCs.

    An account of the major strengths and weaknesses of the methods and results:

    The authors first report (section 4.1) that their structural and energetic features contain information on binding mode, highlighting complexes with reversed binding polarity, for example, and partly discriminating MHC class I from MHC class II structures. This is encouraging but not terribly surprising. Also, with regard to MHC I vs II discrimination, it is not clear how the class II peptides are registered with respect to one another. This needs to be done by alignment on MHC and mapping of structurally-corresponding peptide positions, since the extent of N- and C-terminal peptide overhangs varies between structures and is largely irrelevant to the docking mode. Interactions between the TCR and MHC are ignored in the feature extraction process; it's possible that including these interactions could improve performance. The authors state: "To be noted that not all structures could be successfully modelled by TCRpMHC models, and so we could not submit them to the feature extraction pipeline." It's unclear what effect this could have on the results: if the modeling failures are cases of structures for which no good CDR templates could be identified, then perhaps this could bias the results.

    Section 4.2 reports a negative result: unsupervised learning applied to the extracted features is unable to discriminate binding from non-binding complexes. This suggests that there is not likely to be a simple energetic feature, such as overall binding energy, that reliably discriminates the true binders. In Section 4.3, the authors turn to supervised learning, in which training examples inform prediction by a classifier. One finding is that the pure-sequence approach using Atchley-factor encoding of the TCR:pMHC outperforms the structure-based approaches, though not by much. A combined model incorporating Atchley factors and structural features does slightly better. These results are a little hard to interpret because we don't know how challenging the 10-fold internal cross-validation is. It doesn't sound like there is any attempt to avoid testing on TCR:pMHCs that are nearly identical to TCR:pMHCs in the training sets, and the structural database is highly redundant, containing many slight variants of well-studied systems. It's also not clear how overlap between the template database used for 3D modeling and the testing set was handled; my guess is that since the model building is an external tool this was not controlled. Together, these factors may explain why the results on independent test sets are, for the most part, significantly worse than the cross-validation results. Another take-home message from the independent validation is that the sequence-only method seems to outperform the sequence+structure or structure-only methods. Although these are described as "out-of-sample validation", it's not clear how different these independent TCR:pMHC examples are from the structure dataset on which the model was trained.

    Sections 4.4 and 4.5 report that prediction accuracy varies significantly across epitopes, and this is in part determined by sequence similarity to the structural database (which provides templates for modeling and also constitutes the training set for the model). In section 4.6, the authors determine that the model does not appear to be able to predict binding affinity (as opposed to the binary decision, binding versus non-binding). Finally, in section 4.7 the authors benchmark the predictor against two publicly available, sequence-based predictors. When predicting for epitopes present in their training sets, all methods do reasonably well, with the edge going to the sequence-based ERGO method. When predicting for epitopes not present in their training sets, none of the methods perform very well. The authors state that "these results suggest that the structure-based models developed in this study perform as well as the state-of-the-art sequence-based models in predicting binding to novel pMHC, despite learning from a much smaller training set." This may be true, but the predictions themselves are not much better than random guessing (AUROCs around 0.5-0.6).

    An appraisal of whether the authors achieved their aims, and whether the results support their conclusions:

    I'm doubtful that the proposed methods will form the basis of a practical prediction algorithm. In the absence of ability to generalize to unseen epitopes, simpler sequence-based approaches that leverage the ever-growing dataset of TCR:pMHC interactions seem preferable. I still think the study has value as a template and roadmap for future efforts, and a baseline for comparison. For me, a key unanswered question is whether the model-derived structural features are just a different, slightly noisier way of memorizing sequence, or actually contain orthogonal information that can enhance predictions. It might be possible to gain insight into this question by looking more carefully at the impact of model-building accuracy on performance (the authors use sequence similarity as a proxy, but this is confounded by overlap between the training set and the template set used for modeling). If model-building really adds something, it seems plausible that it does so by accurately capturing physical features of the true binding mode.

    A discussion of the likely impact of the work on the field, and the utility of the methods and data to the community:

    As state above, I think the present work will have a positive impact on the field of TCR:pMHC prediction by critically evaluating the structure-based approach (and also by testing two previously published methods on independent data). I am less convinced of the utility of the specific methods than of the overall conceptual framework, evaluation procedures, and training/testing sets.

    Any additional context you think would help readers interpret or understand the significance of the work:

  4. eLife

    Reviewer #3 (Public Review):

    Milighetti and colleagues describe a structure-based approach to predicting TCR antigen specificity. The area is of high priority in areas ranging from viral immunity to cancer. Overall performance is not greater than sequence based approaches but the authors correctly indicate that accumulation of new structures will lead to improvements. The work is intriguing as ultimately antigen recognition is a structural and biophysical problem, and while sequence-based approaches are enticing in their comparative simplicity, they can only work if the structural and biophysical data is reflected in how sequences are analyzed. That has not been possible yet, and structure based approaches are the way to get there. The manuscript is refreshing in its open take on performance.

    There are some obvious areas for immediate improvement: as the authors discuss, TCRs recognize composite peptide/MHC complexes, yet the analysis is focused on peptides alone. Much has been shown over the years about how peptides influence MHC and vice versa. Thus an accurate assessment needs to take into account the MHC. For example, in the A6-Tax/A2 system the authors use an example, even going back to 1996 we knew that CDR3 loops make strong contacts to the MHC. We have since learned that these strong contacts are a consequence of peptide conformation, which is a consequence of MHC binding... There are similar examples that have been studied at that level of detail. So considering the peptide in the absence of MHC *will* reduce performance. The paper is fine in asking the question "what happens if we do make that separation" - but a point for discussion and a lesson learned is the composite surface needs to be considered.

    There are some other areas that need clarification/comment: the models, for example - a 2A RMSD for a model that is be used for predicting specificity is not really that good. At a high level the footprint might look similar, but the physics of binding is highly dependent on interatomic distances, geometries, etc. The accuracy of structural modeling is a major area that needs improvement, and something the authors can and should comment on.

    The use of databases such as ATLAS is another strength and weakness. The strength lies in the ability to mine the data, but a limitation is found in the very simple brush which is used to classify such data. The data in these repositories relies on a wide range of measurements from different groups with different levels of accuracy and precision as well as thresholds about what is measurable and not.

    Lastly, the authors miss an opportunity to get back to the biophysics encoded by structure: we are still miles away from accurately and repeatedly predicting affinities from structure in *any* system, much less TCR-peptide/MHC structures. The section in the Discussion that addresses this is an opportunity to talk about where we are generally in relating structure to binding - still a long ways to go.

    So overall, I find this an exciting step in the right direction, with the paper identifying some holes and showing what is currently possible. A closer attention to the structural biophysics of TCR recognition of pMHC and protein-protein interactions in general would improve it.

  5. Version published to 10.3389/fphys.2021.730908
  6. Version published to 10.1101/2021.05.19.444843v1 on bioRxiv