Limitations of Current Machine-Learning Models in Predicting Enzymatic Functions for Uncharacterized Proteins
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (Arcadia Science)
Abstract
Thirty to seventy percent of proteins in any given genome have no assigned function and have been labeled as the protein “unknome”. This large knowledge shortfall is one of the final frontiers of biology. Machine-Learning (ML) approaches are enticing, with early successes demonstrating the ability to propagate functional knowledge from experimentally characterized proteins. An open question is the ability of machine-learning approaches to predict enzymatic functions unseen in the training sets. Using a set of Escherichia coli unknowns, we evaluated the current state-of-the-art machine-learning approaches and found that these methods currently lack the ability to integrate scientific reasoning into their prediction algorithms. While human annotators can leverage the plethora of genomic data in making plausible predictions into the unknown, current ML methods not only fail to make novel predictions but also make basic logic errors in their predictions. This underscores the need to include assessments of prediction uncertainty in model output and to test for ‘hallucinations’ (logic failures) as a part of model evaluation. Explainable AI (XAI) analysis can be used to identify indicators of prediction errors, potentially identifying the most relevant data to include in the next generation of computational models.
Article Summary
Many proteins in any genome, ranging from 30% to 70% of the genome, lack an assigned function. This knowledge gap limits the full use of the vast available genomic data. Machine learning has shown promise in transferring functional knowledge within isofunctional families, but it largely fails to predict novel functions not seen in its training data. Understanding these failures can guide the development of better machine-learning methods to help experts make accurate functional predictions for uncharacterized proteins.
Article activity feed
-
-
-
Computational models could help propagate the experimentally validated functional annotations to the correct portion of the protein space
I've wondered whether there might be interesting signatures that could differentiate between 1) inappropriate transfer of functional annotations to seemingly similar proteins vs 2) incomplete annotations, i.e. where the other protein(s) may indeed have the originally hypothesized function AND a second or additional functions on top of this that confuses interpretation. Do you know of any work or models that is attempting to address this?
-
very few of the proteins in UniprotKB54, the most widely used protein function database55, have been linked to experimental data
Curious if you might have a ballpark number in terms of % of entries for which there is direct experimental data? I've been trying to get a sense of this and agree that it's low, but haven't been able to track down a number.
-