Protein Set Transformer: A protein-based genome language model to power high diversity viromics
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (Arcadia Science)
Abstract
Exponential increases in microbial and viral genomic data demand transformational advances in scalable, generalizable frameworks for their interpretation. Standard homology-based functional analyses are hindered by the rapid divergence of microbial and especially viral genomes and proteins that significantly decreases the volume of usable data. Here, we present Protein Set Transformer (PST), a protein-based genome language model that models genomes as sets of proteins without considering sparsely available functional labels. Trained on >100k viruses, PST outperformed other homology- and language model-based approaches for relating viral genomes based on shared protein content. Further, PST demonstrated protein structural and functional awareness by clustering capsid-fold-containing proteins with known capsid proteins and uniquely clustering late gene proteins within related viruses. Our data establish PST as a valuable method for diverse viral genomics, ecology, and evolutionary applications. We posit that the PST framework can be a foundation model for microbial genomics when trained on suitable data.
Article activity feed
-
There has been growing caution around biological foundation models due to532potential biosecurity threats such as generating novel pathogenic viruses or guiding533gain-of-function viral mutations
It might be good to mention this rationale/justification and thought process earlier in the preprint so people understand why the code isn't available right now.
-
Figure 6.
Not sure if done on purpose, but this figure shows up really light and it's difficult to see some details
-
Abstract
When the code is available, it would be good to link it in the abstract since this is a cool tool people will want to use!
-
