Multiomic foundation model predicts epigenetic regulation by zero-shot

Identifying genomic elements is challenging due to their cell type-specific nature and the influence of both genetic and epigenetic regulations. Unraveling the functional roles of genomic elements on genes and cellular states necessitates considerable resources. Here we propose the cis-Regulatory Element Transformer (CREformer), a large deep learning model to uncover functional and regulatory mechanisms through multimodal approaches. CREformer comprises 3 billion parameters, pretrained by integrating the advantages of both bulk and single-cell datasets, encompassing 55 billion nucleotides in bulk multiomic segments and 165 million single-cell paired multiomic profiles. After pretraining, CREformer performs all predictions in a zero-shot manner, which enables predictions in scenarios where no data is available for fine-tuning. This includes the computational inferences of master regulators, enhancers, gene regulatory networks (GRNs), and functional variants, as well as the in silico modeling of epigenetic perturbations, cellular state transitions, and disease treatments. Moreover, novel tumor treatment targets were discovered with CREformer and were validated in vitro. Overall, the foundational, zero-shot capabilities of CREformer have the potential to accelerate research into the comprehensive discovery of functional elements and their dynamics across a broad spectrum.