Quartformer: An Accurate Deep Learning Framework for Phylogenetic Tree Construction

The construction of phylogenetic trees is a fundamental task in evolutionary biology, with the inference of tree topologies being particularly challenging due to the super–exponential growth in the number of possible topologies as the number of species increases. Recent advances in deep learning have offered promising solutions to this challenge, especially approaches that first infer the topologies of all quartets using deep neural networks, and then assemble these quartets into a complete phylogenetic tree using quartet combination algorithms. Building upon these methods, we propose Quartformer, a novel framework that incorporates a sparse attention mechanism to enhance information interaction among quartets derived from the same tree. This allows each quartet to be inferred with richer contextual information, leading to more accurate topology predictions. Experimental results demonstrate that Quartformer outperforms both the original framework and the traditional maximum likelihood method RAxML on simulated datasets. We further validate its improvements on real–biological datasets and theoretically analyze the underlying reasons for its performance gains, which also inform future research directions. Additionally, we systematically evaluate optimization strategies for inference speed and their impact on overall performance. Overall, this work advances the application of deep learning in phylogenetic tree Construction and provides theoretical and practical insights for future methodological developments in the field.