DUAL: deep unsupervised simultaneous simulation and denoising for cryo-electron tomography
Curation statements for this article:-
Curated by eLife
eLife Assessment
This important study presents a computational framework inspired by cycleGAN that enables denoising and realistic simulation of cryo-electron tomography data, addressing central challenges in tomogram cleaning, simulation, and downstream annotation. The approach coherently links several key problems in the field and demonstrates strong performance across benchmark datasets, with additional benefits for particle detection and missing-wedge completion, indicating broad relevance across electron tomography. The evidence is solid, with appropriate quantitative benchmarks and applications to diverse datasets supporting the main claims, although validation on additional, more recent tomograms would further strengthen the conclusions.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Recent biotechnological developments in cryo-electron tomography allow direct visualization of native sub-cellular structures with unprecedented details and provide essential information on protein functions/dysfunctions. Denoising can enhance the visualization of protein structures and distributions. Automatic annotation via data simulation can ameliorate the time-consuming manual labeling of large-scale datasets. Here, we combine the two major cryo-ET tasks together in DUAL, by a specific cyclic generative adversarial network with novel noise disentanglement. This enables end-to-end unsupervised learning that requires no labeled data for training. The denoising branch outperforms existing works and substantially improves downstream particle picking accuracy on benchmark datasets. The simulation branch provides learning-based cryo-ET simulation for the first time and generates synthetic tomograms indistinguishable from experimental ones. Through comprehensive evaluations, we showcase the effectiveness of DUAL in detecting macromolecular complexes across a wide range of molecular weights in experimental datasets. The versatility of DUAL is expected to empower cryo-ET researchers by improving visual interpretability, enhancing structural detection accuracy, expediting annotation processes, facilitating cross-domain model adaptability, and compensating for missing wedge artifacts. Our work represents a significant advancement in the unsupervised mining of protein structures in cryo-ET, offering a multifaceted tool that facilitates cryo-ET research.
Article activity feed
-
eLife Assessment
This important study presents a computational framework inspired by cycleGAN that enables denoising and realistic simulation of cryo-electron tomography data, addressing central challenges in tomogram cleaning, simulation, and downstream annotation. The approach coherently links several key problems in the field and demonstrates strong performance across benchmark datasets, with additional benefits for particle detection and missing-wedge completion, indicating broad relevance across electron tomography. The evidence is solid, with appropriate quantitative benchmarks and applications to diverse datasets supporting the main claims, although validation on additional, more recent tomograms would further strengthen the conclusions.
-
Reviewer #1 (Public Review):
Zeng et al.'s work links several key issues in Cryo Electron Tomography in ways that reinforce each other, inspired by the cycleGAN model, leading to very positive results across several benchmark datasets. The related topics include tomogram cleaning and simulations (two crucial areas in the field), with "spin-off" outcomes in automatic annotation and the completion of the missing wedge. The manuscript covers nearly all essential topics in Tomography, making it very comprehensive and potentially critical in the field. The generalization capabilities on the SHREC 2021 data set are very interesting, although difficult to quantify. I appreciate the approach, but I have serious concerns about some of the limitations of the results presented by the authors.
1. Simplified data versus nowadays challenging …
Reviewer #1 (Public Review):
Zeng et al.'s work links several key issues in Cryo Electron Tomography in ways that reinforce each other, inspired by the cycleGAN model, leading to very positive results across several benchmark datasets. The related topics include tomogram cleaning and simulations (two crucial areas in the field), with "spin-off" outcomes in automatic annotation and the completion of the missing wedge. The manuscript covers nearly all essential topics in Tomography, making it very comprehensive and potentially critical in the field. The generalization capabilities on the SHREC 2021 data set are very interesting, although difficult to quantify. I appreciate the approach, but I have serious concerns about some of the limitations of the results presented by the authors.
1. Simplified data versus nowadays challenging tomography data. It is acknowledged the difficulty in making general tests. In this work, the method shows excellent results on potentially simple data sets (the SHREC 2021, which was used for a benchmark in ET several years ago, but not much used since then) and, even more, the old Relion data set for picking).
2. Reproducibility by the average user. I have found many cases in which a specific software produces excellent results when run by the authors. Still, the average user is lost with the parameters and cannot reproduce these promising results. I propose that the authors address this issue by involving some experimental colleagues and ask them to repeat the work. This is a general concern that applies not only to this work but to many others. I think this consideration is crucial for a field that is growing very quickly and where method development happens at an extraordinary pace... but are all of them generally useful?
-
Reviewer #2 (Public Review):
This study introduces DUAL (Deep Unsupervised simultAneous denoising and simuLation), an unsupervised deep learning framework that jointly addresses denoising and realistic data simulation for cryo-electron tomography (cryo-ET). By leveraging a cyclic, unpaired learning strategy, DUAL avoids reliance on paired clean ground-truth tomograms, which represents a practical advantage over many existing supervised approaches.
Through extensive quantitative evaluations on benchmark datasets, together with qualitative and downstream analyses on diverse experimental tomograms, the authors show that DUAL performs robustly across both denoising and simulation tasks. For denoising, DUAL outperforms several widely used methods on the SHREC 2021 benchmark and achieves the highest particle-picking accuracy on the RELION …
Reviewer #2 (Public Review):
This study introduces DUAL (Deep Unsupervised simultAneous denoising and simuLation), an unsupervised deep learning framework that jointly addresses denoising and realistic data simulation for cryo-electron tomography (cryo-ET). By leveraging a cyclic, unpaired learning strategy, DUAL avoids reliance on paired clean ground-truth tomograms, which represents a practical advantage over many existing supervised approaches.
Through extensive quantitative evaluations on benchmark datasets, together with qualitative and downstream analyses on diverse experimental tomograms, the authors show that DUAL performs robustly across both denoising and simulation tasks. For denoising, DUAL outperforms several widely used methods on the SHREC 2021 benchmark and achieves the highest particle-picking accuracy on the RELION benchmark, indicating strong downstream utility.
For tomogram simulation, the study presents an unsupervised framework that jointly denoises experimental tomograms and generates synthetic volumes that closely resemble experimental data. These simulated tomograms outperform existing approaches in downstream tasks such as particle picking and enable additional applications, including missing-wedge compensation and cross-domain adaptation, without requiring labeled training data.
Overall, this work represents a substantial contribution to the cryo-ET field by providing a versatile unsupervised tool that reduces dependence on labor-intensive manual annotation, enables realistic data augmentation for training downstream models, and facilitates artifact mitigation. As such, DUAL has the potential to accelerate methodological development and progress toward comprehensive in situ structural biology.
-
Reviewer #3 (Public Review):
The paper is titled "DUAL: Deep Unsupervised Simultaneous Simulation and Denoising for Cryo-Electron Tomography." The authors provided two closely related code branches: one for denoising and one for missing-wedge correction. However, I did not find the simulation component. This is important, as the authors state that "the simulation branch provides learning-based cryo-ET simulation to generate synthetic tomograms indistinguishable from experimental ones."
In addition, no pre-trained models were provided. Given that the authors indicate that all training data are publicly available, sharing trained models together with references to the corresponding datasets would significantly facilitate evaluation of the reported performance.
The provided instructions are quite minimal and do not currently support …
Reviewer #3 (Public Review):
The paper is titled "DUAL: Deep Unsupervised Simultaneous Simulation and Denoising for Cryo-Electron Tomography." The authors provided two closely related code branches: one for denoising and one for missing-wedge correction. However, I did not find the simulation component. This is important, as the authors state that "the simulation branch provides learning-based cryo-ET simulation to generate synthetic tomograms indistinguishable from experimental ones."
In addition, no pre-trained models were provided. Given that the authors indicate that all training data are publicly available, sharing trained models together with references to the corresponding datasets would significantly facilitate evaluation of the reported performance.
The provided instructions are quite minimal and do not currently support reproduction of the reported findings. Compared with other cryo-ET software packages, the documentation is insufficient for installation and practical use. The software also does not consistently support standard cryo-ET file formats, particularly during inference for denoising and missing-wedge correction. In particular, volume preparation (in the first notebook of either pipeline) expects MRC input, whereas inference requires NPZ input. This inconsistency makes me believe that the shared code is not tested, and likely is a new wrap up that does not correspond to the version used to generate the results in the paper.
I also found the denoising workflow difficult to interpret. The notebooks require a "clean" target volume as input, but it is not explained how such a volume should be obtained. It is unclear whether any clean volume may be used or whether this should be simulated based on what the user expects to contain in the input. The logic about this introduced prior is not clear. Additionally, it is not clear whether the default configuration parameters provided in the notebooks correspond to those used in the paper or are intended as illustrative examples. I had requested the exact configurations used to produce the reported results to avoid ambiguity.
After many hours of trial, debugging, and experimentation, I was able to train a model for missing-wedge correction using the default parameters, although the process was slow and memory-intensive. However, despite sustained effort over two days, I was not able to perform inference using the trained model. Full-volume inference fails due to shape mismatches, as the network is trained on fixed-size 3D patches but does not support whole-volume inputs. Patch-based inference also fails at the stitching stage due to incompatible output dimensions, even when using standard volume sizes (e.g., 1024 × 1024 × 400 voxels) that work correctly during patch preparation.
While less central, I also found the training time to be close to prohibitive. The notebook sets the number of epochs to two for a toy example and notes that more epochs are required for real experiments. In practice, training for a single tomogram required approximately 16 hours of computation on two high-end GPUs to reach only six epochs, and likely more would be required (100s?). Due to the inference issues described above, I was not able to evaluate the trained model.
-
Author response:
Reviewer #1 (Public Review):
Zeng et al.’s work links several key issues in Cryo Electron Tomography in ways that reinforce each other, inspired by the cycleGAN model, leading to very positive results across several benchmark datasets. The related topics include tomogram cleaning and simulations (two crucial areas in the field), with ”spin-off” outcomes in automatic annotation and the completion of the missing wedge. The manuscript covers nearly all essential topics in Tomography, making it very comprehensive and potentially critical in the field. The generalization capabilities on the SHREC 2021 data set are very interesting, although difficult to quantify. I appreciate the approach, but I have serious concerns about some of the limitations of the results presented by the authors.
We thank the reviewer for the …
Author response:
Reviewer #1 (Public Review):
Zeng et al.’s work links several key issues in Cryo Electron Tomography in ways that reinforce each other, inspired by the cycleGAN model, leading to very positive results across several benchmark datasets. The related topics include tomogram cleaning and simulations (two crucial areas in the field), with ”spin-off” outcomes in automatic annotation and the completion of the missing wedge. The manuscript covers nearly all essential topics in Tomography, making it very comprehensive and potentially critical in the field. The generalization capabilities on the SHREC 2021 data set are very interesting, although difficult to quantify. I appreciate the approach, but I have serious concerns about some of the limitations of the results presented by the authors.
We thank the reviewer for the encouraging assessment of our work and for recognizing the potential importance of integrating tomogram denoising and simulation within a unified unsupervised framework. We appreciate the reviewer’s thoughtful evaluation and the concerns raised regarding the limitations of the current results. We address these concerns in detail below and have revised the manuscript to clarify the scope, evaluation strategy, and practical applicability of DUAL.
(1) Simplified data versus nowadays challenging tomography data. It is acknowledged the difficulty inmaking general tests. In this work, the method shows excellent results on potentially simple data sets (the SHREC 2021, which was used for a benchmark in ET several years ago, but not much used since then) and, even more, the old Relion data set for picking).
We appreciate the reviewer raising this important point regarding dataset difficulty and relevance. The SHREC 2021 dataset was selected because it is currently the most widely used benchmark simulated dataset for cryo-electron tomography and originates from the last SHREC contest specifically designed for evaluating cryo-ET analysis methods. It provides standardized simulated tomograms with known ground truth structures, which enables objective and reproducible quantitative comparison between different methods. The RELION ribosome dataset is also a commonly used experimental benchmark for evaluating particle detection performance. Nevertheless, we agree that demonstrating performance on additional recent and challenging datasets will further strengthen the evaluation of the method. In response to this comment, we have expanded the experimental evaluation in the revised manuscript by applying DUAL to additional recent cryo-ET datasets to further demonstrate its effectiveness on recent tomograms with more complex biological structures and imaging conditions.
Specifically, we added an evaluation on the CZII Cryo-ET Object Identification dataset, a popular competition in 2025 with more than 1,000 participants. This experiment complements the original SHREC 2021 and RELION ribosome benchmark results and shows that DUAL can also be successfully applied to more recent cryo-ET data. The quantitative results and representative visual comparisons (shown above in Figure 1 and 2) are provided in the new section 2.6.
(2) Reproducibility by the average user. I have found many cases in which a specific software producesexcellent results when run by the authors. Still, the average user is lost with the parameters and cannot reproduce these promising results. I propose that the authors address this issue by involving some experimental colleagues and ask them to repeat the work. This is a general concern that applies not only to this work but to many others. I think this consideration is crucial for a field that is growing very quickly and where method development happens at an extraordinary pace... but are all of them generally useful?
We fully agree with the reviewer that reproducibility and usability are critically important for computational methods in cryo-ET. In response to this concern, we substantially improved the accessibility and reproducibility of the DUAL framework and revised the accompanying documentation to make the implementation easier to inspect and use, as two experimental colleagues have used and reproduced the results. The updated software repository now includes improved documentation, a clearer README, practical tutorials, a method-to-implementation description, a code reference, and example workflows demonstrating how to reproduce the experiments described in the manuscript. We also provide pretrained models together with the configuration files used to generate the results reported in the paper. In addition, the revised documentation clarifies the data interface, domain convention, training workflow, model outputs, and the interpretation of the trained translators. We believe that these improvements will significantly facilitate reproducibility and make it easier for users to apply the method to their own datasets.
Reviewer #2 (Public Review):
This study introduces DUAL (Deep Unsupervised simultAneous denoising and simuLation), an unsupervised deep learning framework that jointly addresses denoising and realistic data simulation for cryo-electron tomography (cryo-ET). By leveraging a cyclic, unpaired learning strategy, DUAL avoids reliance on paired clean ground-truth tomograms, which represents a practical advantage over many existing supervised approaches.
We thank the reviewer for the positive summary of our work and for recognizing the advantages of the unsupervised framework in avoiding reliance on paired ground-truth data.
Through extensive quantitative evaluations on benchmark datasets, together with qualitative and downstream analyses on diverse experimental tomograms, the authors show that DUAL performs robustly across both denoising and simulation tasks.
We appreciate the reviewer’s recognition of the robustness of the framework and the evaluation strategy presented in the manuscript.
If feasible, a limited quantitative or qualitative comparison with one or more recently published deep learning approaches for cryo-ET denoising or simulation, such as CryoSamba, or DeepDeWedge, would further strengthen the evaluation and help contextualize DUAL’s performance.
We thank the reviewer for this helpful suggestion. As also recommended by the editor, we extended the experiments to include comparisons with recently proposed methods CryoSamba and DeepDeWedge. These comparisons were performed using the same evaluation metrics used in the current experiments so that the results remain directly comparable. The additional comparisons are added into section 2.6.
Specifically, DUAL was compared with CryoSamba for denoising and with DeepDeWedge for missing wedge compensation on the CZII Cryo-ET Object Identification dataset, a popular competition in 2025 with more than 1,000 participants. The results are shown above in Figure 1 and 2.
Reviewer #3 (Public Review):
The paper is titled “DUAL: Deep Unsupervised Simultaneous Simulation and Denoising for Cryo-Electron Tomography.” The authors provided two closely related code branches: one for denoising and one for missingwedge correction. However, I did not find the simulation component. This is important, as the authors state that “the simulation branch provides learning-based cryo-ET simulation to generate synthetic tomograms indistinguishable from experimental ones.”
We thank the reviewer for carefully examining the released code and for pointing out this source of confusion. We would like to clarify that, in the DUAL framework, simulation and denoising are the two simultaneous branches that are trained jointly, rather than separate sequential modules. The simulation branch learns the transformation from clean/simulated tomograms to realistic experimental cryo-ET tomograms, while the denoising branch learns the reverse transformation from experimental tomograms to the clean domain. Together, these two translators form the cyclic unsupervised learning framework described in the manuscript.
In the original repository release, the organization of the code may not have made this relationship sufficiently clear, which likely led to the impression that only denoising and missing-wedge correction components were provided. To address this issue, we have substantially revised the repository structure and documentation. The updated repository now explicitly documents the two simultaneous branches of DUAL, explains how the simulation and denoising translators interact during training, and provides clear instructions for reproducing both functionalities. We have also added a dedicated method-to-implementation guide, code reference, and tutorial examples that describe the usage of the simulation component and its role in generating realistic synthetic tomograms that are statistically and visually consistent with experimental cryo-ET data.
We believe these revisions clarify the implementation of the simulation branch and make the correspondence between the manuscript and the released code substantially easier to understand and reproduce.
In addition, no pre-trained models were provided. Given that the authors indicate that all training data are publicly available, sharing trained models together with references to the corresponding datasets would significantly facilitate evaluation of the reported performance.
We agree with the reviewer that providing pretrained models will greatly facilitate reproducibility and evaluation by other researchers. In the revised release of the repository, we have provided pretrained models corresponding to the experiments described in the manuscript together with clear references to the datasets used for training.
The provided instructions are quite minimal and do not currently support reproduction of the reported findings.
We appreciate the reviewer highlighting this issue. We have expanded the documentation substantially and provided detailed instructions describing the full workflow required to reproduce the experiments presented in the manuscript. In the revised repository, we added documentation that more explicitly connects the method described in the manuscript with the released implementation. The README summarizes the repository scope and data interface, the tutorial describes the practical workflow for preparing data and running training, and the method and code reference documents describe the mapping between the DUAL formulation and the main implementation files. We believe these additions will make the workflow clearer for users who wish to reproduce or adapt the experiments.
After many hours of trial, debugging, and experimentation, I was able to train a model for missing-wedge correction using the default parameters, although the process was slow and memory-intensive.
We thank the reviewer for investing significant effort to test the software and for reporting this observation. Training large 3D deep learning models on cryo-ET volumes can indeed be computationally demanding. We have clarified the computational requirements in the revised manuscript and provide guidance for efficient training and inference.
Once these points are addressed, I would return to my original request that the authors provide: 3. A fully solved and functional tutorial based on their updated notebooks with all the intermediate results.
We agree that a comprehensive tutorial will be extremely helpful for users. In the revised repository we have provided a complete end-to-end tutorial demonstrating the workflow from raw tomograms to the final outputs including simulated tomograms, denoised tomograms, and missing-wedge-corrected tomograms.
We once again thank the editor and reviewers for their insightful comments and suggestions, which have helped us significantly improve the manuscript and the accompanying software.
-
DUAL: deep unsupervised simultaneous simulation and denoising for cryo-electron tomography https://www.biorxiv.org/content/10.1101/2024.03.02.583135v1
-
DUAL: deep unsupervised simultaneous simulation and denoising for cryo-electron tomography https://www.biorxiv.org/content/10.1101/2024.03.02.583135v1
-