Quantifying organellar ultrastructure in cryo-electron tomography using a surface morphometrics pipeline

Article activity feed

1. Version published to 10.1083/jcb.202204093

   Feb 14, 2023
2. 

   Review Commons

   Apr 25, 2022

   Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

   Learn more at Review Commons

   ---

   Reply to the reviewers

   1. General Statements [optional]

   We would like to thank the reviewers for their prompt and thoughtful input on our manuscript, and their willingness to participate in more portable review through ReviewCommons.

   2. Description of the planned revisions

   Reviewer #1, major comments:

   • A major concern is that the data are reported and analyzed on a per tomogram basis when many tomograms contain multiple mitochondria. Given that the mitochondria appear mostly well separated in Sup. Fig 1 with only a few connections visible, and the high degree of pleomorphism noted by the authors, I would strongly suggest that the authors use each mitochondrion as the basis for …

   Note: This rebuttal was posted by the corresponding author to Review Commons. Content has not been altered except for formatting.

   Learn more at Review Commons

   ---

   Reply to the reviewers

   1. General Statements [optional]

   We would like to thank the reviewers for their prompt and thoughtful input on our manuscript, and their willingness to participate in more portable review through ReviewCommons.

   2. Description of the planned revisions

   Reviewer #1, major comments:

   • A major concern is that the data are reported and analyzed on a per tomogram basis when many tomograms contain multiple mitochondria. Given that the mitochondria appear mostly well separated in Sup. Fig 1 with only a few connections visible, and the high degree of pleomorphism noted by the authors, I would strongly suggest that the authors use each mitochondrion as the basis for reporting their metrics rather than the FOV/tomogram as this would avoid mixing metrics from different mitochondria that may be in different states (e.g., fusion/fission). This would apply to data shown in Figures 3, 4, 5, and 6.

   We appreciate the reviewer suggestion to separate on a per mitochondrion vs per tomogram basis for our analysis. While we do not anticipate that this will significantly change the overall findings, we agree that splitting per mitochondrion will account for any possible variability between mitochondria within the given field of view. Furthermore, we anticipate that this will actually improve our analysis and statistical power by effectively increasing the total sample size per experimental group. For our next revision, we will divide surfaces on a per mitochondrion basis within a given tomogram, and re-run the full analysis pipeline. Additionally, per reviewer request, we will include an output histogram for each measurement per mitochondrion surface in a supplemental figure.

   • In Figure 3C the authors show the combined distribution of OMM-IMM distances within each condition. This may obscure some variability within populations. Individual histograms for all mitochondria should be included as supplementary material. Currently, it is difficult to judge if the peak of the combined distribution is appropriate and impossible to judge the variability between tomograms (preferably mitochondria, see above comment). Additionally, the shape of the distributions appears significantly different between conditions, suggesting that selecting a single peak value as representative and the basis for the statistical tests (Fig 3D) might not be appropriate. Please comment.

   We will include individual histograms for each measurement per mitochondrion surface in a supplemental figure.

   We agree that peak-based statistical tests limit our ability to quantify more complex differences, and this is why we chose to output histograms in addition to violin plots, so that shape differences can be observed qualitatively. A major challenge of shape-based statistical quantification is the assessment of independent samples. By using peak-based quantification, we could assume that each tomogram (and in the planned revision, each mitochondrion) is an independent sample, but for shape distribution this is inappropriate since there is more than one value represented per tomogram. Running a KS test with N equal to the number of tomograms yields no significance even in the visible cases where the shape appears very different.

   However, the number of triangles also poorly represents the number of independent samples, since 1) the number of triangles used to represent a surface is somewhat arbitrary and remeshing can change it dramatically and 2) Our chosen triangle size is considerably smaller than the visually observed feature size in order to allow effective vector voting in the pycurv AVV algorithm. The result of this is that when we use a KS test on the distribution of values per triangle, even visually identical distributions yield p-values below 10^-200.

   We do estimate the approximate smallest feature size during our calculations, since that is used to generate the radius used by pycurv in vector voting, to be 12 nm (the radius hit parameter in pycurv). During a public presentation of this work an audience member suggested that we might use the area implied by this feature size (~450 nm^2) as the size of an independent sample. This would yield around 1000 independent samples per tomogram. Because the choice of feature size is heuristic and manual, this is not as statistically sound as the peak-based metric, which is why we believe that the more conservative peak-based statistical testing is the gold standard for proving differences, but we believe this will be the most reliable way to quantify differences in shape of distributions. We plan to implement this quantification in our revision, and will evaluate whether it gives “expected” statistical results by a bootstrapping approach using subsampling of triangles from the same vs different mitochondria.

   We would welcome reviewer suggestions for additional shape-based metrics and will explore other potential metrics to capture shape as part of our revision. While our peak-based metrics demonstrate our ability to statistically capture small changes in ultrastructure with this method, shape-based quantification will significantly enhance the capability to capture finer changes in structure that may be critical to understand physiologically.

   Once this additional testing is complete, we will add a section to the results section describing choice of statistical framework. We also plan to generate a supplementary table showing the results of the peak-based quantification alongside all shape-based quantifications.

   • In Figure 4C-F, again combined distributions are shown. Authors should include individual histograms for all mitochondria as supplementary material. The diversity of distributions in the metrics are more pronounced than the distances in reported in Fig 3, again making assessment of variability difficult and raising doubt about using the single peak value.

   We will include individual histograms for each measurement per mitochondrion surface in a supplemental figure.

   As we describe above, we will make test several options for distribution-based statistical quantifications and incorporate the results in the manuscript. We expect them to be useful for every measurement we make.

   • It would be helpful to include the curvature or curvedness of the OMM for each mitochondrion in the supplementary material. The data to correlate OMM curvature with elongated/fragmented mitochondria should be available and might be of interest to some readers.

   We will calculate curvedness of the OMM for each mitochondrion and include these data in the supplemental material. The inverse of the curvedness of the OMM gives a reasonable approximation of the radius of the mitochondrial “tube”, a feature which can be challenging to quantify fully automatically, and we agree that this may be of particular interest to some of our readers – particularly if morphology changes or stress-driven changes alter that radius in a statistically significant way!

   Reviewer #1, minor comments:

   • For all data, exact n per condition should be given (in text and captions as appropriate), not a range for the whole set.

   We will report the exact n per condition in text and in captions after we separate our data on a per mitochondrion basis and update the analysis.

   • Fig 5E middle, legend obscures some of the data.

   We will reformat the graph such that the legend does not obscure the data after we separate our data on a per mitochondrion basis and update the analysis.

   Reviewer #2, major comments:

   Barad, Medina et al. presents a new toolkit for the analysis of membrane ultrastructure in cryo-tomograms. More specifically, the toolkit is designed to compare curvature, angles and spacing between different membrane types in mitochondria. These analyses allow for the quantitative comparison of membrane features e.g. for different growth conditions. To demonstrate the utility of the toolkit tomogram datasets of mitochondria in the presence and absence of ER stress were analyzed. The authors conclude that ER stress affects mitochondria morphology through remodeling of the membrane structure. The presented biological results and statistics are convincing and show active mitochondrial membrane remodeling in the cell when exposed to ER stress. It is also clear that there is a need for more quantitative evaluation based on the wealth of tomographic image features and mitochondrial membranes are certainly a well-chosen application. For this purpose, the authors developed a new workflow even though most of the discussed analyses are very specific to mitochondrial structures. Therefore, broader applications of these tools to other organelles are not easily envisaged without significant adaption. In that context, the title and abstract overpromise a much more powerful utility that can be applied to any other membrane analysis. Rather it seems that the proposed workflow is more of a specific tool or a pipeline for mitochondrial inner and outer membrane analysis instead of a toolkit for general morphological analysis. Hence, the manuscript cannot be accepted in its current form. In particular, the structure needs a significant rework of editing to become more comprehensible.

   We appreciate the criticism that our workflow as implemented at the time of preprint is seemingly too focused on mitochondrial membranes and is not general. We’ve overhauled our workflow into a configurable (through a project YML file) scripted workflow that can take a folder with arbitrary segmentations and convert them into high quality meshes, followed by per-triangle quantification of the four primary metrics we describe in the manuscript: inter-membrane distance, intra-membrane through-space distance, curvature, and orientation. Generating fully automated visualization tools is more challenging, because which quantities are measured and how they are sub-classified (e.g., as we did for cristae, junctions, and IBM) is very project-specific; however, we did convert our visualization script into a library of utilities to combine tomograms into experiment objects, with methods to serialize for rapid access and functions for generating statistics and plots. Our converted visualizations script has been reorganized to act as an example of how similar questions could be asked for arbitrary membranes.

   We propose to further demonstrate the generality of this updated approach by segmenting several examples of another organelle, the autophagosome, found in our dataset and applying the workflow to them in a supplementary figure.

   The focussing to a method paper will also require more in-depth descriptions of the methodology in the main text. Although the code is deposited at github, there is no script-based workflow and description presented in the manuscript. Although Figure 1 puts the work into context of tomography, it remains very superficial on the image analysis. What are the input and output formats required for each step to follow the sequence of the workflow and at which steps critical interactive input is needed? What are the hardware requirements (CPU, GPU) or performance characteristics (CPU hours for certain operations)?

   In addition to the changes mentioned above, we also added a “Supplemental Table 1” detailing computational requirements and time for each step.

   We expanded on the description of this approach in the first paragraph of the results section:

   “With this strategy, we were able to segment 32 tomograms containing mitochondria, divided between the elongated and fragmented bulk morphology populations and the two treatment groups (Figure 2, Supplementary Figure 1). The segmentation output was fed into the fully automated surface morphometrics pipeline (Figure 2B, Supplementary Figure 2, Supplementary Table 1). The voxel segmentation was converted to high quality membrane surfaces using the screened poisson algorithm32. Next, these surfaces were converted into triangle graphs and curvedness was estimated using pycurv15, and the distances within and between surfaces as well as the relative orientations of different surfaces were estimated using the resulting graph. Finally, the quantifications for each tomogram were combined into experiments to allow aggregate statistics and visualizations. This 3D surface morphometrics pipeline is configurable for any segmented membrane and is available at https://github.com/grotjahnlab/surface_morphometrics.”

   We added a description of the up to date workflow in the methods section:

   “Software workflow

   The surface morphometrics pipeline is a python 3 scripted workflow with requirements that can be installed as a conda environment contained in an environment.yml file. The workflow is fully scripted and configurable with a config.yml file, and is run in 3 steps, with statistical analysis and visualization as an optional fourth step. First, a segmentation MRC file is converted automatically to a series of surface meshes formatted in the VTP file format. Second, for each mesh, the surface is converted to a graph (tg format) and curvature is estimated using pycurv. Third, orientations and distances between and within surfaces are calculated using the resulting graphs, and a CSV with quantifications as well as a final VTP surface file is output with all quantifications built in. Fourth, the outputs from multiple tomograms are combined for visualization and statistical analysis. Times and computational requirements are shown in supplementary table 1.”

   Figures 3-7 contain colorful 3D renderings of the measured quantities. In addition, they are filled with histograms of every possible quantitative parameter, which often are not very significant or different between. The authors should focus the main results and the figures to show the most relevant and significant findings and put the remaining panels and results into the supplement.

   Figures 3-7 were organized around the different methodologies (inter and intra-membrane spacing, curvature, orientation) but we agree that focusing to the main results of each methodology is sufficient to show the value of these results. We propose to address this criticism by moving figure 4D,F (inter-crista and junction spacing), figure 6 E,G (the junction measurements) and Figure 7 to supplemental figures. These supplemental figures will also be joined by the previously requested OMM curvature analysis and our proposed analysis of autophagosomes.

   One of the key steps is the generation of a smooth surface from a segmented membrane, there is a question whether true membrane disruptions will be smoothed and may be overlooked in this approach. When these disruptions present true membrane ruptures, they may be of particular biological importance. The authors should support the choice and selection of the smoothing parameters in order to illustrate this potential pitfall.

   The smoothing and hole-filling parameters are now configurable using the point_weight and extrapolation_voxels parameters in the config.yml file. Notably, the surfaces used for quantification used minimal smoothing, and any triangles more than a single voxel away from the point cloud were deleted, in order to ensure that the quantifications were minimally impacted by “hallucinated” surfaces. Additionally, the following text was added to the methods section discussion surface reconstruction:

   “A surface mesh was calculated from the oriented point cloud using the screened Poisson algorithm32, with a reconstruction depth of 9, an interpolation weight of 0.7, and a minimum number of samples of 1.5. These settings were chosen to maximize correspondence to the data, rather than smoothness. The resulting surface extended beyond the segmented region, so triangles more than 1 voxel away from the point cloud were deleted. Interpolation weight (point_weight) and the mask distance (extrapolation_voxel) are both configurable in the surface morphometrics pipeline if more aggressive smoothing and hole filling are desirable.”

   Throughout the manuscript, the authors mention statistical significance several times and one of the main aims of the study is perform statistical hypothesis testing. It is important to specify the significance test (not only in the methods) and the p-value in order to support this claim. In the manuscript, the authors use exclusively the Mann-Whitney test. What is the rationale for choosing this test? Have the authors considered comparing the total distributions and not just the peaks with e.g. a Kolmogorov-Smirnov test? For a statistical methods paper, there are also no discussion on error analysis.

   This was a common concern raised by both reviewers, and we agree that a test based on total distribution would be more powerful than only looking at peaks. We address the use of the Kolmogorov-Smirnov test and the limitations we have run into thus far in our response to reviewer 1 in detail. In brief, KS tests tend to vastly overestimate statistical significance because the number of samples (the number of triangles) is vastly larger than the true number of independent features sampled in the data, so that even very similar looking distributions such as those in figure 5C yield p values in the range of 10^-200. We propose several approaches to better estimate the number of independent variables. We will also use a random subsampling approach within individual mitochondria to ensure sampling from the same distribution does not yield statistically significant results.

   In addition to testing additional approaches to incorporate KS testing (based on estimation of number of independent features in each tomogram), we propose to improve our peak-based statistics by estimating a standard error for the peak of each tomogram using a bootstrap approach, getting the peaks from different random subsamples of triangles.

   Reviewer #2, minor comments:

   https://github.com/grotjahnlab/surface_morphometricsshould include an example data set or tutorial for dissemination.

   We are in the process of uploading all frame-averaged tilt series, tomograms, segmentations, and reconstructed surfaces to EMPIAR. Additionally, we propose to implement a complete tutorial including a single tomogram for readier workflow testing, separate from the complete data upload.

   3. Description of the revisions that have already been incorporated in the transferred manuscript

   Reviewer #1, major comments:

   • As the work reported here is heavily computational, additional details about the computer hardware used and the time it took for the calculations to complete would be helpful for readers considering applying the code to their own data.

   We appreciate the suggestion and included Supplementary Table 1 in the supplemental material outlining the computation time per step in our analysis pipeline:

   “Supplemental Table 1. Approximate time and for each step of the surface morphometrics workflow.

   Representative times and computational resources used for each step of the surface morphometrics workflow for each tomogram (unless otherwise noted) by the authors. Most time-intensive calculations were run in parallel on a compute cluster for each tomogram.

   Step

   Human Time (HH:MM)

   Computational Wall Clock time (HH:MM)

   CPU Cores Used

   RAM Used

   Automated initial segmentation (TomoSegMemTV)

   00:10*

   00:10*

   8

   64GB

   Manual segmentation cleanup and classification

   03:00

   N/A

   8

   64GB

   Point cloud conversion and mesh generation

   00:01

   00:03

   4

   16GB

   Graph generation and curvature estimation (pycurv)

   00:01

   01:40

   16

   128GB

   Distance and orientation measurement

   00:01

   00:10

   16

   128GB

   Assembly of outputs from multiple tomograms into dataframes and serialization

   00:01

   00:10

   1

   16GB

   Visualizations and statistical tests

   00:01

   00:10

   1

   16GB

   * Tomosegmemtv is sometimes run iteratively with different settings to improve output. 10 minutes is approximately the time taken for a run without iteration, in the case of good output.”

   Reviewer #1, minor comments:

   • Pink and purple very close, consider alternative pair of colors or different shades to distinguish OMM and IMM

   We kept OMM as purple but changed IMM to orange for Figure 3-7, and will make the associated changes to Figure 2 and Supplementary Movie 1 on final submission.

   • Orientation of scaleboxes/scalebars should be consistent per figure panel. If knowledge of the axes is important to the reader, these should be included as well.

   We followed the reviewer’s suggestion and updated the scale cubes to be standardized per panel.

   • In the last sentence of the introduction, the term "organellar architectures" is used, instead of the previously defined "membrane ultrastructure." Consider changing for clarity.

   We changed “organellar architectures” to “membrane ultrastructure” in the last sentence of the abstract.

   • Inconsistent use of the phrase "cryo-electron tomography" after defining and using "cryo-ET"

   We changed all instances of “cryo-electron tomography” to “cryo-ET” after defining in the first instance in the introduction.

   • Authors argue that the distinction between curvedness and curvature is important and that curvature is less appropriate in this context, but then use curvature in the abstract, throughout introduction and in the results section. Usage can be improved for readability.

   We changed all instances of “curvature” to “curvedness” throughout the text and figure legends.

   • In section "Development of a framework to automate quantification of ultrastructural features of cellular membranes" the second last sentence should read "... higher quality membrane surfaces as compared..."

   We changed “surface” to “surfaces” in text.

   • In section "IMM curvedness is differentially sensitive to Tg treatment in elongated and fragmented mitochondrial networks" the fourth sentence should perhaps read "... despite apparent visual differences, no significant..."

   We changed “difference” to “differences” in text.

   • The term "cell's growth plane" is not clear from the text nor from Fig 6A. Do the authors mean surface of the substrate the cell is growing on?

   We clarified and further defined the “cell’s growth plane” in the text by adding the following phrase:

   “… the cell’s growth plane (i.e. the plane of electron microscopy grid substrate to which the cell is adhered) (Figure 6A).”

   • In Materials and Methods:

   • The authors report that manual back-blotting was used in a Vitrobot. This is non-standard usage and more details should be provided.

   We added the following description to clarify our manual back-blotting procedure on the Vitrobot:

   “After 8 hours of incubation, samples were plunge-frozen in a liquid ethane/propane mixture using a Vitrobot Mark 4 (Thermo Fisher Scientific). The Vitrobot was set to 37° C and 100% relative humidity and blotting was performed manually from the back side of grids using Whatman #1 filter paper strips through the Vitrobot humidity/temperature chamber side port. The Vitrobot settings used to disable automated blotting apparatus were as follows: Blot total: 0, 2; Blot force: 0, 3; Blot time: 0 seconds.”

   • In section "Fluorescence Guided Milling" in the third sentence, the word "based" is repeated, second can be removed.

   We deleted the second instance of “based” in this sentence.

   • Symbol for degree (or the word degree) should be added to angular increment and tilt range for clarity.

   Added degree symbols to the following sentence in the “Tilt Series Data Collection” portion of the materials and methods:

   “Tilt series were acquired using SerialEM software (Mastronarde, 2005) with 2° steps between -60° and +60°.”

   • Capitalization of TomoSegMemTV is inconsistent.

   We changed all mentions to TomoSegMemTV.

   • Fig 3 title - consider replacing "Inter-mitochondrial membrane..." with "Intra-mitochondrial membrane..." for clarity.

   We clarified this point by changing “Inter-mitochondrial membrane distance” to “Distance between inner and outer mitochondrial membranes” in the figure legend:

   “Figure 3. Distance between inner and outer mitochondrial membranes is dependent on mitochondrial network morphology and presence or absence of ER stress.”

   • Fig 3C caption - should explicitly state it is a combined histogram and that the dashed lines correspond to the peak of the pooled data.

   We changed “Quantification of” to “Combined histogram of” and added the sentence ” to each of the relevant figure captions (Fig. 3c, 4c-f, 5b-e, 6d-g, 7c):

   “Dashed vertical lines correspond to peak histogram values of pooled data”

   • Fig 6B and 6C caption - upper and lower parts not explicitly described.

   We modified Fig 6B&C caption to more clearly describe the figure panel:

   “(B) Two representative membrane surface reconstructions of lamellar Tg-treated elongated mitochondria, colored by angle of IMM relative to OMM.

   (C) Two representative membrane surface reconstructions of a less rigidly oriented Tg-treated elongated mitochondria, colored by angle of IMM relative to the growth plane of the cell.”

   Reviewer #2, major comments:

   Title and abstract need to be toned down not to overpromise a very general toolkit. The presented method may be a tool or a collection of scripts - a toolkit can be used to address other types of (membrane) analysis problems. In the end, the analysis builds to a large extent on the previous developments and implementation of PyCurve. Perhaps, the most interesting contribution here is the application of the mesh generation by the Poisson reconstruction method to the segmented membranes, which is, however, well implemented in the used pymeshlab framework. The computation of distances and angles is straightforward.

   We appreciate this critique and do not want to overpromise with our work, although we believe the overhaul to a fully configurable workflow addresses the primary concern. We are quite clear in the text that we build on top of pycurv, and recommend citation of the original tool as well as our pipeline in the github repository as a result. With that said,

   We have changed the title as follows:

   “Quantifying mitochondrial ultrastructure in cryo-electron tomography using a surface morphometrics pipeline”

   We have also renamed our method to the surface morphometrics pipeline to reduce over-implication of generality, and made other small changes to increase degree of detail about what our method is resolving.

   When reading the manuscript, the reader is left in the open whether this is a method paper or a biological results paper. The title/abstract suggests that this is a method paper and the manuscript is more of a mitochondrial membrane report in ER stress. Therefore, the title/abstract does not reflect the manuscript very well.

   We aim to use this manuscript to describe the development of a workflow that enabled novel and interesting biological results. We adjusted the title to better match the combined development of a new pipeline and application to an interesting biological system as proof of concept:

   “Quantifying mitochondrial ultrastructure in cryo-electron tomography using a surface morphometrics pipeline”

   The manuscript also requires substantial structural editing. Several references to Figures are not appearing in the text in the order that the Figure panels are built. Excessive cross-referencing of figures also make the manuscript hard to read.

   We simplified our referencing of figures and made sure the text matched the order of the figure panels.

   The exact morphological discrimination between fragmented and elongated mitochondria is not easily understood from the results section. What is really meant by blinded manual classification? It only became clear when reading the methods. The results section should stand on its own. How is the overall population between fragmented and elongated cells is affected after Tg application?

   To clarify our methodology for blinded classification of mitochondrial network morphologies we included the following text:

   “We categorized cells for mitochondrial network morphology by blinded manual classification in which five researchers were given fluorescence microscopy images of exemplar network morphologies (elongated and fragmented) as references to assign morphologies to the experimental fluorescence micrographs.”

   We targeted similar ratios of elongated and fragmented cells in both vehicle and Tg treated conditions for tomography, but qualitatively saw the expected increase in the elongated population to what has been previously described during Tg treatment. Because of our single cell targeting approach we did not quantify the population shift.”

   Similarly, what is meant by manual classification of IMM, OMM and ER? Is there any clustering involved?

   Our automated segmentation approach labels all membranes, and the separation of the IMM, OMM, and ER membranes is done by an expert user selecting and relabeling each membrane based on cellular context (e.g. IMM is inside of OMM and contains cristae). We have added the following text to clarify our methodology for manual classification of IMM, OMM, and ER:

   “This was followed by manual labeling of membranes into mitochondrial IMM and OMM and ER membrane based on cellular context, as well as manual cleanup of individual membrane segmentations using AMIRA software (Thermo Fisher Scientific).”

   __Reviewer #2, minor comments: __

   What is meant by growth plane? This term is not defined in the manuscript.

   We clarified and further defined the “cell’s growth plane” in the text by adding the following phrase:

   “… the cell’s growth plane (the plane of electron microscopy grid substrate on which the cell is grown) (Figure 6A).”

   What is meant by vehicle treatment? There is no explanation in the main text of the manuscript.

   We clarified and further defined vehicle treatment in the main text by adding the following:

   “We applied our correlative approach to identify and target specific Tg-treated and vehicle (media with DMSO) treated MEFmtGFP cells with either elongated or fragmented mitochondrial network morphologies for cryo-FIB milling and cryo-ET data acquisition and reconstruction.”

   Have the authors noticed/calculated any differences in the width of the cristae?

   We measure this difference in figure 4C (Intra-crista distance). We found significant changes in width/intra-crista distance in response to Tg treatment in both elongated and fragmented morphologies.

   Methods: Automated surface reconstruction: "In cases where the resulting surface was very complex, the surface was simplified..." How was the complexity determined?

   With the updated state of the software, we simplify all surfaces to generate a maximum of 150,000 triangles. This has minimal effect on very small surfaces, but greatly speeds computation on very large surfaces. We corrected the language to match this:

   “The resulting mesh was simplified with quadric edge collapse decimation to produce a surface that represented the membrane with 150,000 triangles or fewer.”

   Methods: Calculation of distances between individual surfaces: "For surfaces with small numbers of triangles, this was accomplished using a distance matrix...". What is the threshold for a small number of triangles?

   As part of our software overhaul we have changed to always using a more memory-efficient KD tree based quantification, since the additional speed for the distance matrix approach is minimal when there are few enough triangles for it to be appropriate, and the hardwired cutoff was not as flexible for different hardware configurations. The updated text is below, but to satisfy any potential reviewer curiosity, the decision was made when the required distance matrix would use more than 128GB of memory. In the case of two identically sized surfaces, this crossover happens when there are approximately 45,000 triangles in each surface.

   “For calculations of distances between respective surface meshes, the minimum distance from each triangle on one surface to the nearest triangle on the other surface was calculated using a KD-tree.”

   Reviewer #2 (Significance (Required)):

   The aim of the paper is well motivated. Cryo-ET is a growth field and there is a need for quantitative parameterization of cryo-ET data. Recently a toolkit for the analysis of filaments from cryo-ET has been published (Dimchev et al. 2021 DOI: 10.1016/j.jsb.2021.107808). Given the specific nature of the implementation, i.e. the membrane structures of mitochondria, I cannot easily see that this implementation will be useful beyond the analysis of mitochondrial membrane structure.

   We hope that we have addressed this concern with generality has been addressed by our previously described updates to the software implementation.

   4. Description of analyses that authors prefer not to carry out.

   Review 2, minor comments:

   Angle between OMM and cristae: Maybe use the average angle of each cristae for comparison or fit a plane for each cristae because you are interested in the angle between the cristae and the OMM and the membrane of the cristae has a lot of uneven surfaces

   We believe that the advantage of our approach is the ability to incorporate more complex geometric information from uneven surfaces such as those seen in cristae. With that said, the ability to quantify metrics for individual cristae in an automated manner would be very appealing, since in many ways cristae are functionally independent compartments. Accomplishing this would require either subdividing the larger surface into individual cristae, which will require development of additional sub-graph processing strategies. Additionally, pairing surfaces to represent opposite sides of a crista will require additional development. While we agree that this will be an excellent extension of the surface morphometrics approach, we feel that the additional development required is out of the scope of this initial manuscript focused on the general workflow. New methods leveraging sub-graph analysis will be explored in future manuscripts.

   Read the original source
3. 

   Review Commons

   Apr 25, 2022

   Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

   Learn more at Review Commons

   ---

   Referee #2

   Evidence, reproducibility and clarity

   Barad, Medina et al. presents a new toolkit for the analysis of membrane ultrastructure in cryo-tomograms. More specifically, the toolkit is designed to compare curvature, angles and spacing between different membrane types in mitochondria. These analyses allow for the quantitative comparison of membrane features e.g. for different growth conditions. To demonstrate the utility of the toolkit tomogram datasets of mitochondria in the presence and absence of ER stress were analyzed. The authors conclude that ER stress affects mitochondria morphology through remodeling of the membrane structure. The presented biological results and …

   Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

   Learn more at Review Commons

   ---

   Referee #2

   Evidence, reproducibility and clarity

   Barad, Medina et al. presents a new toolkit for the analysis of membrane ultrastructure in cryo-tomograms. More specifically, the toolkit is designed to compare curvature, angles and spacing between different membrane types in mitochondria. These analyses allow for the quantitative comparison of membrane features e.g. for different growth conditions. To demonstrate the utility of the toolkit tomogram datasets of mitochondria in the presence and absence of ER stress were analyzed. The authors conclude that ER stress affects mitochondria morphology through remodeling of the membrane structure. The presented biological results and statistics are convincing and show active mitochondrial membrane remodeling in the cell when exposed to ER stress. It is also clear that there is a need for more quantitative evaluation based on the wealth of tomographic image features and mitochondrial membranes are certainly a well-chosen application. For this purpose, the authors developed a new workflow even though most of the discussed analyses are very specific to mitochondrial structures. Therefore, broader applications of these tools to other organelles are not easily envisaged without significant adaption. In that context, the title and abstract overpromise a much more powerful utility that can be applied to any other membrane analysis. Rather it seems that the proposed workflow is more of a specific tool or a pipeline for mitochondrial inner and outer membrane analysis instead of a toolkit for general morphological analysis. Hence, the manuscript cannot be accepted in its current form. In particular, the structure needs a significant rework of editing to become more comprehensible.

   Major comments:

   1. Title and abstract need to be toned down not to overpromise a very general toolkit. The presented method may be a tool or a collection of scripts - a toolkit can be used to address other types of (membrane) analysis problems. In the end, the analysis builds to a large extent on the previous developments and implementation of PyCurve. Perhaps, the most interesting contribution here is the application of the mesh generation by the Poisson reconstruction method to the segmented membranes, which is, however, well implemented in the used pymeshlab framework. The computation of distances and angles is straightforward.
   2. When reading the manuscript, the reader is left in the open whether this is a method paper or a biological results paper. The title/abstract suggests that this is a method paper and the manuscript is more of a mitochondrial membrane report in ER stress. Therefore, the title/abstract does not reflect the manuscript very well.
   3. The manuscript also requires substantial structural editing. Several references to Figures are not appearing in the text in the order that the Figure panels are built. Excessive cross-referencing of figures also make the manuscript hard to read.
   4. The focussing to a method paper will also require more in-depth descriptions of the methodology in the main text. Although the code is deposited at github, there is no script-based workflow and description presented in the manuscript. Although Figure 1 puts the work into context of tomography, it remains very superficial on the image analysis. What are the input and output formats required for each step to follow the sequence of the workflow and at which steps critical interactive input is needed? What are the hardware requirements (CPU, GPU) or performance characteristics (CPU hours for certain operations)?
   5. Figures 3-7 contain colorful 3D renderings of the measured quantities. In addition, they are filled with histograms of every possible quantitative parameter, which often are not very significant or different between. The authors should focus the main results and the figures to show the most relevant and significant findings and put the remaining panels and results into the supplement.
   6. The exact morphological discrimination between fragmented and elongated mitochondria is not easily understood from the results section. What is really meant by blinded manual classification? It only became clear when reading the methods. The results section should stand on its own. How is the overall population between fragmented and elongated cells is affected after Tg application?
   7. Similarly, what is meant by manual classification of IMM, OMM and ER? Is there any clustering involved?
   8. One of the key steps is the generation of a smooth surface from a segmented membrane, there is a question whether true membrane disruptions will be smoothed and may be overlooked in this approach. When these disruptions present true membrane ruptures, they may be of particular biological importance. The authors should support the choice and selection of the smoothing parameters in order to illustrate this potential pitfall.
   9. Throughout the manuscript, the authors mention statistical significance several times and one of the main aims of the study is perform statistical hypothesis testing. It is important to specify the significance test (not only in the methods) and the p-value in order to support this claim. In the manuscript, the authors use exclusively the Mann-Whitney test. What is the rationale for choosing this test? Have the authors considered comparing the total distributions and not just the peaks with e.g. a Kolmogorov-Smirnov test? For a statistical methods paper, there are also no discussion on error analysis.

   Minor comments:

   1. https://github.com/grotjahnlab/surface_morphometrics should include an example data set or tutorial for dissemination.
   2. What is meant by growth plane? This term is not defined in the manuscript.
   3. What is meant by vehicle treatment? There is no explanation in the main text of the manuscript.
   4. Angle between OMM and cristae: Maybe use the average angle of each cristae for comparison or fit a plane for each cristae because you are interested in the angle between the cristae and the OMM and the membrane of the cristae has a lot of uneven surfaces
   5. Have the authors noticed/calculated any differences in the width of the cristae?
   6. Methods: Automated surface reconstruction: "In cases where the resulting surface was very complex, the surface was simplified..." How was the complexity determined?
   7. Methods: Calculation of distances between individual surfaces: "For surfaces with small numbers of triangles, this was accomplished using a distance matrix...". What is the threshold for a small number of triangles?

   Significance

   The aim of the paper is well motivated. Cryo-ET is a growth field and there is a need for quantitative parameterization of cryo-ET data. Recently a toolkit for the analysis of filaments from cryo-ET has been published (Dimchev et al. 2021 DOI: 10.1016/j.jsb.2021.107808). Given the specific nature of the implementation, i.e. the membrane structures of mitochondria, I cannot easily see that this implementation will be useful beyond the analysis of mitochondrial membrane structure.

   Read the original source
4. 

   Review Commons

   Apr 25, 2022

   Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

   Learn more at Review Commons

   ---

   Referee #1

   Evidence, reproducibility and clarity

   Summary:

   Barad and Medina, along with their co-authors, report on the development of a new software toolkit to quantitatively assess membrane structures that are observed in cryo-ET. This new toolkit builds upon existing methodologies by successfully incorporating additional methods and applying this to cryo-ET data to allow for more automated and reliable segmentations. This work addresses a long-standing difficulty in generating membrane segmentations, which are either done manually with huge labor investments or with automated methods that are known to be error prone. The authors demonstrate that their toolkit can generate high …

   Note: This preprint has been reviewed by subject experts for Review Commons. Content has not been altered except for formatting.

   Learn more at Review Commons

   ---

   Referee #1

   Evidence, reproducibility and clarity

   Summary:

   Barad and Medina, along with their co-authors, report on the development of a new software toolkit to quantitatively assess membrane structures that are observed in cryo-ET. This new toolkit builds upon existing methodologies by successfully incorporating additional methods and applying this to cryo-ET data to allow for more automated and reliable segmentations. This work addresses a long-standing difficulty in generating membrane segmentations, which are either done manually with huge labor investments or with automated methods that are known to be error prone. The authors demonstrate that their toolkit can generate high quality segmentations across multiple tomograms with limited manual intervention. They use correlative light and electron microscopy in combination with these segmentations to gain insight into the ultrastructural morphology of mitochondria within embryonic fibroblasts, both under control conditions and under endoplasmic reticulum stress induced by treatment with the drug Thapsigarin. Unlike changes to the ER which are more dramatic under stressed conditions, the changes to the mitochondria are more subtle and impossible to quantify without high quality segmentations. The authors show that inner and outer membrane distances change under stress, and that the distances between cristae, their junctions, and the angle of the cristae with respect to the margin of the mitochondria change. While they characterize the curvedness under the same set of conditions, they report no significant differences.

   Major comments:

   • A major concern is that the data are reported and analyzed on a per tomogram basis when many tomograms contain multiple mitochondria. Given that the mitochondria appear mostly well separated in Sup. Fig 1 with only a few connections visible, and the high degree of pleomorphism noted by the authors, I would strongly suggest that the authors use each mitochondrion as the basis for reporting their metrics rather than the FOV/tomogram as this would avoid mixing metrics from different mitochondria that may be in different states (e.g., fusion/fission). This would apply to data shown in Figures 3, 4, 5, and 6.
   • In Figure 3C the authors show the combined distribution of OMM-IMM distances within each condition. This may obscure some variability within populations. Individual histograms for all mitochondria should be included as supplementary material. Currently, it is difficult to judge if the peak of the combined distribution is appropriate and impossible to judge the variability between tomograms (preferably mitochondria, see above comment). Additionally, the shape of the distributions appears significantly different between conditions, suggesting that selecting a single peak value as representative and the basis for the statistical tests (Fig 3D) might not be appropriate. Please comment.
   • In Figure 4C-F, again combined distributions are shown. Authors should include individual histograms for all mitochondria as supplementary material. The diversity of distributions in the metrics are more pronounced than the distances in reported in Fig 3, again making assessment of variability difficult and raising doubt about using the single peak value.
   • It would be helpful to include the curvature or curvedness of the OMM for each mitochondrion in the supplementary material. The data to correlate OMM curvature with elongated/fragmented mitochondria should be available and might be of interest to some readers.
   • As the work reported here is heavily computational, additional details about the computer hardware used and the time it took for the calculations to complete would be helpful for readers considering applying the code to their own data.
   • Discussion should be expanded to include a comparison of semi-automated segmentations generated here versus manual results from Navarro (Ref 35) & Burt (Ref 54 / doi: 10.1371/journal.pbio.3001319) and how one might estimate the error.

   Minor comments:

   • In the fourth sentence of the third paragraph of the introduction, Hoppe 1992 is cited as evidence of the limitations of work published in 2020, which is confusing. Perhaps the sentence can be re-phrased?
   • Pink and purple very close, consider alternative pair of colors or different shades to distinguish OMM and IMM
   • For all data, exact n per condition should be given (in text and captions as appropriate), not a range for the whole set.
   • Orientation of scaleboxes/scalebars should be consistent per figure panel. If knowledge of the axes is important to the reader, these should be included as well.
   • In the last sentence of the introduction, the term "organellar architectures" is used, instead of the previously defined "membrane ultrastructure." Consider changing for clarity.
   • Inconsistent use of the phrase "cryo-electron tomography" after defining and using "cryo-ET"
   • Authors argue that the distinction between curvedness and curvature is important and that curvature is less appropriate in this context, but then use curvature in the abstract, throughout introduction and in the results section. Usage can be improved for readability.
   • In section "Development of a framework to automate quantification of ultrastructural features of cellular membranes" the second last sentence should read "... higher quality membrane surfaces as compared..."
   • In section "IMM curvedness is differentially sensitive to Tg treatment in elongated and fragmented mitochondrial networks" the fourth sentence should perhaps read "... despite apparent visual differences, no significant..."
   • The term "cell's growth plane" is not clear from the text nor from Fig 6A. Do the authors mean surface of the substrate the cell is growing on?
   • In Materials and Methods:
     • The authors report that manual back-blotting was used in a Vitrobot. This is non-standard usage and more details should be provided.
     • The description of the Leica microscope is insufficient. The objective lens and camera used should be included.
     • In section "Fluorescence Guided Milling" in the third sentence, the word "based" is repeated, second can be removed. A second Pt coat on top of the GIS would also be unusual, please check writing for accuracy.
     • Symbol for degree (or the word degree) should be added to angular increment and tilt range for clarity.
     • Capitalization of TomoSegMemTV is inconsistent.
   • Fig 1B: showing computational steps twice does not provide additional information. Consider just one example. Also, labels for elongated and fragmented would be more useful than the duplicated labels for each computational step.
   • Fig 2A caption - should report actual thickness range measured (as given in Materials and Methods section) instead of estimated range.
   • Fig 3 title - consider replacing "Inter-mitochondrial membrane..." with "Intra-mitochondrial membrane..." for clarity.
   • Fig 3C caption - should explicitly state it is a combined histogram and that the dashed lines correspond to the peak of the pooled data.
   • Fig 5E middle, legend obscures some of the data.
   • Fig 6B and 6C caption - upper and lower parts not explicitly described.

   Significance

   This work primarily describes a technical advancement in methods to analyze cryo-ET data. The novelty arises from the combination of methods and their application rather than completely new ideas or approaches. Demonstrations of the utility of this toolkit based on the authors' analyses are convincing and will likely help a number of researchers in the field who are engaged in explorations of cellular ultrastructure and organelle responses to stimuli. Importantly, this work will help the field move past qualitative descriptions, historically accepted only because quantitative measurements at this level have not been feasible. Overall, in this reviewer's opinion, while the biological findings are modest, the utility of the toolkit for the field is indisputable and the work is of sufficient quality for publication.

   My expertise is in cryo-EM, both single particle analysis and tomography, as well as CLEM workflows, applied mostly to cytoskeletal research and some ER stress. I do not have or strong background in mitochondrial biology nor sufficient computer science expertise to evaluate the numerical methods employed, but based on inspection of the github contents, the screened Poisson reconstruction algorithm is not reimplemented here.

   Read the original source
5. 

   Version published to 10.1101/2022.01.23.477440v3 on bioRxiv

   Feb 3, 2022
6. 

   Version published to 10.1101/2022.01.23.477440v2 on bioRxiv

   Jan 25, 2022
7. 

   Version published to 10.1101/2022.01.23.477440v1 on bioRxiv

   Jan 24, 2022