Penumbria: Advanced 3D cell segmentation for biomedical imaging
Curation statements for this article:-
Curated by eLife
eLife Assessment
This useful study presents Penumbria, a method for segmenting cells in three-dimensional microscopy images, released with publicly available code and documentation. The implementation is technically sound, and the reported performance gains would be a welcome contribution; the advance is incremental, since regression on distances to boundaries is already widespread in the field. The evidence is incomplete: the main text does not state which setting it used for the main competing method, and the setting used is the one that handles data of this kind poorly, so the central claim of better accuracy is unsettled; current leading methods are missing from the comparison; and five architectural and training choices are combined without attributing the gains to any of them. Every evaluation dataset labels nuclei, so the work should present itself as nuclei segmentation; describing and completing the comparisons, and attributing the gains, would raise the strength of evidence.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
Quantitative analysis of three-dimensional cellular architecture is fundamental to understanding tissue organization, disease progression, and drug response. There are effective approaches for 2D segmentation, yet 3D cell segmentation remains a critical bottleneck due to diverse cell morphologies, low signal-to-noise ratios, and data scarcity. We introduce Penumbria, a general-purpose 3D cell segmentation framework that achieves state-of-the-art accuracy across morphologically distinct cell populations and imaging conditions in volumetric microscopy. Penumbria formulates segmentation as a regression problem on distances to cell boundaries, supporting instance reconstruction without shape priors and permitting end-to-end GPU inference. A U-Net-based architecture with xLSTM bottleneck blocks and patch embeddings enables multi-scale feature extraction, long-range modeling of spatial context, and convolutional feature-volume tokenization. The model is extended with two modules: a Global Zernike Phase Layer, which learns Zernike-parameterized phase corrections in the frequency domain to deal with optical aberrations such as defocus and tilt, and a Scaled Geocaps Layer, which samples features at fixed grid locations across multiple spatial scales, routing evidence between them such that a detection is only confident where concordance holds across scales simultaneously. Across four diverse 3D datasets selected to probe the limits of existing methods, Penumbria outperforms Cellpose-SAM on all four and StarDist-3D on three, with comparable accuracy on the fourth, achieving up to a 38% improvement in mean average precision over the second-best method. Penumbria’s strong boundary accuracy further supports downstream analyses such as quantifying membrane dynamics or protein localization. We also release a hand-labeled confocal dataset of 1,543 zebrafish neurons across seven volumes, on which Penumbria can be easily and quickly trained.
Article activity feed
-
eLife Assessment
This useful study presents Penumbria, a method for segmenting cells in three-dimensional microscopy images, released with publicly available code and documentation. The implementation is technically sound, and the reported performance gains would be a welcome contribution; the advance is incremental, since regression on distances to boundaries is already widespread in the field. The evidence is incomplete: the main text does not state which setting it used for the main competing method, and the setting used is the one that handles data of this kind poorly, so the central claim of better accuracy is unsettled; current leading methods are missing from the comparison; and five architectural and training choices are combined without attributing the gains to any of them. Every evaluation dataset labels nuclei, so the work …
eLife Assessment
This useful study presents Penumbria, a method for segmenting cells in three-dimensional microscopy images, released with publicly available code and documentation. The implementation is technically sound, and the reported performance gains would be a welcome contribution; the advance is incremental, since regression on distances to boundaries is already widespread in the field. The evidence is incomplete: the main text does not state which setting it used for the main competing method, and the setting used is the one that handles data of this kind poorly, so the central claim of better accuracy is unsettled; current leading methods are missing from the comparison; and five architectural and training choices are combined without attributing the gains to any of them. Every evaluation dataset labels nuclei, so the work should present itself as nuclei segmentation; describing and completing the comparisons, and attributing the gains, would raise the strength of evidence.
-
Reviewer #1 (Public review):
Summary:
The manuscript proposes an architecture and distance-regression-based logic for 3D cell segmentation.
The architecture combines a U-Net backbone with an xLSTM bottleneck layer and two novel components: a Zernike Phase Layer that is used for input normalization and the Scaled Grid Geocaps that link the input and bottleneck, inspired by the Capsule mechanism.
During training, the network predicts foreground vs. background and regresses a distance transform as output. During inference, these predictions are used as input to a watershed procedure to segment cells in 3D.
The method is evaluated by training on four different 3D cell/nucleus segmentation tasks and compared to StarDist3D and CellPoseSAM baselines, which are also trained on the respective datasets using the same train / val / test splits.
The…
Reviewer #1 (Public review):
Summary:
The manuscript proposes an architecture and distance-regression-based logic for 3D cell segmentation.
The architecture combines a U-Net backbone with an xLSTM bottleneck layer and two novel components: a Zernike Phase Layer that is used for input normalization and the Scaled Grid Geocaps that link the input and bottleneck, inspired by the Capsule mechanism.
During training, the network predicts foreground vs. background and regresses a distance transform as output. During inference, these predictions are used as input to a watershed procedure to segment cells in 3D.
The method is evaluated by training on four different 3D cell/nucleus segmentation tasks and compared to StarDist3D and CellPoseSAM baselines, which are also trained on the respective datasets using the same train / val / test splits.
The authors report improved segmentation quality compared to the baselines.
Strengths:
The experiments are solid; ablation studies evaluate the main architectural components, and the software appears to be well documented (I did not have time to test it).
Weaknesses:
(1) This is not really a weakness but a general point: the main issue in microscopy segmentation is that a lot of training data is needed if training a network from scratch. This results in a very laborious data annotation process. This is what modern approaches like CellPoseSAM and microSAM address by providing strong pre-trained networks that work well out-of-the-box for many imaging settings and can then be fine-tuned with a few corrected images to achieve state-of-the-art results. Having a better 3D segmentation approach is of course still worthwhile, but does not address the main practical challenge. I would suggest clarifying this in the introduction.
(2) The authors implement a well-known strategy to segmentation in microscopy: regressing the distance transform. They acknowledge some prior work (Scherr et al., Lux and Matula), but this approach is more prevalent than acknowledged, including for 3D cell segmentation. For example, microSAM [1] implements cell instance segmentation by regressing the distance to the cell boundary and centroid, which is applied in 3D by CochleaNet [2]. See also [3] for an overview of different distance- or flow-field-regression-based cell segmentation approaches. I would suggest performing a more thorough literature review and acknowledging more prior work.
(3) The baseline comparisons are not described in any detail, are questionable for CellPoseSAM, and lack current state-of-the-art models:
3.1) The details on training and inference for the baseline methods are missing. This is particularly missing for CellPoseSAM. Here, it would be important to clarify whether the model is initialized randomly or with CellPoseSAM's pretrained weights and which of the two different 3D segmentation modes is used (see next point). It would also make sense to report both CellPoseSAM in zero-shot mode and finetuned for all cases.
3.2) CellPoseSAM supports two different 3D segmentation modes: either a "2.5D" segmentation mode, where the network is applied across all three axes, the resulting flow maps are merged, and then used for segmentation. And an "overlap-based stitching mode", where segmentation is applied only in the XY plane and slices are then merged based on overlap. It is unclear which of these modes is used. Ideally both should be reported; the "2.5D" mode often leads to worse results for anisotropic data, and we almost always prefer the overlap-based stitching in practice.
3.3) Two important state-of-the-art baselines are missing:
i) microSAM [1], which is widely used for cell segmentation based on a pretrained 2D network that is extended to 3D using a similar strategy as CellPoseSAM's overlap-based stitching mode. It would make sense to report both zero-shot and fine-tuned performance.
ii) uSeg3D [4]: this method implements a generalized 3D stitching approach that works independently of the underlying architecture and reports significant improvements.
Improving and extending the baseline experiments would give a much better picture of the practical relevance of this work.References:
[1] https://www.nature.com/articles/s41592-024-02580-4
[2] https://www.biorxiv.org/content/10.1101/2025.11.16.688700v1.abstract
-
Reviewer #2 (Public review):
Summary:
In their manuscript "Penumbria: Advanced 3D cell segmentation for biomedical imaging", the authors describe their 3D nuclei segmentation pipeline and compare it against state-of-the-art methods for nuclei and cell segmentation.
The introduction is clear, with state-of-the-art methods, for instance, segmentation of cells from microscopy images, and motivates the method with science that can be done with cell segmentation. Related work is appropriately mentioned with the caveat that it conflates cell and nuclei segmentation, and some related work is missing.
The Results section describes the loss, the architecture novelties, and evaluation results on nuclei segmentation, not cell segmentation. Here it becomes clear that Penumbria addresses 3D nuclei segmentation, i.e., instance segmentation of …
Reviewer #2 (Public review):
Summary:
In their manuscript "Penumbria: Advanced 3D cell segmentation for biomedical imaging", the authors describe their 3D nuclei segmentation pipeline and compare it against state-of-the-art methods for nuclei and cell segmentation.
The introduction is clear, with state-of-the-art methods, for instance, segmentation of cells from microscopy images, and motivates the method with science that can be done with cell segmentation. Related work is appropriately mentioned with the caveat that it conflates cell and nuclei segmentation, and some related work is missing.
The Results section describes the loss, the architecture novelties, and evaluation results on nuclei segmentation, not cell segmentation. Here it becomes clear that Penumbria addresses 3D nuclei segmentation, i.e., instance segmentation of medium-sized, mostly convex-ish and well-separated objects in single-channel 3D volumes.
Penumbria combines several choices and innovations:
- loss: boundary distance regression
- architecture
- 3D U-VixLSTM
- Zernike Layer
- SGG
- combine multiple scales in SGG
- sampling strategy to look for bright and dark objects.The authors show on four nuclei segmentation benchmarks from confocal images that Penumbria outperforms CellPose-SAM (in either zero-shot or fine-tuned mode) and Stardist-3D in most cases, except for some where Stardist-3D performs better. They report results of an IoU threshold sweep to measure instance segmentation performance.
Penumbria is available on GitHub under a non-commercial license. I have not tested it yet, and therefore assume that the code is in full working condition and reproduces the reported results.
Strengths:
(1) The method may outperform state-of-the-art methods for cell segmentation as shown in three of four 3D benchmarks.
(2) The code is available.
Weaknesses:
Major
(1) Similar to the established confusion across the literature, all the authors' evaluation datasets do not label cells, but cellular nuclei. This is important to understand because the methods they compare against, CellPose-SAM and Stardist-3D, demonstrate their performance on a variety of datasets, some of which label cellular membranes instead of nuclei. The cell segmentation task is often considered harder than the nuclei segmentation task, because objects consistently touch each other, the separation may have noise-induced holes, and object shapes may be more diverse than for nuclei.
(2) The addition of U-VixLSTM, SGG, multi-scale input, Zernike layers, and distance transform regression sounds great, but a comparison to architectures or conventional losses used to train any combination is missing. Five innovations/choices combined simultaneously without an attempt to attribute.
(3) There is no clear delineation of 2D and 3D approaches. E.g. CellPose-SAM is not 3D, but sequential 2D execution with optional averaging of doing this along all axes.
(4) For the 3-fold cross-validated evaluation: It sounds like they were all trained for each benchmark independently? Did Stardist-3D and CellPose SAM start from their existing 'general' weights or from scratch (not possible with SAM, I know). Not described at all.
(5) Experiments to evaluate generalization performance are missing (train on all datasets, compare across datasets, ...).
(6) The Zernike layer optimizes only coefficients {3,4,12}; the evaluation could be described as rudimentary. Table 3 is missing cross-validation. Since mouse_skull is (a lot) different and differences for the others are minor, I would be curious how significant these results are. The following text is unclear: "This reflects how we developed these modifications in turn." How so?
(7) Training and inference: input and output block size missing, GPU memory requirements undefined, discuss block size vs. batch size. 50% overlap block-inference with feathering not compared to other overlap ratios, just "works best". Test-time augmentation 8x rotation (what are those rotations? Why 8? How much does it contribute? Compare the cost for 8xs test time augmentation against skipping "background" heuristic.)
Minor
(1) What is the incoming contrast range and bit depth?
(2) Stardist-3D's weakness is not that it requires convexity (it requires star-convexity) but that it picks one hypothesis leading to low-resolution outlines. This has been overcome by Starfinity (https://linkinghub.elsevier.com/retrieve/pii/S0092867421013398; https://github.com/multiFISH/EASI-FISH).
(3) Heatmap regression is not the same as distance regression; the manuscript does not explain what it is and uses terms interchangeably. This should be clarified. The reference for heatmap is 14; for distances, there are others, e.g. doi:10.1007/978-3-030-00934-2_36.
(4) Only one evaluation example compares against CellPose-SAM zero-shot; I would add this to all of them. The mentioned place switch would be interesting to see in a plot instead of text.
(5) "We replicated this failure on multiple machines." How does the machine matter here? Did the authors do cross-validation across different computers, instead of different random seeds? This sounds expensive!
(6) Brightness-based sampling could be achieved by augmentation?
(7) In the text "We use the following bicubic upsampling factors[...]", I think the authors mean tricubic, or cubic (because they upscale only one axis). Bicubic makes no sense, but people say this in the literature; we can do better. Why integer upscaling factors instead of exact ones? Are the authors concerned about that one incomplete border pixel?
(8) "Furthermore, it is not confined to Linux-based environments such as Segmamba [32], an important consideration given that biologists often work with Windows machines."
This is confusing. Is this a comment about somebody's publicly available repository? This is not relevant to evaluate the method and should be discussed separately. Does it work in WSL?(9) Downsampling by factors >2 with bicubic interpolation sounds wrong; should the authors use averaging?
-
Author response:
We thank the reviewers for their thoughtful comments. We will address the substantive concerns by broadening our evaluation to additional 3D datasets, strengthening comparisons against competitive baselines, and systematically ablating Penumbria's architectural components. We are confident that these experiments will establish Penumbria as a state-of-the-art 3D instance-segmentation method across a substantially broader range of segmentation challenges in biological imaging, without the pronounced dataset-specific failure modes observed for several competing approaches.
Scope and dataset difficulty.
We will expand our benchmark to include additional 3D datasets spanning nuclei, whole cells, and organelles. However, we do not accept two assertions made in the review: first, that nuclear segmentation is inherently easier …
Author response:
We thank the reviewers for their thoughtful comments. We will address the substantive concerns by broadening our evaluation to additional 3D datasets, strengthening comparisons against competitive baselines, and systematically ablating Penumbria's architectural components. We are confident that these experiments will establish Penumbria as a state-of-the-art 3D instance-segmentation method across a substantially broader range of segmentation challenges in biological imaging, without the pronounced dataset-specific failure modes observed for several competing approaches.
Scope and dataset difficulty.
We will expand our benchmark to include additional 3D datasets spanning nuclei, whole cells, and organelles. However, we do not accept two assertions made in the review: first, that nuclear segmentation is inherently easier than whole-cell segmentation; and second, that our existing datasets contain “mostly convex-ish and well-separated” objects (which we address separately below). Parhyale illustrates why the first assertion is unwarranted: despite being a nuclear-label dataset, its nuclei are extremely densely packed, with boundaries frequently so poorly distinguishable that even manual identification of individual instances is challenging. Historically, StarDist-3D has been the clear state-of-the-art method on this dataset, with little effective competition, whereas Penumbria matches its performance without imposing a star-convex shape prior. Conversely, the PlantSeg study reported an Adapted Rand error of only 0.075 on 3D cell-wall-labelled leaf tissue and approximately 90–99% correctly segmented whole cells across six leaf-surface specimens [1].
Segmentation difficulty is an empirical property, not a consequence of what is being segmented or how it is stained.
Diversity in biological targets or staining modalities does not necessarily translate into diversity of segmentation challenges, let alone greater difficulty. Constructing a meaningful benchmark therefore requires expert judgment in selecting datasets spanning challenging conditions, including irregular morphology, crowding, poor contrast, anisotropy, and imaging artifacts, informed by previous benchmarking results. The same empirical standard must apply to the reviewers' characterization of our datasets as “mostly convex-ish and well-separated.” Neither characterization is supported by quantitative evidence or relevant citations, and even if both were established, they would not by themselves demonstrate that the datasets are easy to segment. We will therefore quantitatively characterize the datasets, including their morphology and crowding, and examine how these properties relate to segmentation performance across methods.
Zero-shot generalization.
We will evaluate Cellpose-SAM zero-shot across all datasets, using segmentation accuracy (SA) at IoU 0.5 below 0.30 as a deliberately permissive indicator of severe instance-segmentation failure. Even at this threshold, substantial merging or missed instances can make the output unsuitable for reliable instance-level quantitative analysis without extensive correction.
Cell pose-SAM comparison and baselines.
We appreciate that the reviewers have since acknowledged the baseline training configurations reported in our Supplementary Material, which specified that Cellpose-SAM was fine-tuned from pretrained weights, while StarDist-3D was trained from scratch. In retrospect, these details would have benefited from greater visibility in the main text, and we will move them there in the revision. We also identified an important omission in our Cellpose-SAM inference configuration. The Cellpose-SAM paper specifies 1,000 dynamics iterations, flow3D_smooth=2, and removal of masks below 1,000 pixels for its 3D experiments. These settings differ substantially from generic API defaults and materially improve performance on several of our datasets. We will therefore repeat the evaluation using the published 3D settings and transparently report both the original and corrected configurations, with the latter serving as the principal comparison where applicable.
u-Segment3D.
The reviewers recommend u-Segment3D because it “reports large improvements,” without specifying over which methods. Its comparisons with Cellpose involve earlier cyto models, not Cellpose-SAM, which was introduced nearly a year after u-Segment3D's original preprint. Moreover, u-Segment3D is a 2D-to-3D aggregation framework rather than an independent segmentation model. Its supplementary material reports dataset-specific gradient smoothing and a 200-voxel size filter, without documenting equivalent settings for its Cellpose 3D baseline. As our own experiments demonstrate, such settings can materially affect 3D accuracy. The reported improvements therefore do not establish competitiveness with a properly configured and fine-tuned Cellpose-SAM. We will instead prioritize StarDist-3D, micro-SAM, and a corrected Cellpose-SAM comparison, reporting all inference configurations transparently.
Architectural testing.
While distance-transform regression is an established technique, as acknowledged in our manuscript, Penumbria's principal methodological contributions lie in its architecture and the integration of its components. We will expand our ablations by removing key architectural elements individually and measuring the resulting changes in performance. These experiments will quantify the contribution of each element within the complete pipeline. We look forward to submitting the revised manuscript incorporating these improvements.
(1) Wolny, A., Cerrone, L., Vijayan, A., Tofanelli, R., Barro, A. V., Louveaux, M., et al. (2020). Accurate and versatile 3D segmentation of plant tissues at cellular resolution. eLife, 9, e57613.
-
-
-
Penumbria: Advanced 3D cell segmentation for biomedical imaging [new] ...formulates segmentation as a regression on boundary distances, leveraging U-Net, xLSTM, Zernike phase correction, and multi-scale Geocaps.
-
Penumbria: Advanced 3D cell segmentation for biomedical imaging https://www.biorxiv.org/content/10.64898/2026.06.30.735527v1
-
Penumbria: Advanced 3D cell segmentation for biomedical imaging https://www.biorxiv.org/content/10.64898/2026.06.30.735527v1
-
