Open RGB Imaging Workflow for Morphological and Morphometric Analysis of Fruits using AI: A Case Study on Almonds

  1. Mas-Gómez Jorge
  2. Rubio Manuel
  3. Dicenta Federico
  4. Martínez-García Pedro José

Reviewed by

Read the full article See related articles

Discuss this preprint

Start a discussion What are Sciety discussions?

Listed in

Log in to save this article

Abstract

High-throughput phenotyping is addressing the current bottleneck in phenotyping within breeding programs. Imaging tools are becoming the primary resource for improving the efficiency of phenotyping processes and providing large datasets for genomic selection approaches. The advent of AI brings new advantages by enhancing phenotyping methods using imaging, making them more accessible to breeding programs. In this context, we have developed an open Python workflow for analyzing morphology and heritable morphometric traits using AI, which can be applied to fruits and other plant organs. This workflow has been implemented in almond (Prunus dulcis ), a species where efficiency is critical due to its long breeding cycle. Over 25,000 kernels, more than 20,000 nuts, and over 600 individuals have been phenotyped, making this the largest morphological study conducted in almond. As result, new heritable morphometric traits of interest have been identified. These findings pave the way for more efficient breeding strategies, ultimately facilitating the development of improved cultivars with desirable traits.

Article activity feed

  1. GigaScience

    AbstractHigh-throughput phenotyping is addressing the current bottleneck in phenotyping within breeding programs. Imaging tools are becoming the primary resource for improving the efficiency of phenotyping processes and providing large datasets for genomic selection approaches. The advent of AI brings new advantages by enhancing phenotyping methods using imaging, making them more accessible to breeding programs. In this context, we have developed an open Python workflow for analyzing morphology and heritable morphometric traits using AI, which can be applied to fruits and other plant organs. This workflow has been implemented in almond (Prunus dulcis), a species where efficiency is critical due to its long breeding cycle. Over 25,000 kernels, more than 20,000 nuts, and over 600 individuals have been phenotyped, making this the largest morphological study conducted in almond. As result, new heritable morphometric traits of interest have been identified. These findings pave the way for more efficient breeding strategies, ultimately facilitating the development of improved cultivars with desirable traits.Competing Interest StatementThe authors have declared no competing interest.Footnotes https://github.com/jorgemasgomez/almondcv2 Abbreviations:GPUGraphics Processing UnitYOLOYou Only Look OnceSAMSegment Anything ModelROIRegion of InterestFunder Information DeclaredMinisterio de Ciencia y Universidades, España, PID2021-127421OB-I00, FPU20/00614Fundación Séneca

    This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giaf157), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

    Reviewer 3: Yu Jiang

    The present study entitled "Open RGB Imaging Workflow for Morphological and Morphometric Analysis of Fruits using AI: A Case Study on Almonds" reported the development of a Python-based image analysis pipeline that extract morphological traits of almond nut shells and kernels. A case study was conducted to use the developed pipeline to analyze breeding populations of 665 genotypes and extract both general morphology traits such as height, length, area, aspect ratio, etc. and specialized traits for almond such as width at three heights, vertical and horizontal symmetry, etc. Further, each nut shell or kernel was weighed, so models were established to use the weight and morphological traits to predict the thickness of each nut shell or kernel. In addition to morphological traits, morphometric (or shape) was extracted for each nut shell or kernel. Clustering analysis was performed on the morphometric traits to identify variability among genotypes. To further validate the efficacy of the extracted traits, broad-sense heritability was calculated and used as a criterion.

    The major contribution of this study is the integration of different components (e.g., camera calibration, image segmentation, and morphological/morphometric trait extraction, etc.) as a user-accessible, open-source Python implementation for the plant breeding community, especially for almond breeders. However, there several aspects that could be further improved.

    First, the present study showed the most number of samples that were phenotyped by the proposed pipeline among recent efforts on almond nut shell and/or kernel phenotyping. However, there was no clear evidence to demonstrate direct benefits to ongoing almond breeding. Certain traits (e.g., aspect ratio, tip/top/side curvatures) could be included in a breeding program, but what's the significance of including these traits in breeding programs. Are they crucial to either improve the productivity, quality, or other management practices or processing practices for the almond industry, especially given breeding context?

    Second, the pipeline uses deep learning-based segmentation which is powerful to handle complex background. Based on the limited figures or example images in the GitHub repo, the background is mostly single colored (e.g., white or black) without appearances that may confuse even conventional segmentation, especially if image color is calibrated. Assuming most of the almond nut shell and kernel analyses would be done in a laboratory condition, it is not convincing why conventional segmentation methods may not be preferred if both illumination and camera configuration can be well controlled. Ultimately, the question is whether it is worthy the effort of labeling hundreds of images to fine-tune a deep learning segmentation model compared to a careful hardware-software design to make operation more efficient. Or with the simplified background, vision foundation models such as SAM will be sufficient.

    Third, in the Introduction section, some technical statements should be revised to make them accurate. For example, image segmentation is a core computer vision task rather than relying on computer vision algorithms. One-stage and two-stage strategies are used to differentiate models for object detection not image segmentation. Further, Faster RCNN is an object detection model and cannot do image segmentation. It is highly recommended that the authors could find a computer science or engineering colleague to proofread the technical statements to ensure the accuracy.

    Last, it is appreciated the authors effort on making an open-source software for the community. However, the dataset can be equally important to advance the scientific discovery and technology development. Is there any plan to make the dataset publicly available to help facilitate the development of additional computer vision algorithms for almond phenotyping?

  2. GigaScience

    AbstractHigh-throughput phenotyping is addressing the current bottleneck in phenotyping within breeding programs. Imaging tools are becoming the primary resource for improving the efficiency of phenotyping processes and providing large datasets for genomic selection approaches. The advent of AI brings new advantages by enhancing phenotyping methods using imaging, making them more accessible to breeding programs. In this context, we have developed an open Python workflow for analyzing morphology and heritable morphometric traits using AI, which can be applied to fruits and other plant organs. This workflow has been implemented in almond (Prunus dulcis), a species where efficiency is critical due to its long breeding cycle. Over 25,000 kernels, more than 20,000 nuts, and over 600 individuals have been phenotyped, making this the largest morphological study conducted in almond. As result, new heritable morphometric traits of interest have been identified. These findings pave the way for more efficient breeding strategies, ultimately facilitating the development of improved cultivars with desirable traits.Competing Interest StatementThe authors have declared no competing interest.Footnotes https://github.com/jorgemasgomez/almondcv2 Abbreviations:GPUGraphics Processing UnitYOLOYou Only Look OnceSAMSegment Anything ModelROIRegion of InterestFunder Information DeclaredMinisterio de Ciencia y Universidades, España, PID2021-127421OB-I00, FPU20/00614Fundación Séneca

    This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giaf157), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

    Reviewer 2: Qi Wang

    We would like to thank you for submitting your manuscript to our journal.The manuscript proposes an AI-powered open RGB imaging workflow for morphological and morphometric analysis of fruits or other plant organs, with almonds as a case study. The workflow, developed in Python, covers the full pipeline from image pre-processing, segmentation model development and deployment, to trait measurement and analysis. It aims to improve the efficiency and accuracy of phenotyping in breeding programs by addressing the limitations of traditional methods, such as their time-consuming and labor-intensive nature. However, there are still the following problems in this paper that need further improvement: 1.Table formatting: Some of the tables in the manuscript do not follow the formatting standards of the journal. The authors are encouraged to revise them accordingly to ensure clarity, consistency, and ease of understanding. 2.Formula presentation: Certain mathematical formulas are not clearly formatted and appear disorganized. The authors should re-typeset the equations to improve readability and provide clearer explanations for each formula. 3.Introduction: The introduction could be strengthened by more thoroughly explaining the relationship between phenotypic data and breeding. The authors may also discuss how phenotyping data supports genomic selection and accelerates breeding via high-throughput workflows. 4.Methods section: While the paper clearly explains how morphological traits and kernel thickness are measured, it does not sufficiently explain how this data contributes to breeding decisions. The authors should elaborate on how the extracted traits are applied in practical breeding or selection strategies. 5.Lack of algorithmic novelty: While the integration of existing tools is commendable, the core methods used (e.g., YOLO, SAHI) are based on publicly available models, without introducing new algorithmic components or comparative ablation studies. The authors are advised to clarify the unique contribution of their workflow, especially in terms of engineering integration or practical usability. 6.Limited evaluation metrics: The performance of segmentation models is only reported using error percentage. The inclusion of standard metrics such as IoU, Precision, Recall, and F1-score would allow for a more comprehensive evaluation and comparison across models (see LLRL methods). 7.Figures and captions: Currently, figure images and their descriptions are placed separately, which may reduce readability. It is recommended to place figure captions immediately beneath or alongside the figures to enhance the paper's coherence and user-friendliness. 8.Trait extension suggestions: In order to enhance the expressiveness and resolution of phenotypic trait modeling, authors are advised to refer to the relevant research on extracting fine-grained phenotypic features in plant images in recent years. For example, PlanText proposed a progressive visual guidance strategy to help improve the modeling quality of phenotypic traits in images. Therefore, I would like to give a "Major Revision" recommendation.

  3. GigaScience

    AbstractHigh-throughput phenotyping is addressing the current bottleneck in phenotyping within breeding programs. Imaging tools are becoming the primary resource for improving the efficiency of phenotyping processes and providing large datasets for genomic selection approaches. The advent of AI brings new advantages by enhancing phenotyping methods using imaging, making them more accessible to breeding programs. In this context, we have developed an open Python workflow for analyzing morphology and heritable morphometric traits using AI, which can be applied to fruits and other plant organs. This workflow has been implemented in almond (Prunus dulcis), a species where efficiency is critical due to its long breeding cycle. Over 25,000 kernels, more than 20,000 nuts, and over 600 individuals have been phenotyped, making this the largest morphological study conducted in almond. As result, new heritable morphometric traits of interest have been identified. These findings pave the way for more efficient breeding strategies, ultimately facilitating the development of improved cultivars with desirable traits.Competing Interest StatementThe authors have declared no competing interest.Footnotes https://github.com/jorgemasgomez/almondcv2 Abbreviations:GPUGraphics Processing UnitYOLOYou Only Look OnceSAMSegment Anything ModelROIRegion of InterestFunder Information DeclaredMinisterio de Ciencia y Universidades, España, PID2021-127421OB-I00, FPU20/00614Fundación Séneca

    This work has been peer reviewed in GigaScience (see https://doi.org/10.1093/gigascience/giaf157), which carries out open, named peer-review. These reviews are published under a CC-BY 4.0 license and were as follows:

    Reviewer 1: Yuvraj Chopra

    The methods described in this article represent a useful tool for fast and reliable morphometric analysis of almonds with potential applications in fruits. The pipeline is technically sound, and publicly available workflow will advance the adoption of this technology. However, there are critical concerns which needs to be addressed before the manuscript could be further proceeded for publication in the journal - Major Comments -

    1. Authors claim this technique as a new phenotyping tool with breakthrough implications; however, I object to this claim. Numerous studies have utilized this technique in plant phenotyping to the extent that labeling it as a new phenotyping tool may not be ideal. Additionally, for kernel or seed morphometrics, a wide array of user-friendly, open-source tools have already been developed and are readily available, for example,
    1. Within horticultural crops, workflows and studies (not acknowledged in this article) are available that can be adapted or modified to do the same thing. For example - publications from as early as 2020 used machine learning models to measure size and mass of almonds, however, this relevant study was not acknowledged by the authors https://onlinelibrary.wiley.com/doi/full/10.1111/jfpe.13374 . Discuss how the presented method is better than the aforementioned article, justify the claim 'breakthrough'.
    2. The authors claim successfully testing the pipeline for apples and strawberries. Information for fruit size can be extracted from 2D images; however, the example results show only length, width, circularity, and ellipse ratio. How do these parameters assist fruit breeders? Since it is segmentation based classification using a reference scale, the aforementioned tools particularly SeedExtractor can generate similar results. Does it qualify the tool for integration into fruit crops breeding pipeline? Moreover, fruit breeders require on-tree analysis, recent advancements have enabled 3D sensing for significantly better detection particularly using cost effective RGB-D cameras.

    Minor Comments -

    1. Change title - This study uses deep learning models which are a type of AI. AI is a broader term. Additionally, the potential for utility in fruit breeding pipeline appears to be limited. Suggested - Open RGB imaging workflow for morphological and morphometric analysis of almond kernels using deep learning.
    2. The video tutorial showed using YOLOv8, mention it in the methods. Add information for all settings used in CVAT.
  4. Version published to 10.1101/2025.05.05.652179 on bioRxiv