Internal and External Validation of an Ensemble Learning Model Integrating Zygote Morphokinetics with Conventional Embryo Assessment for Blastocyst Prediction
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
To perform internal and external validation of a gradient-boosted decision tree (GBDT) fusion model that integrates zygote morphokinetic parameters with conventional embryo assessment features for blastocyst prediction, and to compare its discriminative performance against senior embryologists.
Study design
TRIPOD Type 2a/2b validation study. The GBDT fusion model, details of which are described in a companion paper, was developed using 84 zygote-stage morphokinetic parameters and eight conventional embryo assessment features. Internal validation was performed on 631 two-pronuclei zygotes from 218 treatment cycles at a university-affiliated reproductive center. External validation used a publicly available dataset (Wang et al., bioRxiv 2023) comprising over 500 embryos with time-lapse imaging frames and blastocyst outcome labels. Model performance was assessed using a seven-metric evaluation framework: area under the receiver operating characteristic curve (AUC), F1 score, area under the precision-recall curve (AUPRC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Three senior embryologists provided independent embryo assessments using standard morphological criteria, with majority-vote consensus serving as the comparator. Discriminative performance was compared using the DeLong test; agreement was quantified with Cohen’s kappa. Decision curve analysis (DCA) evaluated clinical net benefit, and calibration was assessed using the Brier score, calibration slope, and Hosmer–Lemeshow goodness-of-fit test. Generalized estimating equations accounted for within-cycle clustering.
Results
The GBDT fusion model achieved an AUC of 0.78 (95% CI 0.74–0.82), AUPRC of 0.72, F1 score of 0.73, sensitivity of 0.74, specificity of 0.77, PPV of 0.72, and NPV of 0.79 on internal validation. External validation on the Wang et al. dataset yielded an AUC of 0.76 (95% CI 0.71–0.81), representing an acceptable performance decrement (ΔAUC = 0.02). The model significantly outperformed embryologist consensus assessment (embryologist AUC 0.70; DeLong P < 0.001), with moderate agreement between model and embryologists ( κ = 0.56). DCA demonstrated net clinical benefit across threshold probabilities of 0.15–0.55. Calibration was acceptable (Brier score 0.164; calibration slope 0.94; Hosmer–Lemeshow P = 0.32).
Conclusions
The GBDT fusion model demonstrates robust internal and external discriminative performance for blastocyst prediction, exceeding the accuracy of senior embryologists. These findings support its potential utility as an objective decision-support tool for early embryo triage approximately 30 hours post-insemination.
