Domain shifts decrease the accuracy of machine learning algorithms to detect myocardial ischemia in ECG recordings: A multi-database analysis

Machine learning algorithms have shown excellent accuracy in detecting myocardial ischemia from electrocardiograms, but their clinical reliability remains uncertain. In this study, we explored how domain shifts between different datasets affect the performance of these algorithms. We used six publicly available 12-lead ECG databases, containing a total of 55,953 recordings, and applied several machine learning approaches, including generalized linear models, random forests, gradient boosting, deep neural networks, and ensemble learning. The models were evaluated under different conditions—within individual datasets, across all datasets combined, and using leave-one-dataset-out validation.When trained and tested on the same dataset, the models performed very well, with area under the curve values exceeding 0.90. However, performance dropped notably when the models were tested on data from different sources. Sensitivity decreased substantially, while specificity remained relatively stable. Further analysis showed that variations in ECG patterns across datasets contributed to these differences in performance.Overall, our results demonstrate that while machine learning algorithms can detect myocardial ischemia accurately within familiar data, their ability to generalize across diverse datasets is limited. Addressing dataset heterogeneity and improving model robustness will be essential before such systems can be reliably implemented in clinical practice.