A nationwide deep learning pipeline to predict stroke and COVID-19 death in atrial fibrillation
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
Deep learning (DL) and machine learning (ML) models trained on long-term patient trajectories held as medical codes in electronic health records (EHR) have the potential to improve disease prediction. Anticoagulant prescribing decisions in atrial fibrillation (AF) offer a use case where the benchmark stroke risk prediction tool (CHA 2 DS 2 -VASc) could be meaningfully improved by including more information from a patient’s medical history. In this study, we design and build the first DL and ML pipeline that uses the routinely updated, linked EHR data for 56 million people in England accessed via NHS Digital to predict first ischaemic stroke in people with AF, and as a secondary outcome, COVID-19 death. Our pipeline improves first stroke prediction in AF by 17% compared to CHA 2 DS 2 -VASc (0.61 (0.57-0.65) vs 0.52 (0.52-0.52) area under the receiver operating characteristics curves, 95% confidence interval) and provides a generalisable, opensource framework that other researchers and developers can build on.
Article activity feed
-
SciScore for 10.1101/2021.12.20.21268113: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
Software and Algorithms Sentences Resources The logistic regression and random forest models were built using the Python sklearn package (0.24.2) and fit with their default configurations (refer to sklearn documentation - https://scikit-learn.org/stable/modules/classes.html), with the exception of max iterations being set to 3000 for the logistic regression model. Pythonsuggested: (IPython, RRID:SCR_001658)Both DL models were built with the PyTorch package (1.9.0). PyTorchsuggested: (PyTorch, RRID:SCR_018536)Results from OddPub: Thank you for sharing your code.
Results from LimitationRecognizer: We detected the following sentences …SciScore for 10.1101/2021.12.20.21268113: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
Software and Algorithms Sentences Resources The logistic regression and random forest models were built using the Python sklearn package (0.24.2) and fit with their default configurations (refer to sklearn documentation - https://scikit-learn.org/stable/modules/classes.html), with the exception of max iterations being set to 3000 for the logistic regression model. Pythonsuggested: (IPython, RRID:SCR_001658)Both DL models were built with the PyTorch package (1.9.0). PyTorchsuggested: (PyTorch, RRID:SCR_018536)Results from OddPub: Thank you for sharing your code.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:There are also several key limitations which prevented us from maximising the potential performance from DL and ML architectures. Firstly, graphical processing units (GPUs) and some parallel computing methods are currently restricted on the NHS Digital’s TRE for England meaning that it was not possible to train models on larger datasets (e.g. 10,000+) or create DL architectures with more layers. This also prevented us from including an individual’s full medical history (e.g. no repeating medical codes) and only allowed us to include the 100 most recent medical codes up to the target inclusion event. Secondly, the NHS Digital TRE for England does not yet facilitate the use or creation of code embeddings pre-trained with other models. This transfer learning approach was adopted by the teams behind BEHRT4 and MedBERT5 and builds on the performance gains demonstrated by large language models such as BERT2 and GPT-329. Lastly, medical codes stored in structured EHR data are just one type of data modality and do not reflect the full diversity of an individual’s medical history. Even before adding new types of data to the TRE such as genetics, imaging and free text, there are observational values such as systolic blood pressure and cholesterol / HDL ratio which could be included in future models. In addition to addressing the above, the next phase of our work will aim to improve the clinical interpretability of our DL and ML pipeline. For this study, we chose to compare model perfor...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
Results from scite Reference Check: We found no unreliable references.
-
