Update on a Model to Minimize Population Health Loss in Times of Scarce Surgical Capacity During the COVID-19 Crisis and Beyond
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
No abstract available
Article activity feed
-
-
SciScore for 10.1101/2020.07.26.20157040: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Since all models are a simplification of reality, our model has several limitations. First, the survival data used were not always derived from high-quality evidence. Although survival with treatment might be validly estimated from national registries, the survival without treatment is harder to be unbiasedly estimated. The …
SciScore for 10.1101/2020.07.26.20157040: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement not detected. Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code and data.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Since all models are a simplification of reality, our model has several limitations. First, the survival data used were not always derived from high-quality evidence. Although survival with treatment might be validly estimated from national registries, the survival without treatment is harder to be unbiasedly estimated. The surgeries that were evaluated are often part of standard clinical practice. Therefore, data might be biased (e.g. selection bias in the survival without treatment because patients opt for palliative care), or not available (it would be unethical now to perform RCTs evaluating surgery versus no surgery). Instead, best available evidence was used, which in part included evidence from more historical RCTs. As such, data might be biased, and as a result, so are the estimates from our model. Because of this limitation, our approach is simply to aggregate transparently and systematically the best currently available evidence using a model. Second, it was assumed that all surgeries are successful. The current model does not simulate adverse events, like major bleedings or death due to surgery. Also, the potential reduction of QoL due to these adverse events were not incorporated, nor the QoL reduction of a temporary period of recovery after surgery. Because of these assumptions, the overall QALYs associated with the surgery should not be interpreted as an absolute estimate. They are the maximum possible QALYs that can be acquired by performing the surgery. Howeve...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-
SciScore for 10.1101/2020.07.26.20157040: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
Since all models are a simplification of reality, our model has several limitations. First, the survival data used were not always derived from high-quality evidence. Although survival with treatment might be validly estimated from national registries, the survival without treatment is harder to be unbiasedly estimated. The surgeries that were evaluated are often part of standard clinical practice. Therefore, data might be biased (e.g. …
SciScore for 10.1101/2020.07.26.20157040: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
Since all models are a simplification of reality, our model has several limitations. First, the survival data used were not always derived from high-quality evidence. Although survival with treatment might be validly estimated from national registries, the survival without treatment is harder to be unbiasedly estimated. The surgeries that were evaluated are often part of standard clinical practice. Therefore, data might be biased (e.g. selection bias in the survival without treatment because patients opt for palliative care), or not available (it would be unethical now to perform RCTs evaluating surgery versus no surgery). Instead, best available evidence was used, which in part included evidence from more historical RCTs. As such, data might be biased, and as a result, so are the estimates from our model. Because of this limitation, our approach is simply to aggregate transparently and systematically the best currently available evidence using a model. Second, it was assumed that all surgeries are successful. The current model does not simulate adverse events, like major bleedings or death due to surgery. Also, the potential reduction of QoL due to these adverse events were not incorporated, nor the QoL reduction of a temporary period of recovery after surgery. Because of these assumptions, the overall QALYs associated with the surgery should not be interpreted as an absolute estimate. They are the maximum possible QALYs that can be acquired by performing the surgery. However, these assumptions were considered reasonable to achieve the main goal of this study: when surgery without delay is compared to surgery with delay, the harm in both scenarios is similar and therefore cancel out. Third, because the health loss in 50 weeks was simply converted to loss per month, a linear approximation was effectively used to quantify urgency by delaying surgery up to a year. Some surgeries did show a slightly steeper decrease in the period up to 32 weeks delay. The data needed to validly model this decay in QALYs per unit of time for all surgeries likely don’t exist: most of the estimates of time to no effect on survival were based on observational studies, which are likely biased. A more detailed approximation would be possible using a more individualized model which also models the natural grow of tumours, or aneurysms, and validly model the development of metastasis. It was not feasible to develop this for all evaluated surgeries. Instead, we opted for a more pragmatic approach. Fourth, QoL weights were derived from expert-opinion. In this approach the patient is not involved. Experts interpret the health states and give weights, thereby our approach takes a societal perspective. There are also multiple methodological, ethical, and contextual disadvantages of using QALYs, but it should be noted that most of those discussion are more about utilitarian principles.84 Fifth, the potential impact on QoL of delaying a semi-elective surgery was not included. This impact might differ across surgeries. It might be hypothesized that surgeries performed after already a long disease history (e.g. kidney transplant) might have less “waiting time disutility” than recently diagnosed diseases (e.g. mammacarcinoma). Part of the input parameters were based on national registry data, but a substantial amount of the input originated from various international sources. Therefore, with some modifications, the model can easily be adapted to different contexts. Therefore, this study can be considered the first step towards a triaging strategy which optimizes surgical benefit in times of scarcity in surgical capacity, such as during the COVID-19 pandemic. To improve validity, it is however essential to periodically review the literature and update the model with higher quality evidence, much like a living systematic review.85 If accepted, a wider range of surgeries should be considered, implementation strategies should be explored and evaluated, and the model should be applied to a variety of settings. Conclusion By transparently aggregating best available evidence, our decision model support prioritization of surgical care in times of scarce surgical capacity (e.g. during pandemics) from a utilitarian perspective. Our approach quantifies the expected health loss due to delay for semi-elective surgeries in an academic hospital in the Netherlands. This approach can help to minimize health losses when trying to overcome delay in surgeries across disciplines. This approach is more transparent, more evidence-based, and more consistent than the alternative strategy of triaging based on expert opinion. It should be noted that evidence from well-controlled comparison studies is often lacking. Instead, adjusted estimates from observational studies are often the best available evidence for benefit of surgery and the effects of delay on survival. Therefore, model inputs should be periodically updated with newer, higher quality evidence. Finally, our approach should be placed in the context of other ethical perspectives and combined with capacity management tools. If accepted, we believe this tool should be implemented on a large scale, in order to minimize health loss of the accumulating group of patients awaiting surgery. Authors contribution : Conceptualization (Lead), Data curation (Lead), Formal analysis (Lead) Investigation (Lead), Methodology (Lead), Validation (Equal), Visualization (Lead), Writingoriginal draft (Lead) Eline Krijkamp: Conceptualization (Lead), Data curation (Equal), Formal analysis (Lead), Investigation (Lead), Methodology (Lead), Validation (Supporting), Visualization (Lead), Writing-original draft (Lead) Jan Busschbach: Conceptualization (Equal), Investigation (Equal), Methodology (Equal), Supervision (Supporting), Validation (Equal), Writing-review & editing (Equal) Geert Geleijnse: Conceptualization (Equal), Data curation (Equal), Investigation (Supporting), Supervision (Supporting), Validation (Equal), Visualization (Supporting), Writing-review & editing (Equal) Isabel Retel Helmrich: Data curation (Equal), Investigation (Equal), Methodology (Supporting), Writing-review & editing (Supporting) Sophie Bruinsma: Data curation (Equal), Investigation (Equal), Writing-review & editing (Supporting) Céline van Lint: Data curation (Equal), Investigation (Equal), Writing-review & editing (Supporting) Ernest van Veen: Data curation (Supporting), Investigation (Supporting), Writing-review & editing (Supporting) Ewout Steyerberg: Methodology (Supporting), Validation (Equal), Writing-review & editing (Supporting) Kees Verhoef: Investigation (Supporting), Methodology (Supporting), Validation (Equal), Writing-review & editing (Supporting) Jan van Saase: Conceptualization (Equal), Investigation (Equal), Supervision (Supporting), Validation (Equal), Writing-original draft (Equal), Writing-review & editing (Equal) Hester Lingsma: Conceptualization (Lead), Investigation (Lead), Supervision (Lead), Writingoriginal draft (Equal), Writing-review & editing (Equal) Rob Baatenburg de Jong: Conceptualization (Lead), Investigation (Lead), Supervision (Lead), Validation (Supporting), Writing-original draft (Equal), Writing-review & editing (Equal) Benjamin Gravesteijn Funding No specific funds were rewarded for this project. Patient and public involvement Already in the early phases, in the process of the design of this study, patient and public involvement was achieved by communicating our ideas via press-releases, interviews and open discussions. The aim for having this discussion was to identify barriers and facilitators of our approach, also possibly informing the study design. Of course, it was not appropriate to include patient and/or public in the analysis of the study and writing the manuscript. However, medical experts, representing a part of the public, were included in the process of data collection for the quality of life values and further development of the methods. Moreover, the department of surgery within our hospital, an important end-user of our approach, tested the approach on face-validity and helped writing the manuscript, thereby showing their support. Currently, we are also presenting our approach to patient federations. They have already expressed widespread interest and support to our approach. Finally, we are discussing with these stakeholders possible other barriers and facilitators for implementation which were not already identified. Disclosures Isabel Retel Helmrich and Ernest van Veen are supported by the European Union 7th Framework program (Center-TBI, EC grant 602150). Eline Krijkamp is supported by the Society for Medical Decision Making (SMDM) fellowship through a grant by the Gordon and Betty Moore Foundation (GBMF7853). Acknowledgement We are grateful for Lisa Caulley for her suggestions to revise the final manuscript. We are grateful for H. Karreman and C. Van der Velden-van der Graaf for the work they have done for the quality of life data collection. Moreover, we want to thank Ruben Goedhart, Esther van Spronsen and Linda van der Sluijs – van der Beek for extracting the data from the electronic patient registry. Manuscript details Word count: 3100 Word count abstract = 241 Keywords: health care planning, COVID-19, population health, prioritization, simulation model, surgery delay Appendix A An overview per disease of the distribution and source of the input parameters and a graphical representation of the output of the model. Appendix B A summary of the estimates of the decision model and an overview of the counts, duration, and length of stay of the included surgeries in our hospital. Appendix C In-dept methods description.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
About SciScore
SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore is not a substitute for expert review. SciScore checks for the presence and correctness of RRIDs (research resource identifiers) in the manuscript, and detects sentences that appear to be missing RRIDs. SciScore also checks to make sure that rigor criteria are addressed by authors. It does this by detecting sentences that discuss criteria such as blinding or power analysis. SciScore does not guarantee that the rigor criteria that it detects are appropriate for the particular study. Instead it assists authors, editors, and reviewers by drawing attention to sections of the manuscript that contain or should contain various rigor criteria and key resources. For details on the results shown here, including references cited, please follow this link.
-
SciScore for 10.1101/2020.07.26.20157040: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
Since all models are a simplification of reality, our model has several limitations. First, the survival data used were not always derived from high-quality evidence. Although survival with treatment might be validly estimated from national registries, the survival without treatment is harder to be unbiasedly estimated. The surgeries that were evaluated are often part of standard clinical practice. Therefore, data might be biased (e.g. …
SciScore for 10.1101/2020.07.26.20157040: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
NIH rigor criteria are not applicable to paper type.Table 2: Resources
No key resources detected.
Results from OddPub: Thank you for sharing your code.
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
Since all models are a simplification of reality, our model has several limitations. First, the survival data used were not always derived from high-quality evidence. Although survival with treatment might be validly estimated from national registries, the survival without treatment is harder to be unbiasedly estimated. The surgeries that were evaluated are often part of standard clinical practice. Therefore, data might be biased (e.g. selection bias in the survival without treatment because patients opt for palliative care), or not available (it would be unethical now to perform RCTs evaluating surgery versus no surgery). Instead, best available evidence was used, which in part included evidence from more historical RCTs. As such, data might be biased, and as a result, so are the estimates from our model. Because of this limitation, our approach is simply to aggregate transparently and systematically the best currently available evidence using a model. Second, it was assumed that all surgeries are successful. The current model does not simulate adverse events, like major bleedings or death due to surgery. Also, the potential reduction of QoL due to these adverse events were not incorporated, nor the QoL reduction of a temporary period of recovery after surgery. Because of these assumptions, the overall QALYs associated with the surgery should not be interpreted as an absolute estimate. They are the maximum possible QALYs that can be acquired by performing the surgery. However, these assumptions were considered reasonable to achieve the main goal of this study: when surgery without delay is compared to surgery with delay, the harm in both scenarios is similar and therefore cancel out. Third, because the health loss in 50 weeks was simply converted to loss per month, a linear approximation was effectively used to quantify urgency by delaying surgery up to a year. Some surgeries did show a slightly steeper decrease in the period up to 32 weeks delay. The data needed to validly model this decay in QALYs per unit of time for all surgeries likely don’t exist: most of the estimates of time to no effect on survival were based on observational studies, which are likely biased. A more detailed approximation would be possible using a more individualized model which also models the natural grow of tumours, or aneurysms, and validly model the development of metastasis. It was not feasible to develop this for all evaluated surgeries. Instead, we opted for a more pragmatic approach. Fourth, QoL weights were derived from expert-opinion. In this approach the patient is not involved. Experts interpret the health states and give weights, thereby our approach takes a societal perspective. There are also multiple methodological, ethical, and contextual disadvantages of using QALYs, but it should be noted that most of those discussion are more about utilitarian principles.84 Fifth, the potential impact on QoL of delaying a semi-elective surgery was not included. This impact might differ across surgeries. It might be hypothesized that surgeries performed after already a long disease history (e.g. kidney transplant) might have less “waiting time disutility” than recently diagnosed diseases (e.g. mammacarcinoma). Part of the input parameters were based on national registry data, but a substantial amount of the input originated from various international sources. Therefore, with some modifications, the model can easily be adapted to different contexts. Therefore, this study can be considered the first step towards a triaging strategy which optimizes surgical benefit in times of scarcity in surgical capacity, such as during the COVID-19 pandemic. To improve validity, it is however essential to periodically review the literature and update the model with higher quality evidence, much like a living systematic review.85 If accepted, a wider range of surgeries should be considered, implementation strategies should be explored and evaluated, and the model should be applied to a variety of settings. Conclusion By transparently aggregating best available evidence, our decision model support prioritization of surgical care in times of scarce surgical capacity (e.g. during pandemics) from a utilitarian perspective. Our approach quantifies the expected health loss due to delay for semi-elective surgeries in an academic hospital in the Netherlands. This approach can help to minimize health losses when trying to overcome delay in surgeries across disciplines. This approach is more transparent, more evidence-based, and more consistent than the alternative strategy of triaging based on expert opinion. It should be noted that evidence from well-controlled comparison studies is often lacking. Instead, adjusted estimates from observational studies are often the best available evidence for benefit of surgery and the effects of delay on survival. Therefore, model inputs should be periodically updated with newer, higher quality evidence. Finally, our approach should be placed in the context of other ethical perspectives and combined with capacity management tools. If accepted, we believe this tool should be implemented on a large scale, in order to minimize health loss of the accumulating group of patients awaiting surgery. Authors contribution : Conceptualization (Lead), Data curation (Lead), Formal analysis (Lead) Investigation (Lead), Methodology (Lead), Validation (Equal), Visualization (Lead), Writingoriginal draft (Lead) Eline Krijkamp: Conceptualization (Lead), Data curation (Equal), Formal analysis (Lead), Investigation (Lead), Methodology (Lead), Validation (Supporting), Visualization (Lead), Writing-original draft (Lead) Jan Busschbach: Conceptualization (Equal), Investigation (Equal), Methodology (Equal), Supervision (Supporting), Validation (Equal), Writing-review & editing (Equal) Geert Geleijnse: Conceptualization (Equal), Data curation (Equal), Investigation (Supporting), Supervision (Supporting), Validation (Equal), Visualization (Supporting), Writing-review & editing (Equal) Isabel Retel Helmrich: Data curation (Equal), Investigation (Equal), Methodology (Supporting), Writing-review & editing (Supporting) Sophie Bruinsma: Data curation (Equal), Investigation (Equal), Writing-review & editing (Supporting) Céline van Lint: Data curation (Equal), Investigation (Equal), Writing-review & editing (Supporting) Ernest van Veen: Data curation (Supporting), Investigation (Supporting), Writing-review & editing (Supporting) Ewout Steyerberg: Methodology (Supporting), Validation (Equal), Writing-review & editing (Supporting) Kees Verhoef: Investigation (Supporting), Methodology (Supporting), Validation (Equal), Writing-review & editing (Supporting) Jan van Saase: Conceptualization (Equal), Investigation (Equal), Supervision (Supporting), Validation (Equal), Writing-original draft (Equal), Writing-review & editing (Equal) Hester Lingsma: Conceptualization (Lead), Investigation (Lead), Supervision (Lead), Writingoriginal draft (Equal), Writing-review & editing (Equal) Rob Baatenburg de Jong: Conceptualization (Lead), Investigation (Lead), Supervision (Lead), Validation (Supporting), Writing-original draft (Equal), Writing-review & editing (Equal) Benjamin Gravesteijn Funding No specific funds were rewarded for this project. Patient and public involvement Already in the early phases, in the process of the design of this study, patient and public involvement was achieved by communicating our ideas via press-releases, interviews and open discussions. The aim for having this discussion was to identify barriers and facilitators of our approach, also possibly informing the study design. Of course, it was not appropriate to include patient and/or public in the analysis of the study and writing the manuscript. However, medical experts, representing a part of the public, were included in the process of data collection for the quality of life values and further development of the methods. Moreover, the department of surgery within our hospital, an important end-user of our approach, tested the approach on face-validity and helped writing the manuscript, thereby showing their support. Currently, we are also presenting our approach to patient federations. They have already expressed widespread interest and support to our approach. Finally, we are discussing with these stakeholders possible other barriers and facilitators for implementation which were not already identified. Disclosures Isabel Retel Helmrich and Ernest van Veen are supported by the European Union 7th Framework program (Center-TBI, EC grant 602150). Eline Krijkamp is supported by the Society for Medical Decision Making (SMDM) fellowship through a grant by the Gordon and Betty Moore Foundation (GBMF7853). Acknowledgement We are grateful for Lisa Caulley for her suggestions to revise the final manuscript. We are grateful for H. Karreman and C. Van der Velden-van der Graaf for the work they have done for the quality of life data collection. Moreover, we want to thank Ruben Goedhart, Esther van Spronsen and Linda van der Sluijs – van der Beek for extracting the data from the electronic patient registry. Manuscript details Word count: 3100 Word count abstract = 241 Keywords: health care planning, COVID-19, population health, prioritization, simulation model, surgery delay Appendix A An overview per disease of the distribution and source of the input parameters and a graphical representation of the output of the model. Appendix B A summary of the estimates of the decision model and an overview of the counts, duration, and length of stay of the included surgeries in our hospital. Appendix C In-dept methods description.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
About SciScore
SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore is not a substitute for expert review. SciScore checks for the presence and correctness of RRIDs (research resource identifiers) in the manuscript, and detects sentences that appear to be missing RRIDs. SciScore also checks to make sure that rigor criteria are addressed by authors. It does this by detecting sentences that discuss criteria such as blinding or power analysis. SciScore does not guarantee that the rigor criteria that it detects are appropriate for the particular study. Instead it assists authors, editors, and reviewers by drawing attention to sections of the manuscript that contain or should contain various rigor criteria and key resources. For details on the results shown here, including references cited, please follow this link.
-
