Eleven key measures for monitoring general practice clinical activity during COVID-19: A retrospective cohort study using 48 million adults’ primary care records in England through OpenSAFELY
Curation statements for this article:-
Curated by eLife
eLife assessment
This paper presents an important effort to develop an open source software framework for monitoring trends and variations in healthcare over time in England. They demonstrate a compelling example of how this system can track key health care indicators over the course of the COVID-19 pandemic. The paper will likely be mainly of interest to stakeholders in England, but could inspire the creation of similar systems in other countries.
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (eLife)
Abstract
The COVID-19 pandemic has had a significant impact on delivery of NHS care. We have developed the OpenSAFELY Service Restoration Observatory (SRO) to develop key measures of primary care activity and describe the trends in these measures throughout the COVID-19 pandemic.
Methods:
With the approval of NHS England, we developed an open source software framework for data management and analysis to describe trends and variation in clinical activity across primary care electronic health record (EHR) data on 48 million adults.
We developed SNOMED-CT codelists for key measures of primary care clinical activity such as blood pressure monitoring and asthma reviews, selected by an expert clinical advisory group and conducted a population cohort-based study to describe trends and variation in these measures January 2019-December 2021, and pragmatically classified their level of recovery one year into the pandemic using the percentage change in the median practice level rate.
Results:
We produced 11 measures reflective of clinical activity in general practice. A substantial drop in activity was observed in all measures at the outset of the COVID-19 pandemic. By April 2021, the median rate had recovered to within 15% of the median rate in April 2019 in six measures. The remaining measures showed a sustained drop, ranging from a 18.5% reduction in medication reviews to a 42.0% reduction in blood pressure monitoring. Three measures continued to show a sustained drop by December 2021.
Conclusions:
The COVID-19 pandemic was associated with a substantial change in primary care activity across the measures we developed, with recovery in most measures. We delivered an open source software framework to describe trends and variation in clinical activity across an unprecedented scale of primary care data. We will continue to expand the set of key measures to be routinely monitored using our publicly available NHS OpenSAFELY SRO dashboards with near real-time data.
Funding:
This research used data assets made available as part of the Data and Connectivity National Core Study, led by Health Data Research UK in partnership with the Office for National Statistics and funded by UK Research and Innovation (grant ref MC_PC_20058).The OpenSAFELY Platform is supported by grants from the Wellcome Trust (222097/Z/20/Z); MRC (MR/V015757/1, MC_PC-20059, MR/W016729/1); NIHR (NIHR135559, COV-LT2-0073), and Health Data Research UK (HDRUK2021.000, 2021.0157).
Article activity feed
-
-
Author Response
Reviewer #1 (Public Review):
This article describes the development and refinement of an open-source software framework that is used to track how the COVID-19 pandemic impacted healthcare use in England over a range of key healthcare use indicators.
Important strengths of this study include the high coverage of 99% of practices in England, the development of health care indicators with the input of a clinical advisory group, extensive online documentation, and rigorous safeguards for the protection of patient confidentiality.
Perhaps the largest limitation is that only high-level descriptive data on the monthly volume of health outcomes are presented. It is not clear whether the system could be used to generate more fine-grained or stratified information, ex. weekly or daily data, or data stratified by important …
Author Response
Reviewer #1 (Public Review):
This article describes the development and refinement of an open-source software framework that is used to track how the COVID-19 pandemic impacted healthcare use in England over a range of key healthcare use indicators.
Important strengths of this study include the high coverage of 99% of practices in England, the development of health care indicators with the input of a clinical advisory group, extensive online documentation, and rigorous safeguards for the protection of patient confidentiality.
Perhaps the largest limitation is that only high-level descriptive data on the monthly volume of health outcomes are presented. It is not clear whether the system could be used to generate more fine-grained or stratified information, ex. weekly or daily data, or data stratified by important characteristics of practices or of patient characteristics. As such, the utility of the system for answering new scientific questions is unclear, and also what the utility and long-term potential uses of this system will be past the COVID-19 pandemic.
OpenSAFELY allows access to the full primary care record for patients registered with a TPP or EMIS practice in England.This includes medical diagnoses, clinical tests, prescriptions, as well as demographic details such as age, sex, ethnicity. Dates attached to these records allow for daily analyses to be performed. This data is updated weekly. Through linkage of other data sources, it also provides information such as hospital admissions, registered deaths or COVID-19 testing data. Detailed subgroup analysis is possible; OpenSAFELY has already been used to understand disease risk 1, monitor vaccination coverage 2,3 and novel treatments 4, assess patient safety 5, inform public health guidance and policy and much more6. These are all widely applicable beyond the COVID-19 pandemic.
Reviewer #3 (Public Review):
This manuscript by Fisher and colleagues documents the change in clinical activity in English general practices during the COVID-19 pandemic according to a set of indicators of clinical activity. The indicators include measures of clinical reviews (e.g. blood pressure, asthma, chronic obstructive pulmonary disease, medication, and cardiovascular risk reviews), blood tests (e.g. cholesterol, liver function, thyroid function, full blood counts, diabetes monitoring blood tests, and kidney function). All these measures saw a drop during the pandemic, to a varying degree, and some recovered afterwards but others did not.
Clinical activity was measured using SNOMED CT codes, which are standard codes used for recording clinical events in UK GP records.
Strengths:
This is a large and comprehensive study including data from 99% of general practices in England. The indicators are clinically relevant, cover a broad range of disease areas, and have been chosen in a sensible manner, involving relevant stakeholders such as GPs, pharmacists, and pathologists.
The OpenSAFELY platform has the ability to enable federated analyses to be run on raw coded data of almost all patients registered with a GP in England.
The study demonstrates the value of OpenSAFELY in being able to monitor clinical activity in general practice at a detailed level, which is essential for planning and improving health services. The statistical methodology is broadly sound.
Weaknesses:
The measures are all related to chronic physical diseases in adults, with a particular focus on cardiometabolic and respiratory conditions. There are no measures related to mental health, maternal or child health.
Results from preliminary analyses of a wider range of clinical conditions can be found in our previous work7. This includes mental health and female and reproductive health with details on why these were not covered by the initial key measures described.
The description of the measures does not distinguish between different types of clinical activity e.g. lab tests, clinical measurements, or diagnoses, and all are lumped together as 'codes'. This is a peculiarity of the way that information is recorded in GP systems - many different types of clinical information (such as diagnoses and lab tests) are recorded using a SNOMED CT 'code', and only the exact code differentiates what type of information is in the record.
Multiple codes of different types can arise from a single encounter, all of which could be indicative of a clinical event of interest. The codelists for each key measure, available at opencodelists.org shows the type of clinical activity (e.g procedure or observable entity) captured by each code within the codelist (see e.g.https://www.opencodelists.org/codelist/opensafely/red-blood-cell-rbc-tests/576a859e/#tree).
The codelists were broad and comprehensive, but it is unclear how necessary this is because for some measures e.g. lab tests, laboratories typically record a particular type of test using a single standardised code. Instead of using a broad set of codes in the analysis, the authors could have initially verified which codes are associated with the clinical activity being measured (e.g. a numerical value of a blood pressure measurement) in all practices, as I would expect the same single or small number of codes would be used in all practices. This would have provided a smaller and simpler final codelist.
Supplementary table 1 shows up to 5 of the most common codes for each key measure across the two electronic health record (EHR) systems used in this analysis. This shows that whilst a single code is often used for many of the clinical activities assessed here, there are exceptions and there can be variation in coded activity between different EHR systems. We have previously described how design features of EHR systems can impact clinical practice 8. Broad codelists allow us to capture activity across multiple EHR systems.
- Williamson, E. J. et al. Factors associated with COVID-19-related death using OpenSAFELY. Nature 584, 430–436 (2020).
- Trends and clinical characteristics of 57.9 million COVID-19 vaccine recipients: a federated analysis of patients’ primary care records in situ using OpenSAFELY | British Journal of General Practice. https://bjgp.org/content/early/2021/11/08/BJGP.2021.0376.
- Parker, E. P. et al. Factors associated with COVID-19 vaccine uptake in people with kidney disease: an OpenSAFELY cohort study. BMJ Open 13, e066164 (2023).
- Green, A. C. A. et al. Trends, variation, and clinical characteristics of recipients of antiviral drugs and neutralising monoclonal antibodies for covid-19 in community settings: retrospective, descriptive cohort study of 23.4 million people in OpenSAFELY. BMJ Med. 2, (2023).
- Collaborative, T. O. et al. Potentially inappropriate prescribing of DOACs to people with mechanical heart valves: a federated analysis of 57.9 million patients’ primary care records in situ using OpenSAFELY. 2021.07.27.21261136 https://www.medrxiv.org/content/10.1101/2021.07.27.21261136v1 (2021) doi:10.1101/2021.07.27.21261136.
- OpenSAFELY Pubmed search results. PubMed https://pubmed.ncbi.nlm.nih.gov/?term=OpenSAFELY.
- OpenSAFELY NHS Service Restoration Observatory 2: changes in primary care activity across six clinical areas during the COVID-19 pandemic | medRxiv. https://www.medrxiv.org/content/10.1101/2022.06.01.22275674v1.
- Suboptimal prescribing behaviour associated with clinical software design features: a retrospective cohort study in English NHS primary care | British Journal of General Practice. https://bjgp.org/content/70/698/e636.
-
eLife assessment
This paper presents an important effort to develop an open source software framework for monitoring trends and variations in healthcare over time in England. They demonstrate a compelling example of how this system can track key health care indicators over the course of the COVID-19 pandemic. The paper will likely be mainly of interest to stakeholders in England, but could inspire the creation of similar systems in other countries.
-
Reviewer #1 (Public Review):
This article describes the development and refinement of an open-source software framework that is used to track how the COVID-19 pandemic impacted healthcare use in England over a range of key healthcare use indicators.
Important strengths of this study include the high coverage of 99% of practices in England, the development of health care indicators with the input of a clinical advisory group, extensive online documentation, and rigorous safeguards for the protection of patient confidentiality.
Perhaps the largest limitation is that only high-level descriptive data on the monthly volume of health outcomes are presented. It is not clear whether the system could be used to generate more fine-grained or stratified information, ex. weekly or daily data, or data stratified by important characteristics of …
Reviewer #1 (Public Review):
This article describes the development and refinement of an open-source software framework that is used to track how the COVID-19 pandemic impacted healthcare use in England over a range of key healthcare use indicators.
Important strengths of this study include the high coverage of 99% of practices in England, the development of health care indicators with the input of a clinical advisory group, extensive online documentation, and rigorous safeguards for the protection of patient confidentiality.
Perhaps the largest limitation is that only high-level descriptive data on the monthly volume of health outcomes are presented. It is not clear whether the system could be used to generate more fine-grained or stratified information, ex. weekly or daily data, or data stratified by important characteristics of practices or of patient characteristics. As such, the utility of the system for answering new scientific questions is unclear, and also what the utility and long-term potential uses of this system will be past the COVID-19 pandemic.
-
Reviewer #2 (Public Review):
The authors developed 11 key measures of clinical activity in primary care and measured changes in the frequency of these measures throughout the first 1.5 years of the COVID-19 pandemic. The biggest strength of the study is the data source, which comprises records from 99% of general practices in England. The biggest limitation lies in the analysis of the data: The authors used only descriptive statistics for the investigation of time trends and have not accounted for long-term time trends (only one "control year" was considered). Still, owing to the large study size, the time trends observed are convincing. The work is of high significance to the field because the OpenSAFELY platform will enable the continuous and real-time monitoring of primary care activity.
-
Reviewer #3 (Public Review):
This manuscript by Fisher and colleagues documents the change in clinical activity in English general practices during the COVID-19 pandemic according to a set of indicators of clinical activity. The indicators include measures of clinical reviews (e.g. blood pressure, asthma, chronic obstructive pulmonary disease, medication, and cardiovascular risk reviews), blood tests (e.g. cholesterol, liver function, thyroid function, full blood counts, diabetes monitoring blood tests, and kidney function). All these measures saw a drop during the pandemic, to a varying degree, and some recovered afterwards but others did not.
Clinical activity was measured using SNOMED CT codes, which are standard codes used for recording clinical events in UK GP records.
Strengths:
This is a large and comprehensive study including …
Reviewer #3 (Public Review):
This manuscript by Fisher and colleagues documents the change in clinical activity in English general practices during the COVID-19 pandemic according to a set of indicators of clinical activity. The indicators include measures of clinical reviews (e.g. blood pressure, asthma, chronic obstructive pulmonary disease, medication, and cardiovascular risk reviews), blood tests (e.g. cholesterol, liver function, thyroid function, full blood counts, diabetes monitoring blood tests, and kidney function). All these measures saw a drop during the pandemic, to a varying degree, and some recovered afterwards but others did not.
Clinical activity was measured using SNOMED CT codes, which are standard codes used for recording clinical events in UK GP records.
Strengths:
This is a large and comprehensive study including data from 99% of general practices in England. The indicators are clinically relevant, cover a broad range of disease areas, and have been chosen in a sensible manner, involving relevant stakeholders such as GPs, pharmacists, and pathologists.
The OpenSAFELY platform has the ability to enable federated analyses to be run on raw coded data of almost all patients registered with a GP in England.
The study demonstrates the value of OpenSAFELY in being able to monitor clinical activity in general practice at a detailed level, which is essential for planning and improving health services. The statistical methodology is broadly sound.
Weaknesses:
The measures are all related to chronic physical diseases in adults, with a particular focus on cardiometabolic and respiratory conditions. There are no measures related to mental health, maternal or child health.
The description of the measures does not distinguish between different types of clinical activity e.g. lab tests, clinical measurements, or diagnoses, and all are lumped together as 'codes'. This is a peculiarity of the way that information is recorded in GP systems - many different types of clinical information (such as diagnoses and lab tests) are recorded using a SNOMED CT 'code', and only the exact code differentiates what type of information is in the record.
The codelists were broad and comprehensive, but it is unclear how necessary this is because for some measures e.g. lab tests, laboratories typically record a particular type of test using a single standardised code. Instead of using a broad set of codes in the analysis, the authors could have initially verified which codes are associated with the clinical activity being measured (e.g. a numerical value of a blood pressure measurement) in all practices, as I would expect the same single or small number of codes would be used in all practices. This would have provided a smaller and simpler final codelist.
-
-