Self-Reported SARS-CoV-2 Vaccination Is Consistent with Electronic Health Record Data among the COVID-19 Community Research Partnership

  1. Ashley H. Tjaden
  2. Lida M. Fette
  3. Sharon L. Edelstein
  4. Michael Gibbs
  5. Amy N. Hinkelman
  6. Michael Runyon
  7. Roberto P. Santos
  8. William S. Weintraub
  9. Joshua Yukich
  10. Diane Uschner
  11. the COVID-19 Community Research Partnership Study Group

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

Introduction: Observational studies of SARS-CoV-2 vaccine effectiveness depend on accurate ascertainment of vaccination receipt, date, and product type. Self-reported vaccine data may be more readily available to and less expensive for researchers than assessing medical records. Methods: We surveyed adult participants in the COVID-19 Community Research Partnership who had an authenticated Electronic Health Record (EHR) (N = 41,484) concerning receipt of SARS-CoV-2 vaccination using a daily survey beginning in December 2020 and a supplemental survey in September–October 2021. We compared self-reported information to that available in the EHR for the following data points: vaccine brand, date of first dose, and number of doses using rates of agreement and Bland–Altman plots for visual assessment. Self-reported data was available immediately following vaccination (in the daily survey) and at a delayed interval (in a secondary supplemental survey). Results: For the date of first vaccine dose, self-reported “immediate” recall was within ±7 days of the date reported in the “delayed” survey for 87.9% of participants. Among the 19.6% of participants with evidence of vaccination in their EHR, 95% self-reported vaccination in one of the two surveys. Self-reported dates were within ±7 days of documented EHR vaccination for 97.6% of the “immediate” surveys and 92.0% of the “delayed” surveys. Self-reported vaccine product details matched those in the EHR for over 98% of participants for both “immediate” and “delayed” surveys. Conclusions: Self-reported dates and product details for COVID-19 vaccination can be a good surrogate when medical records are unavailable in large observational studies. A secondary confirmation of dates for a subset of participants with EHR data will provide internal validity.

Article activity feed

  1. Version published to 10.3390/vaccines10071016
  2. ScreenIT

    SciScore for 10.1101/2022.05.13.22275054: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsConsent: All participants provided informed consent, and Institutional Review Board (IRB) approval was provided by the Wake Forest School of Medicine IRB.
    IRB: All participants provided informed consent, and Institutional Review Board (IRB) approval was provided by the Wake Forest School of Medicine IRB.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    No key resources detected.

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Our study has a number of important limitations. The CCRP is a U.S. healthcare system-based convenience sample that is predominantly non-Hispanic White and about a quarter are healthcare workers. Participants active in the study are likely to be generally interested in their health and may have greater awareness of the details surrounding vaccination. These results may not be consistent in populations of varying race/ethnicity, occupation, education level, health literacy, or region. Importantly, the results are relevant only to the U.S. population where the details of COVID-19 vaccines have been discussed extensively in the media and may not be generalizable to other regions with different media landscapes, and vaccine attitudes and availability. Additionally, there were significant differences between participants with and without documentation of vaccination in EHR: female participants, healthcare workers, participants in the South East, and participants residing in urban areas were more likely to have vaccine information in their EHR (Appendix Table 2). There were also differences by enrollment site. This difference in availability of EHR data highlights the importance of accepting self-reported vaccine dates particularly where EHR vaccination data may be sparse. Thus, we have not computed sensitivity or specificity of self-reported receipt of vaccination. Additionally, due to the structure of the vaccination data we received from EHR (unnumbered doses), it was difficult ...

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04342884RecruitingCOVID-19 Community Research Partnership

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2022.05.13.22275054v1 on medRxiv