Development and content validation of the Symptoms Evolution of COVID-19: a patient-reported electronic daily diary in clinical and real-world studies

  1. Diana Rofail
  2. Nadine McGale
  3. Joseph Im
  4. Alissa Rams
  5. Krystian Przydzial
  6. Vera Mastey
  7. Sumathi Sivapalasingam
  8. Anna J. Podolanczuk

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Background

At the onset of the COVID-19 pandemic, there was limited understanding of symptom experience and disease progression. We developed and validated a fit-for-purpose disease-specific instrument to assess symptoms in patients with COVID-19 to inform endpoints in an interventional trial for non-hospitalized patients.

Methods

The initial drafting of the 23-item Symptoms Evolution of COVID-19 (SE-C19) Instrument was developed based on the Centers for Disease Control and Prevention symptom list and available published literature specific to patients with COVID-19 as of Spring 2020. The measurement principles outlined in the Food and Drug Administration (FDA) Patient-Reported Outcomes (PRO) guidance and the FDA's series of four methodological Patient-Focused Drug Development guidance documents were also considered. Following initial development, semi-structured qualitative interviews were conducted with a purposive sample of 30 non-hospitalized COVID-19 patients. Interviews involved two stages: (1) concept elicitation, to obtain information about the symptoms experienced as a result of COVID-19 in the patients’ own words, and (2) cognitive debriefing, for patients to describe their understanding of the SE-C19 instructions, specific symptoms, response options, and recall period to ensure the content of the SE-C19 is relevant and comprehensive. Five clinicians treating COVID-19 outpatients were also interviewed to obtain their insights on symptoms experienced by patients and provide input on the SE-C19.

Results

Patients reported no issues regarding the relevance or appropriateness of the SE-C19 instructions, including the 24-h recall period. The comprehensiveness of the SE-C19 was confirmed against the conceptualization of the patient experience of symptoms developed in the qualitative research. Minor conceptual gaps were revealed to capture nuances in the experience of nasal and gustatory symptoms and systemic manifestations of sickness. Almost all items were endorsed by patients as being appropriate, well understood, and easy to respond to. The clinicians largely approved all items, response options, and recall period.

Conclusions

The qualitative research provided supportive evidence of the content validity of the SE-C19 to assess the symptoms of outpatients with COVID-19, and its use in clinical trials to evaluate the benefit of treatment. Minor changes may be considered to improve conceptual clarity and ease of responding.

Article activity feed

  1. Version published to 10.1186/s41687-022-00448-9
  2. ScreenIT

    SciScore for 10.1101/2021.07.06.21259654: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    EthicsConsent: Ethics: Study documents, including the protocol, demographic and health information form, interview guide, screener, informed consent form, and SE-C19, received ethical approval from the New England Independent Review Board (study number: 1291666) prior to any contact with patients.
    IRB: Ethics: Study documents, including the protocol, demographic and health information form, interview guide, screener, informed consent form, and SE-C19, received ethical approval from the New England Independent Review Board (study number: 1291666) prior to any contact with patients.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Transcripts were coded in ATLAS.ti software22 using an open, inductive coding approach.
    ATLAS.ti
    suggested: None

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    The current study has a few limitations. Some outpatients were no longer experiencing symptoms at the time of the interviews, as they were completed an average of 12 days after their positive diagnosis. However, outpatients had vivid memories of their past symptoms and were able to provide detailed feedback for the SE-C19. Due to the small sample size of the current study, further research should be conducted to confirm the validity and reliability of all the SE-C19 questions, using classical and modern test theory to support the scoring of the measure and define responders. This will also provide evidence for the minor adjustments suggested based on the current sample’s feedback. Upon completion of further analyses, this provides a valid, evidence-based PRO assessment of COVID-19 that can be considered for future use by patients and healthcare professionals. While the current study focused on outpatients in the acute phase of COVID-19, additional research could further explore the symptom trajectory and impact on daily lives in long-term COVID.26 In this context, the SE-C19 could be a useful tool to better understand persistent symptoms.

    Results from TrialIdentifier: We found the following clinical trial numbers in your paper:

    IdentifierStatusTitle
    NCT04425629RecruitingSafety, Tolerability, and Efficacy of Anti-Spike (S) SARS-Co…

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.07.06.21259654v1 on medRxiv