Comparison of Mental Health Symptoms Prior to and During COVID-19: Evidence from a Systematic Review and Meta-analysis of 134 Cohorts

  1. Ying Sun
  2. Yin Wu
  3. Suiqiong Fan
  4. Tiffany Dal Santo
  5. Letong Li
  6. Xiaowen Jiang
  7. Kexin Li
  8. Yutong Wang
  9. Amina Tasleem
  10. Ankur Krishnan
  11. Chen He
  12. Olivia Bonardi
  13. Jill T. Boruff
  14. Danielle B. Rice
  15. Sarah Markham
  16. Brooke Levis
  17. Marleine Azar
  18. Ian Thombs-Vite
  19. Dipika Neupane
  20. Branka Agic
  21. Christine Fahim
  22. Michael S. Martin
  23. Sanjeev Sockalingam
  24. Gustavo Turecki
  25. Andrea Benedetti
  26. Brett D. Thombs

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Objectives

The rapid pace, high volume, and limited quality of mental health evidence that has been generated during COVID-19 poses a barrier to understanding mental health outcomes. We sought to summarize results from studies that compared mental health outcomes during COVID-19 to outcomes assessed prior to COVID-19 in the same cohort in the general population and in other groups for which data have been reported.

Design

Living systematic review.

Data Sources

MEDLINE (Ovid), PsycINFO (Ovid), CINAHL (EBSCO), EMBASE (Ovid), Web of Science Core Collection: Citation Indexes, China National Knowledge Infrastructure, Wanfang, medRxiv (preprints), and Open Science Framework Preprints (preprint server aggregator).

Eligibility criteria for selecting studies

For this report, we included studies that compared general mental health, anxiety symptoms, or depression symptoms, assessed January 1, 2020 or later, to the same outcomes collected between January 1, 2018 and December 31, 2019. Any population was eligible. We required ≥ 90% of participants pre-COVID-19 and during COVID-19 to be the same or the use of statistical methods to address missing data. For population groups with continuous outcomes for at least two studies in an outcome domain, we conducted restricted maximum-likelihood random-effects meta-analyses. Worse COVID-19 mental health outcomes are reported as positive. Risk of bias of included studies was assessed using an adapted version of the Joanna Briggs Institute Checklist for Prevalence Studies.

Results

As of April 11, 2022, we had reviewed 94,411 unique titles and abstracts and identified 137 unique eligible studies with data from 134 cohorts. Almost all studies were from high-income (105, 77%) or upper-middle income (28, 20%) countries. Among adult general population studies, we did not find changes in general mental health (standardized mean difference of change [SMD change = 0.11, 95% CI -0.00 to 0.22) or anxiety symptoms (SMD change = 0.05, 95% CI -0.04 to 0.13), but depression symptoms worsened minimally (SMD change = 0.12, 95% CI 0.01 to 0.24). Among women or females, mental health symptoms worsened by minimal to small amounts in general mental health (SMD change = 0.22, 95% CI 0.08 to 0.35), anxiety symptoms (SMD change = 0.20, 95% CI 0.12 to 0.29), and depression symptoms (SMD change = 0.22, 95% CI 0.05 to 0.40). Of 27 other analyses across outcome domains, among subgroups other than women or females, 5 analyses suggested minimal or small amounts of symptom worsening, and 2 suggested minimal or small symptom improvements. No other subgroup experienced statistically significant changes across outcome domains. In the 3 studies with data from March to April 2020 and later in 2020, symptoms either were unchanged from pre-COVID-19 at both time points or increased initially then returned to pre-COVID-19 levels. Heterogeneity measured by the I 2 statistic was high (e.g., > 80%) for most analyses, and there was concerning risk of bias in most studies.

Conclusions

High risk of bias in many studies and substantial heterogeneity suggest that point estimates should be interpreted cautiously. Nonetheless, there was general consistency across analyses in that most symptom change estimates were close to zero and not statistically significant, and changes that were identified were of minimal to small magnitudes. There were, however, small negative changes for women or females in all domains. It is possible that gaps in data have not allowed identification of changes in some vulnerable groups. Continued updating is needed as evidence accrues.

Funding: Canadian Institutes of Health Research (CMS-171703; MS1-173070; GA4-177758; WI2-179944); McGill Interdisciplinary Initiative in Infection and Immunity Emergency COVID-19 Research Fund (R2-42).

Registration: PROSPERO (CRD42020179703); registered on April 17, 2020.

Article activity feed

  1. Version published to 10.1101/2021.05.10.21256920v2 on medRxiv
  2. ScreenIT

    SciScore for 10.1101/2021.05.10.21256920: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variablenot detected.
    RandomizationTwo independent reviewers evaluated titles and abstracts in random order.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    We searched MEDLINE (
    MEDLINE
    suggested: (MEDLINE, RRID:SCR_002185)
    Ovid), PsycINFO (Ovid), CINAHL (EBSCO), EMBASE (Ovid), Web of Science Core Collection: Citation Indexes, China National Knowledge Infrastructure, Wanfang, medRxiv (preprints), and Open Science Framework Preprints
    PsycINFO
    suggested: (PsycINFO, RRID:SCR_014799)
    EMBASE
    suggested: (EMBASE, RRID:SCR_001650)
    , RStudio Version 1.2.5042), using the rma.uni function in the metafor package.
    RStudio
    suggested: (RStudio, RRID:SCR_000432)

    Results from OddPub: Thank you for sharing your data.

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Strengths and Limitations: Strengths of our systematic review include using rigorous best-practice methods; searching 9 databases, including 2 Chinese databases; not restricting inclusion by language; and the ability to update rapidly as evidence emerges via our living systematic review approach. There are also limitations that suggest some level of caution in interpreting results. First, aside from several population-level randomly sampled surveys, most of the studies included in our systematic review had limitations related to study sampling frames and recruitment methods, response and follow-up rates, and management of missing follow-up data. Second, heterogeneity was high in most of the meta-analyses that we conducted. Third, only a handful of studies reported results from the fall months of 2020, and, although the few studies that did suggested that symptoms were stable or reduced from earlier in the pandemic, more data are needed. Fourth, although we were able to synthesize results from several vulnerable groups, including older adults and people with pre-existing medical conditions, there were few studies with results for other vulnerable groups. It is possible that some groups may be experiencing important negative mental health effects of the pandemic and were not included in the studies we identified. Fifth, some potentially important outcomes, such as loneliness, were infrequently studied and not included in the present report. Sixth, the evidence base is rapidly e...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.05.10.21256920v1 on medRxiv