A systematic review of observational methods used to quantify personal protective behaviours among members of the public during the COVID-19 pandemic, and the concordance between observational and self-report measures in infectious disease health protection

  1. Rachel Davies
  2. Fiona Mowbray
  3. Alex F. Martin
  4. Louise E. Smith
  5. G. James Rubin

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Objectives

To assess the quantity and quality of studies using an observational measure of behaviour during the COVID-19 pandemic, and to narratively describe the association between self-report and observational data for behaviours relevant to controlling an infectious disease outbreak.

Design

Systematic review and narrative synthesis of observational studies.

Data sources

We searched Medline, Embase, PsychInfo, Publons, Scopus and the UK Health Security Agency behavioural science LitRep database from inception to 17th September 2021 for relevant studies.

Study selection

We included studies which collected observational data of at least one of three health protective behaviours (hand hygiene, face covering use and maintaining physical distance from others (‘social distancing’) during the COVID-19 pandemic. Studies where observational data were compared to self-report data in relation to any infectious disease were also included.

Data extraction and synthesis

We evaluated the quality of studies using the NIH quality assessment scale for observational studies, extracted data on sample size, setting and adherence to health protective behaviours, and synthesized results narratively.

Results

Of 27,279 published papers on COVID-19 relevant health protective behaviours that included one or more terms relating to hand hygiene, face covering and social distancing, we identified 48 studies that included an objective observational measure. Of these, 35 assessed face covering use, 17 assessed hand hygiene behaviour and seven assessed physical distancing. The general quality of these studies was good. When expanding the search to all infectious diseases, we included 21 studies that compared observational versus self-report data. These almost exclusively studied hand hygiene. The difference in outcomes was striking, with self-report over-estimating observed adherence by up to a factor of five in some settings. In only four papers did self-report match observational data in any domains.

Conclusions

Despite their importance in controlling the pandemic, we found remarkably few studies assessing protective behaviours by observation, rather than self-report, though these studies tended to be of reasonably good quality. Observed adherence tends to be substantially lower than estimates obtained via self-report. Accurate assessment of levels of personal protective behaviour, and evaluation of interventions to increase this, would benefit from the use of observational methods.

Article activity feed

  1. Version published to 10.1186/s12889-022-13819-0
  2. ScreenIT

    SciScore for 10.1101/2021.12.22.21268226: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Ethicsnot detected.
    Sex as a biological variablenot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    The study protocol is available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021261360 Search strategies: For aims one and two, we searched the following electronic databases from inception to 17th September 2021: Medline, Embase, PsycInfo, Publons, Scopus, and the Public Health England (
    Medline
    suggested: (MEDLINE, RRID:SCR_002185)
    PsycInfo
    suggested: (PsycINFO, RRID:SCR_014799)
    Publons
    suggested: (Publons, RRID:SCR_014020)
    Databases were searched for articles containing MeSH terms or keywords relating to COVID-19 (e.g. “SARS-CoV-2”, “novel coronavirus”), hand hygiene, physical distancing, or face coverings (e.g. “hand washing”, “face mask”, “physical distancing”) and an observational method (e.g. “observational study”, “videorecording”).
    MeSH
    suggested: (MeSH, RRID:SCR_004750)
    For aim three, we searched Medline, Embase and PsycInfo from inception to 17th September 2021.
    Embase
    suggested: (EMBASE, RRID:SCR_001650)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Limitations: Several limitations should be considered for this systematic review. First, our conclusions are limited by the availability of data in the literature. The relative absence of observational data relating to face covering wearing or physical distancing is an important result in its own right, but also limits our ability to assess the adequacy of self-report for these behaviours. Second, while we made efforts to search widely for relevant studies, including in COVID-19 specific databases, it is possible that we missed some studies which used terminology relating to an observational method that we did not include in our search. Given the rapidity with which the COVID-19 literature has expanded, with approximately a quarter of a million papers appearing in Scopus alone in less than two years, it is likely that additional studies will have been added to the databases that we searched in the time taken between completing our search and publication of this paper. In this review, we have not attempted to pool the rates of behaviour observed in the various studies. The differing contexts in the studies we included means that any pooled estimate would not be meaningful. For example, it is probably not useful to compare rates of observed hand hygiene among healthcare workers working on COVID-19 wards 26 with those among high school students attending their graduations49. Conclusions: The COVID-19 pandemic witnessed an explosion in research covering every aspect of the crisis...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    Results from scite Reference Check: We found no unreliable references.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2021.12.22.21268226v1 on medRxiv