Assessing the quality, readability and reliability of online information on COVID-19

  1. Paulo Cardoso Lins-Filho
  2. Millena Mirella Silva de Araújo
  3. Thuanny Silva de Macêdo
  4. Andressa Kelly Alves Ferreira
  5. Maria Cecilia Freire de Melo
  6. Jaciel Leandro de Melo Freitas
  7. Elizabeth Louisy Marques Soares da Silva
  8. Thaise Urbano Caldas
  9. Gustavo Pina Godoy
  10. Arnaldo de França Caldas Jr.

Reviewed by

Read the full article

Listed in

Log in to save this article

Abstract

During a public health crisis, the dissemination of reliable information, advice and experts’ opinions is essential for improving public comprehension of potential or actual health threats and enables the public to take informed decisions about risk mitigation measures. This study aimed to assess the quality, reliability and readability of internet-based information on COVID-19 available on Brazil’ most used search engines. A total of 68 websites were selected through Google, Bing, and Yahoo. The websites content quality and reliability were evaluated using the DISCERN questionnaire, the Journal of American Medical Association (JAMA) criteria, and the presence of the Health on Net (HON) certification. Readability was assessed by the Flesch Reading Ease adapted to Brazilian Portuguese (FRE-BP). The web contents were considered moderate to low quality according to DISCERN and JAMA mean scores. Most of the sample presented very difficult reading levels and only 7.4% displayed HON certification. Websites of Governmental and health-related authorship nature showed lower JAMA mean scores and quality and readability measures did not correlate to the webpages content type. COVID-19 related contents available online were considered of low to moderate quality and not accessible to general population. These findings indicate the need for further efforts on improving the quality of health-related content on internet, especially during public health emergencies.

Article activity feed

  1. Version published to 10.33448/rsd-v9i12.10680
  2. ScreenIT

    SciScore for 10.1101/2020.05.30.20117614: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Sites on the internet were identified using the three most accessed search engines by internet users in Brazil: Google www.google.com), Bing (www.bing.com) and Yahoo www.yahoo.com), respectively, 97.59%, 1.2% and 1.04% of accesses in April 2020 (Statcounter 2020).
    Google
    suggested: (Google, RRID:SCR_017097)
    Yahoo
    suggested: None
    Data were submitted to statistical analysis, all tests were applied considering an error of 5% and the confidence interval of 95%, and the analyzes were carried out using SPSS software version 23.0 (SPSS Inc. Chicago, IL, USA).
    SPSS
    suggested: (SPSS, RRID:SCR_002865)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    This was a limitation for this investigation. However, the search engines used for the consultation represent 99.8% of the access of Brazilian internet users (Statcounter 2020). In addition, to cover a reasonable amount of data, the first 100 consecutive websites of each search engine were accessed. Regarding the present sample of Brazilian websites, COVID-19 contents were considered of low to moderate quality and low readability based on the parameters adopted. This pattern just reasonably correlated with the nature of websites’ authorship. These findings indicate the need for further efforts on improving the quality of health-related content on internet. Health authorities might apply this evidence to measure the effect of the transmission of information on the population and define better risk communication strategies.

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.05.30.20117614v1 on medRxiv