Online information on face masks: analysis of websites in Italian and English returned by different search engines

  1. Shaily Mehta
  2. Daria Ghezzi
  3. Alessia Catalani
  4. Tania Vanzolini
  5. Pietro Ghezzi

Reviewed by

Read the full article See related articles

Listed in

Log in to save this article

Abstract

Countries have major differences in the acceptance of face mask use for the prevention of COVID-19. This work aims at studying the information online in different countries in terms of information quality and content.

Design

Content analysis.

Method

We analysed 450 webpages returned by searching the string ‘are face masks dangerous’ in Italy, the UK and the USA using three search engines (Bing, Duckduckgo and Google) in August 2020. The type of website and the stance about masks were assessed by two raters for each language and inter-rater agreement reported as Cohen’s kappa. The text of the webpages was collected from the web using WebBootCaT and analysed using a corpus analysis software to identify issues mentioned.

Results

Most pages were news outlets, and few (2%–6%) from public health agencies. Webpages with a negative stance on masks were more frequent in Italian (28%) than English (19%). Google returned the highest number of mask-positive pages and Duckduckgo the lowest. Google also returned the lowest number of pages mentioning conspiracy theories and Duckduckgo the highest. Webpages in Italian scored lower than those in English in transparency (reporting authors, their credentials and backing the information with references). When issues about the use of face masks were analysed, mask effectiveness was the most discussed followed by hypercapnia (accumulation of carbon dioxide), contraindication in respiratory disease and hypoxia, with issues related to their contraindications in mental health conditions and disability mentioned by very few pages.

Conclusions

This study suggests that: (1) public health agencies should increase their web presence in providing correct information on face masks; (2) search engines should improve the information quality criteria in their ranking; (3) the public should be more informed on issues related to the use of masks and disabilities, mental health and stigma arising for those people who cannot wear masks.

Article activity feed

  1. Version published to 10.1136/bmjopen-2020-046364
  2. ScreenIT

    SciScore for 10.1101/2020.10.23.20218271: (What is this?)

    Please note, not all rigor criteria are appropriate for all manuscripts.

    Table 1: Rigor

    Institutional Review Board Statementnot detected.
    Randomizationnot detected.
    Blindingnot detected.
    Power Analysisnot detected.
    Sex as a biological variablenot detected.

    Table 2: Resources

    Software and Algorithms
    SentencesResources
    Search strategy: The search string “are face masks dangerous” (with no question mark) was decided as it was amongst the top suggestions by Google when typing “are face mask”.
    Google
    suggested: (Google, RRID:SCR_017097)
    All the webpages were archived on the Internet Archive (https://archive.org/web/) on 31 August 2020.
    Internet Archive
    suggested: (Internet Archive, RRID:SCR_001682)

    Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).

    Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:
    Finally, it is important to note the limitations of the study and its methodology. An obvious limitation is the use of a search string that focused on dangers associated with masks. While this was chosen to address the research question of studying information that could promote mask hesitancy, it clearly resulted in returning more mask-negative information that that present in the whole Web. Also, a limitation is the use of search engines to obtain a sample of the infosphere. Because search engines aim to return high information quality webpages, these could have not returned the lower quality webpages. On the other hand, there are no means of obtaining a random, non-ranked, sample of websites without using search engines, that all have proprietary, non-disclosed, ranking criteria. Also the comparison of two different languages that we attempted might be influenced by cultural differences in the social acceptance of masks and different ranking by the search engines. It should not be forgotten that websites are not the only form of information online. Studies have shown that one quarter of YouTube videos spread misinformation on COVID-19. Because search engines return very few videos, we did not have a sufficient number to try to analyze them separately. Studies searching video sources and social networks for information specifically on face masks should also be done in order to have a more comprehensive picture of the information available online. Social networks, in particu...

    Results from TrialIdentifier: No clinical trial numbers were referenced.

    Results from Barzooka: We did not find any issues relating to the usage of bar graphs.

    Results from JetFighter: We did not find any issues relating to colormaps.

    Results from rtransparent:
    • Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
    • Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
    • No protocol registration statement was detected.

    About SciScore

    SciScore is an automated tool that is designed to assist expert reviewers by finding and presenting formulaic information scattered throughout a paper in a standard, easy to digest format. SciScore checks for the presence and correctness of RRIDs (research resource identifiers), and for rigor criteria such as sex and investigator blinding. For details on the theoretical underpinning of rigor criteria and the tools shown here, including references cited, please follow this link.

  3. Version published to 10.1101/2020.10.23.20218271v1 on medRxiv