Analysis of the factors affecting the adoption and compliance of the NHS COVID-19 mobile application: a national cross-sectional survey in England
This article has been Reviewed by the following groups
Listed in
- Evaluated articles (ScreenIT)
Abstract
To conduct an independent study investigating how adults perceive the usability and functionality of the ‘National Health Service (NHS) COVID-19’ application (app). This study aims to highlight strengths and provide recommendations to improve adoption of future contact tracing developments.
Design
A 60-item, anonymous online questionnaire, disseminated through social media outlets and email lists by a team from Imperial College London.
Setting
England.
Participants
Convenience sample of 1036 responses, from participants aged 18 years and above, between December 2020 and January 2021.
Primary outcome measures
Evaluate the compliance and public attitude towards the ‘NHS COVID-19’ app regarding its functionality and features. This included whether participants’ expectations were met, and their thoughts on the app privacy and security. Furthermore, to distinguish how usability, perception, and adoption differed with varying demographics and user values.
Results
Fair compliance with the app features was identified, meeting expectations of the 62.1% of participants who stated they downloaded it after weighted analysis. However, participants finding the interface challenging were less likely to read information in the app and had a lesser understanding of its functionality. Furthermore, little understanding regarding the app’s functionality and privacy concerns was a possible reason why users did not download it. A readability analysis of the text revealed information within the app was conveyed at a level that may be too complex for up to 43% of the UK population. The study highlighted issues related to the potential of false positives caused by the design choices in the ‘Check-In’ feature.
Conclusion
This study showed that while the ‘NHS COVID-19’ app was viewed positively, there remained issues regarding participants’ perceived knowledge of app functionality, potentially affecting compliance. Therefore, we recommended improvements regarding the delivery and presentation of the app’s information, and highlighted the potential need for the ability to check out of venues to reduce the number of false positive contacts.
Article activity feed
-
-
-
SciScore for 10.1101/2021.03.04.21252924: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement IRB: These participants were instead directed to a different set of questions that focused on trying to find out the reasons why they did not download it, and if any suggested changes would have changed their mind. 2.3. Survey Deployment: The study received ethical approval by the Science Engineering Technology Research Ethics Committee of Imperial College London (SETREC ref.:
Consent: Survey respondents needed to consent to take part in the study, and were required to be above the age of 18. 2.4.Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Res… SciScore for 10.1101/2021.03.04.21252924: (What is this?)
Please note, not all rigor criteria are appropriate for all manuscripts.
Table 1: Rigor
Institutional Review Board Statement IRB: These participants were instead directed to a different set of questions that focused on trying to find out the reasons why they did not download it, and if any suggested changes would have changed their mind. 2.3. Survey Deployment: The study received ethical approval by the Science Engineering Technology Research Ethics Committee of Imperial College London (SETREC ref.:
Consent: Survey respondents needed to consent to take part in the study, and were required to be above the age of 18. 2.4.Randomization not detected. Blinding not detected. Power Analysis not detected. Sex as a biological variable not detected. Table 2: Resources
Software and Algorithms Sentences Resources Automatic proximity based contact tracing is performed using an API developed jointly by Apple and Google, which requires each mobile device to constantly transmit a unique randomised identifying code (which is changed over a period of time) using the Bluetooth Low Energy Protocol. Googlesuggested: (Google, RRID:SCR_017097)The analyses were carried out using the Python programming language with the commonly used Pandas and SciPy Python packages. 2.5. Textual Analysis: The Flesch Reading Ease test was used to assess the overall readability of the explanatory body text on the on-boarding screens, and the “About” section of the app. Pythonsuggested: (IPython, RRID:SCR_001658)SciPysuggested: (SciPy, RRID:SCR_008058)Results from OddPub: We did not detect open data. We also did not detect open code. Researchers are encouraged to share open data when possible (see Nature blog).
Results from LimitationRecognizer: We detected the following sentences addressing limitations in the study:Study Limitations: As our work endeavored to investigate the compliance and usage of the main features and tools contained within the app, we ideally required the survey to be active at the same time as users were able to use the app outside their home. However due to the rapidly changing nature of the COVID-19 pandemic, not long after the “NHS COVID-19” app was released, the UK was placed into a 2nd and 3rd lockdown period with restrictions on movement and outside activity. We therefore tried to mitigate the effect of this on the results of our study by specifically referring users to how they would have used the app in the period between the first and second lockdown in all compliance questions involving the venue “Check-In” feature, which understandably was only able to be used during this time period. Our study also specifically focused on the “NHS COVID-19” app rolled out in England and Wales. Hence, not all of our conclusions might be applicable to other contact tracing apps rolled out around the world, not just because of their potentially different designs but also due to the different demographic characteristics of their particular geography. Despite our statistical calculation of an appropriate sample size required for the study, the study failed to attract an adequate cohort of participants from Wales. We also noticed that, as our study was advertised as an “Usability and Compliance” study, we received a significantly higher proportion of responses from people who ...
Results from TrialIdentifier: No clinical trial numbers were referenced.
Results from Barzooka: We did not find any issues relating to the usage of bar graphs.
Results from JetFighter: We did not find any issues relating to colormaps.
Results from rtransparent:- Thank you for including a conflict of interest statement. Authors are encouraged to include this statement when submitting to a journal.
- Thank you for including a funding statement. Authors are encouraged to include this statement when submitting to a journal.
- No protocol registration statement was detected.
-