The impact of a SmartPhone applicatiOn for skin cancer risk assessmenT on the healthcare system (SPOT-study): A randomized controlled trial

Background

Artificial intelligence (AI)-based mobile health (mHealth) smartphone apps for skin cancer detection are increasingly available to the general population, but their impact on care is unclear.

Methods

The SPOT study is an investigator-initiated and -designed, unblinded, randomized controlled trial. Participants from a Dutch non-profit health insurance living in and around region Rotterdam the Netherlands, were recruited between August and December 2021. Participants were randomly assigned (3:2) to either free access to an AI-based mHealth app for skin cancer risk detection or care-as-usual. The primary endpoint was the difference in healthcare consumption for (pre)malignant and benign skin lesions at 12-months follow-up in the intention-to-treat population. Secondary endpoints included differences in the proportion of surgical interventions, overall use of dermatological care, and costs.

Findings

Among the 19,009 participants, the incidence of claims for (pre)malignant skin lesions was 2·8-fold higher than among non-responders. Within the group of study participants, the skin cancer incidence was higher among the intervention group compared to the control group at 12 months follow-up (2·7% (n=305) vs. 2·3% (n=171); risk difference (RD) 0·4% (95% confidence interval (CI) -0·07 to 0·85), p = 0·10), though this difference was not statistically significant. Furthermore, participants in the intervention group had significantly more claims for benign skin lesions (3·9% (n=443) vs. 2·6% (n=198), RD 1·3 (95% CI 0·7 to 1·7), p < 0·001), underwent more surgical interventions, and had higher mean costs per participant (€63 (95% CI 58 to -67), vs. €47 (41 to -52); p<0·001) compared to controls.

Interpretation

In the first 12 months of this study, access to an AI-based mHealth app for skin cancer risk detection showed a modest trend toward a higher rate of skin cancer detection compared to care-as-usual. However, it also resulted in significantly more dermatological care for benign skin lesions.

Funding

DSW and SkinVision®

Research in context

Evidence before this study

Prior to the start of this study, we conducted a PubMed search for articles published between January 1, 2011, and December 31, 2021, using the search terms artificial intelligence AND skin cancer . This search resulted in 809 articles which were screened for relevance. We also included the results of one prospective validation study for which we had conducted the analyses ourselves, but which had not yet been published at that time. Several commercial companies have implemented such algorithms in smartphone-based mobile health (mHealth) applications, making them available to the general public. A systematic review and meta-analysis reported that AI-based apps assessing skin cancer risk from macroscopic images achieved varying sensitivity and specificity, depending on the algorithm and the type of skin cancers detected. Four studies had prospectively validated the specific mHealth app investigated in this study against histopathology, with reported sensitivities ranging from 57% to 87% and specificities from 27% to 83%. Despite promising indications, no randomised controlled trials have yet evaluated the effectiveness of AI-based mHealth apps for skin cancer screening in the general population.

Added value of this study

The SPOT study is, to our knowledge, the first randomised controlled trial to investigate how implementing an AI-based skin cancer risk detection app in the general population affects skin cancer detection and healthcare use for benign skin lesions. We found a modestly higher skin cancer incidence amongst those who were offered to use the app, though this difference was not statistically significant. However we also found those who were offered to use the app had a significantly larger increase in healthcare visits and procedures for benign skin lesions. Suggesting that implementation in the general population may involve a possible trade-off between increased skin cancer detection and unnecessary care due to overdiagnosis.

Implications of all the available evidence

Even though research in a sterile setting shows potential for implementation of AI-based mHealth apps, the results from this study suggests that nationwide implementation of an mHealth with its current accuracy is not the most optimal strategy. Targeted implementation in higher-risk populations may offer a more favourable balance between benefits and harms.