An evaluation of the accuracy of smartphone apps that claim to risk-stratify moles and skin lesions has found that they are poorly regulated and often cannot be trusted to produce accurate results.
Looking at a total of nine studies evaluating six apps, researchers at the University of Birmingham and the University of Nottingham found that methodological decisions made by instigators of the studies probably led to overestimation of the apps’ real-world performance.
The systematic review has been published in the British Medical Journal.
The studies evaluating the efficacy of the apps used images taken by experts rather than at-home app users. In addition, many studies did not identify whether lesions identified as ‘low risk’ by the apps were in fact benign, further compromising the conclusions that can be drawn from the evaluations.
Recommendations made by the apps were commonly inconsistent with clinical histopathological results, and some apps gave conflicting management advice for the same lesions.
Several companies have made headlines claiming their AI products can outperform clinicians when diagnosing skin cancer. The apps are designed to use specialised imaging algorithms to detect possible skin cancers.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataApps with medical leanings are ‘class I’ devices in the EU, meaning they’re judged to be a low-risk technology and can receive CE marking more easily than higher-risk software and devices.
Institute of Applied Health Research professor of biostatistics and study co-author Jon Deeks said: “Regulators need to become alert to the potential harm that poorly performing algorithm-based diagnostic or risk monitoring apps create. We rely on the CE mark as a sign of quality, but the current CE mark assessment processes are not fit for protecting the public against the risks that these apps present.”
Of the six apps evaluated, only one – SkinVision – is currently on the market. MelApp and Mole Detective were withdrawn from the market after US Federal Trade Commission investigations, while Dr Mole and Spotmole no longer appear to be available. An early version of SkinVision called SkinScan was also evaluated, which is unrelated to the skinScan app produced by TeleSkin. Two of the efficacy studies assessed one and three apps respectively without disclosing their names.
SkinVision was found to have a sensitivity of 80% and a specificity of 78% for the detection of malignant of premalignant lesions but was poorly validated against expert recommendations. The app currently has a CE mark and can operate in Europe as a medical device.
A statement from SkinVision says: “SkinVision aims to raise awareness of skin cancer and provide a regulated medical device to help individuals assess their risk and get to the doctor in time. The latest research proves that our algorithm can detect 95% of cases of skin cancer. SkinVision has assisted in finding over 40,000 cases of skin cancer already. These objective facts prove that the clinical benefits of the service outweigh the risks.”
