How to Demonstrate the Glucose Specificity of a Non-Invasive CGM: A Case Study of the SKAMo-2 Clinical Trial and Neogly™
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Every non-invasive continuous glucose monitoring (NI-CGM) technology introduced into the landscape faces the same skeptical question, from regulators, clinicians, and competing developers alike: is the candidate signal actually specific to glucose, or does an apparently reasonable accuracy figure simply reflect a model fitting to motion, temperature, calibration offset, or trial-duration artifact? Existing evaluation practice does not answer this question directly. NI-CGM performance is instead reported almost exclusively with metrics inherited from minimally invasive, subcutaneous CGM, the Mean Absolute Relative Difference (MARD), Clarke/Parkes error grids, and ISO 15197-style agreement rates, which were designed for sensors whose glucose specificity is already chemically established and which therefore take specificity as a premise rather than treating it as a result to be demonstrated.
Methods
We present a methodology for demonstrating NI-CGM technology glucose specificity during the algorithm-development phase, and illustrate it with a case study based on a quantum-cascade-laser (QCL) photoacoustic NI-CGM device (Neogly™) evaluated in the SKAMo-2 free-living clinical trial (eight participants with type 1 diabetes). The methodology combines a white-noise control, a constant-glycemia control, a sensor-ablation control that removes the candidate physical signal while retaining auxiliary covariates, and explicit reporting of the train/test generalization level, so that a reported MARD can be read as evidence of specificity rather than taken on faith.
Results
Removing the mid-infrared photoacoustic (PA) signal from the model while retaining all auxiliary sensors (accelerometer, skin temperature, hygrometry, PPG) degraded performance at every generalization level tested, inter-patient MARD rose from 35.0% with the PA signal to 43.1% without it, and intra-experimentation MARD rose from 22.5% to 23.9%, providing direct, internal evidence that the PA channel itself, and not merely the auxiliary covariates, carries glucose-specific information. At the same time, an algorithm trained on pure Gaussian noise produced a MARD of 25% over short test windows, and a trivial constant-glycemia predictor outperformed every machine-learning model tested when generalization was extended from a single recording to an unseen patient (MARD 55% for the naive constant model versus 37% for a deep neural network on inter-patient splits). Reported in isolation, any of these MARD values is uninterpretable; reported against one another, they jointly demonstrate that the signal is specific to glucose while also bounding how much of the headline accuracy figure that specificity currently explains.
Conclusions
We propose a specificity-demonstration methodology for NI-CGM technology development, comprising (1) signal quality gating prior to any algorithm benchmarking, (2) a white-noise control to test for genuine information content, (3) a constant-glycemia control to expose trial-duration bias, (4) a sensor-ablation control that isolates the contribution of the candidate physical signal from auxiliary covariates, (5) explicit reporting of the data-splitting generalization level (intra-experimentation, intra-patient, inter-patient). This methodology answers a question that precedes clinical accuracy reporting and that recognized clinical frameworks such as the IFCC Working Group on CGM’s Dynamic Glucose Regions guideline are not designed to answer: not how accurate is the device, but is the device measuring glucose at all. We argue that without these controls, MARD and error-grid values for NI-CGM are not comparable across studies and may either overstate clinical readiness or undermine promising technologies. We recommend that this specificity methodology be applied routinely once a candidate NI-CGM sensor reaches algorithm-development stage, alongside and as a deliberate complement to IFCC-style clinical accuracy reporting once the device is mature enough for that evaluation.
