False-Positive Calls of Reimplemented Published Analyses for Covert Command-Following on Electroencephalograms Recorded Without Study Commands

Read the full article See related articles

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

Objective

To measure how often published electroencephalographic analyses for covert command-following call patients positive without commands, and their detection of genuine responses.

Methods

This cross-sectional study applied 11 reimplemented published analyses to electroencephalograms without commands: 974 clinical electroencephalography patients and, with rules fixed, 730 new patients; 38 with prolonged disorders of consciousness; and 544 comatose patients after cardiac arrest. Outcomes: false calls (share called positive) against a 5% benchmark, and genuine-response detection.

Results

Three of 10 command-following analyses called significantly more than 5% of clinical electroencephalography patients positive, replicated in new patients: Goldfine 2011, a spectral test, 23.3% (95% CI, 20.2%-26.6%), above its stated 5%; du Bois 2026, a criterion for entry to brain-computer interface training, 22.2% (95% CI, 19.2%-25.5%); and Guger 2018, a P300 accuracy threshold, 13.8% (95% CI, 11.5%-16.3%), near its approximate 12% chance rate (24.5% over 2 runs). The other 7 called 0.0% to 6.9%. After cardiac arrest: 11.5% to 27.3%. All 3 detected genuine responses. Identified contributors: a variance ignoring within-command dependence (Goldfine 2011), selection over configurations and times (du Bois 2026) and a permissive threshold (Guger 2018). Goldfine 2011’s command-level correction, fixed before testing, called 0.8% of new patients and 2.4% after arrest, detecting 93.4% of stroke imagery-versus-rest sessions but few between imagery types.

Interpretation

Reimplemented published analyses called up to 27% of patients positive without commands, more with repeated runs. A positive result needs its complete call rule’s false-call rate and detection at that rate; imagery-versus-rest detection may reflect task engagement, not imagery.

Article activity feed