Filter bank common spatial pattern and envelope-based features in multimodal EEG-fTCD brain-computer interfaces

Brain-computer interfaces (BCIs) exploit brain activity to bypass neuromuscular control with the aim of providing alternative means of communication with the surrounding environment. Such systems can significantly improve the quality of life for patients suffering from severe motor or speech impairment. Multimodal BCIs have been introduced recently to enhance the performance of BCIs utilizing single modality. In this paper, we aim to improve the performance of multimodal BCIs combining Electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD). The BCIs included in the study utilized two different paradigms to infer user intent including motor imagery (MI) and flickering mental rotation (MR)/word generation (WG) paradigms. Filter Bank Common Spatial Pattern (FBCSP) algorithm was used to extract features from the EEG data. Several time series features were extracted from the envelope of the fTCD signals. Wilcoxon rank sum test and linear kernel Support vector machines (SVM) were used for feature selection and classification respectively. Additionally, a probabilistic Bayesian fusion approach was used to fuse the information from EEG and fTCD modalities. Average accuracies of 94.53%, 94.9% and 96.29% were achieved for right arm MI versus baseline, left arm MI versus baseline, and right arm MI versus left arm MI respectively. Whereas average accuracies of 95.27%, 85.93% and 96.97% were achieved for MR versus baseline, WG versus baseline, and MR versus WG respectively. Our results show that EEG- fTCD BCIs with the proposed analysis techniques outperformed the multimodal EEG-fNRIS BCIs in comparison.