Novel Framework for Multimodal Classification of Consumer Preference using EEG signals and eye-tracking data

Marketing plays a crucial role in enhancing consumer awareness and affinity towards a product. Consumer research examines the motivations, trends, and patterns underlying purchasing behavior, thereby enabling institutions to refine their offerings and marketing strategies to improve their alignment with consumer decision-making and increase purchase probability. Consumer neuroscience aims to study the neurological factors influencing consumer preference, which cannot be determined using traditional methods like questionnaires and self-assessment. Machine learning techniques have proven effective in this domain, yet deep learning architectures that effectively fuse electroencephalography (EEG) signals with eye-tracking (ET) data remain largely unexplored. In this study, we propose a novel hybrid "Wide & Deep" framework for multimodal classification of consumerpreference using EEG and eye-tracking data from the Neuma dataset. The proposed architecture employs an InceptionTime network with early fusion to process concatenated multimodal input (25 channels), extracting multi-scale temporal featuresthrough parallel convolutions with varying kernel sizes (10, 20, 40). The learned 128-dimensional neural embeddings are combined with 44 time-domain statistical features (temporal mean and standard deviation across all channels) and passed to a CatBoost gradient boosting classifier, forming a neural-symbolic ensemble. A modality dropout strategy (30% probability) is introduced during training to prevent over-reliance on eye-tracking signals and force robust EEG representation learning. The framework employs a grokking-oriented training regime with extended epochs (100), high weight decay (0.02), and strict checkpointing based on validation loss. The model is evaluated using 5-fold stratified cross-validation to assess generalizability. The proposed framework achieves a mean accuracy of 77.51% (±0.49%) and a weighted F1-score of 74.60% (±0.21%) in classifying consumer preferences, representing a significant improvement over prior methods on this dataset. This study demonstrates the effectiveness of multi-scale convolutional architectures combined with gradient boosting ensemble methodsfor multimodal consumer neuroscience applications.