A Filter-Based Malicious Traffic Detection Method Based on Unsupervised Models

To address the current challenges in malicious traffic detection—namely the low accuracy of single unsupervised models and the strong dependence on labeled data in supervised approaches—this paper proposes a filtering-based malicious traffic detection method leveraging multiple unsupervised models. The proposed framework first extracts normal traffic samples from the dataset and employs a Bidirectional Long Short-Term Memory (BiLSTM) network to capture their temporal characteristics, producing representation vectors for training an autoencoder. The trained autoencoder is then used to perform the first round of filtering on the raw traffic data. Subsequently, a Local Outlier Factor (LOF) algorithm and an attention-enhanced autoencoder are introduced to further refine the suspicious traffic identified in the first stage through two additional rounds of filtering. Ultimately, all samples identified as normal during any stage are combined with the malicious samples detected in the final round to achieve complete classification and labeling of the original dataset. Experimental results on publicly available attack datasets demonstrate that the proposed method outperforms existing state-of-the-art models. It requires no manual data annotation, enables multi-dimensional detection from temporal, local density, and global dependency perspectives, and effectively captures various obfuscation strategies used in malicious traffic. The method achieves detection accuracy and precision rates exceeding 99%.