Mitigating Metamorphic Malware Through Adversarial Learning Techniques

Antivirus (AV) engines provide a standard security measure to combat malicious attacks. However, most engines fail to generalise over metamorphic malware that changes its form over time in unpredictable ways. To combat this, we proposed an adversarial solution which generates a suite of new malicious mutants of existing malware that are undetectable by many existing AV engines and can therefore be used to develop new countermeasures. Specifically, we utilised an Evolutionary Algorithm (EA) to discover un-detectable mutants, guided by three different fitness measures: the evasiveness of the variants, and their behavioural and structural similarity to the original malware. We conducted experiments over three classes of malware to evaluate the effectiveness of the EA variants. Results show that the EA can generate a diverse set of mutants that evade detection by a significantly higher proportion of AV detectors than the original malware. Furthermore, we investigated whether the mutants created using the EA can be used to train better machine learning models. This includes the use of a pretrained Natural Language Processing (NLP) model in a transfer learning setting to show improved classification of metamorphic malware using the evolved variants as part of the training data. Our findings underscore the potential of utilizing diverse variants in training data to bolster the classification of metamorphic malware, thereby enhancing the resilience of security systems against evolving threats.