Detection of Unknown Cyber Attacks Using Convolution Kernels Over Attributed Language Models

Abstract

Over the past decade the Internet has emerged as a key communication platform for businesses and private users demonstrated by an exponential increase of hosts connected to the Internet as well as Internet users. In light of emerging technologies such as Internet of Things, Information Technology (IT) becomes increasingly pervasive in modern society. The growing reliance on IT as well as the ongoing increase of attack sophistication are driving the exposure of organizations to computer and network attacks. A major challenge for state-of-the-art attack detection methods is their inability to reliably detect so called "zero-day" attacks which aim for the exploitation of unknown, and hence unprotected vulnerabilities in computer systems. This work outlines research on self-learning methods that allow for the detection of unknown as well as variants of known computer and network attacks. The effectiveness of the proposed methods is demonstrated in experimental studies presented in this dissertation focusing on two use cases: detection of unknown network attacks and detection of unknown malware. Focal point of the first part of the thesis is the detection of unknown network attacks at the application layer. Based on the results of a comparative analysis of various supervised and unsupervised learning methods applied to the problem of unknown attack detection, a similarity measure is proposed that bridges the gap between network protocol analysis and language model-based anomaly detection to specifically boost the detection of web application attacks. Motivated by the research results, an extension of conventional language models, attributed language models, are proposed which allow to represent data points in a unified vector space that combines syntactic and sequential features. To this end, a novel data representation, ck-grams, is introduced which binds sequential features to network protocol context based on the syntactic analysis of re-assembled network packet payloads. Experimental results suggest that utilization of ck-gram features significantly improves the detection accuracy for web application attacks compared to conventional language model-based features. Moreover, by using ck-grams, unknown application vulnerabilities can be identified based on the contribution of suspicious features associated with a particular network protocol token to the overall norm of the vector space representation of assembled malicious application layer messages. Motivated by the ongoing proliferation of sophisticated malware, the second part of the thesis outlines research on self-learning methods to detect unknown and variants of known malware instances on computer systems based on the classification of malware families over structured behavioral features.