Distributed Abnormal Behavior Detection Approach Based on Deep Belief Network and Ensemble SVM Using Spark

The emergence of Internet connectivity has led to a significant increase in the volume and complexity of cyber attacks. Abnormal behavior detection systems are valuable tools for ensuring the security in computer networks. However, due to the huge amount and ever increasing diversity of the intrusions, the existing intrusion detection systems, which use machine learning techniques to learn a classifier based on a handcrafted feature vector, are not robust enough to detect sophisticated attacks which cause a high false alarm rate. Therefore, building a flexible in-depth defense system to detect abnormal behavior requires an ability to automatically learn powerful features and analyze large amounts of network traffic. To address these concerns, this paper proposes a novel distributed approach for the detection of abnormal behavior in large-scale networks. The developed model discovers the abnormal behavior from large-scale network traffic data using a combination of a deep feature extraction and multi-layer ensemble support vector machines (SVMs) in a distributed way. First, we perform a non-linear dimensionality reduction, achieved through a distributed deep belief networks on large-scale network traffic data. Then, the obtained features are fed to the multi-layer ensemble SVM. The construction of the ensemble is accomplished through the iterative reduce paradigm based on Spark. Empirical results show a promising gain in performance compared with other existing models.