Big Data-Aware Intrusion Detection System in Communication Networks: a Deep Learning Approach

One of the most important parameters that hackers have always considered is obtaining information about the status of computer networks, such as hacking into databases and computer networks used in defense systems. Hence, these networks are always exposed to dangerous attacks. On the other hand, networks and hosts face a large amount of data every second. Hence, intrusion detection mechanisms have to mine this growing mountain of data for possible intrusive patterns from the security perspective. This environment and conditions make it hard to detect intrusions fast and accurately. Therefore, to identify such intrusions, it is necessary to design an intrusion detection system using big data techniques that can handle these types of data that have big data nature in detecting unauthorized access to a communication network. Therefore, this article employs a big data-aware deep learning method to design an efficient and effective Intrusion Detection System (IDS) to cope with these challenges. We designed a specific architecture of Long Short-Term Memory (LSTM), and this model can detect complex relationships and long-term dependencies between incoming traffic packets. Through this way, we could reduce the number of false alarms and increase the accuracy of the designed intrusion detection system. Moreover, using big data analytic techniques can improve the speed of deep learning algorithms in this paper, which have low execution speed due to their high complexity. Actually, using these techniques increases the speed of execution of our complex model. Our extensive experiments are on the BigDL directly on top of the Spark framework and train with the NSL-KDD dataset. Results show that the proposed algorithm, called BDL-IDS, outperforms other IDS schemes, such as traditional machine learning and Artificial Neural Network, in terms of detection rate (20%), false alarm rate (60%), accuracy (15%), and training time (70%).

Automated summary

This article presents an efficient and effective Intrusion Detection System (IDS) using big data-aware deep learning method. By designing a specific architecture of Long Short-Term Memory (LSTM), the system can detect complex relationships and long-term dependencies in incoming traffic packets, reducing false alarms and increasing accuracy. Utilizing big data analytic techniques improves the speed of deep learning algorithms for detecting unauthorized access to communication networks.