Detection of the Advanced Persistent Threat Attack Using Traffic Analysis and Machine Learning Technique

Abstract

The prevalence of Information and Communication (ICT) technology has the major hurdle of cyber threats that appear in the form of viruses, worms, trojans, and malware. These threats hamper the performance of systems and result in financial and strategic losses for organizations and countries. Advanced Persistent Threat (APT) is a major threat which is differentiated from normal cyber threats due to its specially crafted design, capability to maneuver silently and keep internal communication confidential using encryption. The affected organizations remain reluctant to release their log data and audit reports because of the apprehension of revealing the internal physical system layout. All these facts combined make the detection of APTs a challenging task. This research proposes a two-step APT detection model based on an unsupervised machine-learning technique of clustering and the Markov Stat-transition model. The first step is the detection of abnormal activity in the data. The second step is the verification of the attack using a state-condition model. The first step uses clustering to identify data values falling outside the clusters. The second step, the verification step, uses a state-transition model that verifies the abnormal behavior by identifying the system in an unknown state. This verification is counter-checked by the statistical analysis results. The NSL-KDD data set is used for the detection of four different attacks. The performance of supervised machine learning classifiers is also evaluated for prediction accuracy. The performance of these classifiers is based on the accuracy of data labeling. Decision tree and KNN classifiers provided the best accuracy values. It is found that the proposed two-step detection approach identifies unknown APT attacks with 93.75% accuracy. The solution can be used as a generic APT detection system that can be applied to different scenarios