Insider Threat Detection in a Hybrid IT Environment Using Unsupervised Anomaly Detection Techniques

Abstract

This dissertation analyses insider threat detection in hybrid IT environments with unsupervised anomaly detection techniques. Insider threats, including those committed by trusted persons with granted access, are considered to be one of the most challenging to alleviate cybersecurity threats because they resemble legal user behavior and do not have labelled datasets to train supervised models. Hybrid infrastructures, an integration of on-premise and cloud resources, also make detection harder as they create large, heterogeneous and fragmented logs. In order to cope with such challenges, this paper presents a detection system that uses isolation forest and local outlier factor algorithms. Multi-source organisational data, such as authentication, file, email, HTTP, device and LDAP logs, were pre-processed and loaded into enriched user profiles, with psychometric attributes added where possible. The framework was assessed by the CERT Insider Threat Dataset v6.2, where the results indicated that both algorithms were effective in detecting anomalous behaviours: Isolation Forest was effective in detecting global outliers, whereas Local Outlier Factor was good in detecting subtle local outliers. It was found through the comparative analysis that the strength of each method was complementary, and they should be used together when stratifying users into high-, medium-, and low-risk groups. Although it still has constraints in terms of synthetic data, real-time implementation, and ecological validity, the study is relevant in the development of anomaly-based detection methods and offers viable information to organisations wishing to be proactive in curbing insider threats