Image classification using machine learning for digital forensic investigations

Abstract

The exponential increase in the production of manipulated digital media, along with the sheer abundance of digital forensic evidence creates a serious problem to the task of a law enforcement agency and practitioners of digital forensics. Available image classification forensics tools that do exist are largely commercial and narrow in scope, thus limiting the viability of wide usage in a forensics application. The following dissertation will fill these gaps by proposing an open source and machine learning driven image classification system to classify images related to forensics, with particular sensitivity to weapons and narcotics, which are common categories of cybercrime. The ResNet-50 convolutional neural network model was trained using transfer learning based on more than 47,000 labelled images and optimised to obtain the highest accuracy and robustness rates. The system is designed to include the use of explainable artificial intelligence (XAI) methods (such as Grad-CAM visualisations) to provide improved transparency, interpretability, and possible admissibility in courts of law. Statistical performance assessment, test of robustness under the degradations to occur in practice and usability testing by practitioners were carried out. The benchmark comparison with the state-of-the-arts studies reveals the proposed solution to outperform the traditional support vector machine and Bag-of-Visual-Words approaches in addition to solving the problem of scalability and explainability. The novelty of the presented dissertation lies in the fact that multi-label forensic classification, open-source applicability, and XAI have already been integrated into one framework, and a transparent, reproducible, and practical contribution to the digital forensics field has been introduced. The limitations in terms of representativeness of the databases and resistance to adversarial attacks are discussed, and the future research of federated learning, adversarial robustness, and combination to forensic triage systems are suggested.