ML-CHIEFS: Machine Learning-based Corneal-specular Highlight Imaging for Enhancing Facial Recognition Security

Abstract

Machine learning (ML) has significantly improved facial recognition systems' (FRS) accuracy, robustness, and reliability, making them one of the most viable biometric identity verification solutions in various authentication applications. However, there are concerns about using FRSs, including privacy violations, fake presentations, potential biases, and security issues. Furthermore, the remarkable advancement of AI-leveraged production and manipulation techniques of fictitious human facial images, DeepFake, elevates spreading misinformation and creating deception for identity theft, which becomes critical security and privacy threat. In this dissertation, ML-CHIEFS: Machine Learning-based Corneal-specular Highlight Imaging for Enhancing Facial Recognition Security, we researched and developed unique technologies to resolve significant FRS challenges, including Deepfakes and identity thefts, liveness presentation attacks (PA), and master face dictionary attacks (MFDA). We propose countermeasures against facial biometric PAs, detect DeepFakes, and identify MFDA using intelligent ML-based specular highlights detections upon the hypothesis that the existing facial spoofings fail to coordinate their counterfeits with the reflective components. First, we designed a software-based facial liveness detection method named Apple in My Eyes (AIME). AIME is intended to detect the liveness against spoofing for mobile device security using challenge-response testing. Our comprehensive experimental results reveal that AIME can efficiently detect PAs with high accuracy at around 200 ms against different types of sophisticated presentation attacks without any costly extra sensors nor involving users' active responses. Second, we proposed novel ML-based DeepFake detection technologies, including CHIEFS (Corneal-Specular Highlights Imaging for Enhancing Fake-Face Spotter), MobiDeep (Mobile DeepFake Detection through ML-based Corneal-Specular Backscattering), and READFake (Reflection and Environment-Aware DeepFake). CHIEFS detects various corneal-specular and facial highlights features and inspects the ensemble of the highlights with the surrounding environmental factors. The empirical results show that it improves the detection accuracy from 86.05% with the reflection shape similarity alone to 99.00% with the ResNet-50-V2 architecture. MobiDeep is a real-time, cloudless, lightweight mobile application for human visual DeepFake detection using ML technologies, which achieved high accuracy (98.7%) and rapid detection speed in detecting sophisticated DeepFake images within 200 ms. The READFake detection technique uses specular highlights on various facial and body parts and environmental factors. We have conducted extensive experiments to evaluate the performance of READFake using different input parameters and advanced DNN architectures on multiple public DeepFake datasets. The experimental results indicate that READFake achieves better accuracy (99.0%) than the SOTA methods in detecting DeepFake images. Finally, we develop a novel countermeasure against MFDAs using a Reflection-based Identification (DARI) system. Using a lightweight and low-latency vision transformer, we build a feature extractor network to identify the inconsistencies among the facial image's specular highlights and physiological characteristics. The empirical results show that DARI achieves very high detection accuracy ranging from 97.83% to 99.56% on public GAN-face detection datasets and instantaneous detection speed (less than 11 ms) against SOTA master face dictionary attacks.