Novel Deepfakes Detection Strategies: Insights from Prosopagnosia

Abstract

The credibility of audio and video content, which is essential to our perception of reality, is increasingly challenged by advancements in deepfake generation techniques. Existing detection models primarily focus on identifying anomalies and digital artifacts. However, the rapid evolution of technology enables the creation of sophisticated deepfakes that can evade these methods. This thesis investigates the effectiveness of different facial features for deepfake detection in images and face recognition in individuals with prosopagnosia. It examines whether there is a correlation between the facial features prioritized by AI models for deepfake detection and those emphasized in training programs aimed at enhancing face recognition in individuals with prosopagnosia. Additionally, it assesses the impact of occluding each facial feature during training on AI model performance and identifies which facial elements individuals with prosopagnosia find most challenging to recognize. Inspired by research into prosopagnosia, which highlights the importance of internal facial features like the eyes and nose, this study proposes a novel approach to deepfake detection. The methodology involves identifying critical facial features, applying face cut-out techniques to create training images with various occlusions, and evaluating AI models trained on these datasets using EfficientNet-B7 and Xception models. The results indicate that models trained with occluded datasets performed better, with the EfficientNet-B7 model achieving a higher accuracy rate (92%) when core facial elements (eyes and nose) were covered, compared to models trained on datasets without occlusions or with occlusions covering external features. This suggests that focusing on features outside the face’s center improves detection accuracy. The findings also highlight that facial cues beneficial for individuals with prosopagnosia do not uniformly translate to equivalent value for AI models. This research demonstrates that detection systems can be more effective by focusing on a small region of the face, contributing significantly to the improvement of deepfake detection methods and enhancing our understanding of face recognition processes.