On the Security of End-to-End Encrypted Messaging and Calling Applications

Abstract

In recent years, the use of end-to-end encrypted messaging and calling applications has risen, driven by the need for secure communications. While these applications protect against unauthorized access, concerns about potential vulnerabilities have emerged. End-to-end encryption (E2EE) aims to safeguard private communications, yet fears of eavesdropping and communication manipulation linger, especially from government entities or attackers. Despite its effectiveness, E2EE integrity can be compromised, like through key substitution attacks. Worries center on authentication ceremonies and potential user errors leading to man-in-the-middle (MitM) attacks. Additionally, the introduction of client-side scanning (CSS) in secure applications to detect harmful content raises privacy concerns. CSS's local processing or endpoint filtering could undermine the promised encryption guarantees.

In this dissertation, we delve into the complexities surrounding the use of end-to-end encrypted messaging and calling applications, addressing issues of impersonations, MitM attacks, authentication ceremonies, and the topic of CSS technology. Our work provides a systematic analysis of E2EE functionality and authentication ceremonies in popular applications. We propose an automated approach to enhance and streamline the authentication ceremony within encrypted applications. Our work highlights vulnerabilities in voice-based authentication and stresses the need for stronger security measures. Additionally, we investigate the risks of using social media networks in the authentication ceremony and examine potential threats related to CSS technologies and their impact on E2EE principles.

Our dissertation provides the following contributions: First, we conduct a comprehensive security analysis of existing studies, identifying flaws and vulnerabilities in widely used encrypted applications, particularly focusing on authentication ceremonies. Second, we explore automated methods to enhance the authentication ceremony and reduce reliance on user interaction. Third, we undertake simulated investigations to identify potential vulnerabilities arising from exclusive reliance on a voice channel for the authentication ceremony in a real-world end-to-end encrypted application. This could compromise the security of static media and textual communications. The insights from our study suggest enhancing the security of end-to-end encrypted apps by using phonetically distinct words for codes, implementing warnings for suspicious voice code usage, employing multiple authentication channels, and prioritizing ongoing research for stronger security measures. Fourth, we introduce a novel investigation targeting social media authentication ceremonies, illustrating potential risks associated with user impersonation through counterfeit accounts. Our study suggests enhancing social media authentication security by displaying comprehensive user details, promoting hands-on user verification, using visual cues for unique identifiers, advocating continuous monitoring and adaptation, and fortifying end-to-end encrypted applications with multi-channel authentication. Lastly, we introduce an encrypted keyboard to address concerns related to CSS technology.