I Feel it in Your Fingers: The Automatic Analysis of Dyadic Online Chats Using Typing Behaviour

Abstract

There is more to human-human interaction than simple spoken words. Even though our concentration is on what people say (the verbal content of the message they are exchanging), we are still capable of perceiving and understanding the variety of nonverbal behavioural signals humans present when communicating with each other. These nonverbal behavioural cues, which include facial expressions, vocalisations, postures, gestures, etc., can assist in understanding, beyond the literal meaning of the words we hear, the social and emotional aspects of the interaction in which we are engaged in.

Today, over 4 billion people use the Internet to connect, work, study, entertain, etc. Typically, we cannot see our interlocutor's face or read his or her facial expressions. We also cannot hear the person's voice or tell how their tone changes. While the ability to remain anonymous online is often regarded as one of the most significant benefits of the Internet, the absence of nonverbal behavioural cues makes the task of recognising people's identities and characteristics a challenging one. This, in turn, could promote different types of misuse in hyperspace, including cybercrimes, child predators, love scams, and many more. The literature suggests that when individuals type, they unconsciously produce nonverbal behavioural cues that manifest their inner states (e.g., emotions, personality, etc.). Cues like these leak information related to the actual state of an individual instead of what that individual wants to appear like \cite{Vinciarelli2009}. In this respect, typing behaviour (also known as keystroke dynamics) is the study of the distinctive typing patterns that reveal real-time information about the author of the text \cite{Fairhurst2012}.

Typing behaviour has been the focus of behavioural biometrics research for more than 30 years \cite{Roy2022}. Yet, the efforts were mainly focused on user identification depending on how they type on a keyboard. In recent years, there has been a growing interest in using typing behaviour to infer individuals' social information (i.e., age, gender, emotion, education level, etc.) \cite{Roy2022}. This dissertation goes in such a direction by investigating the interplay between typing behaviour - which can be detected during interactive chats collected with a key-logging platform - and socially relevant phenomena, namely Gender, Personality, and Conflict Management Styles, and to develop methodologies that can infer such phenomena from the way people type. The rationale behind the use of typing behaviour is that it does not focus on content (what people type) but on nonverbal aspects of communication (how people type) that preserve the privacy of the individuals involved and are language-independent. Furthermore, approaches for sensing typing behaviour do not require any specialised hardware other than the keyboard and are, therefore, inexpensive and unobtrusive. Finally, given that keyboards are probably one of the most common and widely used elements in any computer or interactive device, the results obtained in the experiments carried out in this thesis are likely to generalise to a much broader spectrum of settings.

A total of 30 text-based chats produced by 60 participants were used to conduct the experiments for this thesis. The chat conversations focused on the Winter Survival Task (WST), a scenario frequently employed in social psychology. To fulfil the research goals, the proposed approach focuses on the identification of the features that can be extracted from online chats and that account for typing behaviour. Once such features are identified and extracted, the next step is the development of machine learning approaches capable of mapping the extracted features into the social phenomena of interest (gender, personality, and CMS). The work carried out in this research has found that the simple act of typing, a task that humans routinely do in everyday life, can carry a great deal of information related to their demographics (e.g., gender) and their social behaviour (e.g., personality and conflict management styles). This means that analysing typing behaviour using artificial intelligence techniques can help construct socially intelligent machines capable of detecting and perceiving social contexts similarly to humans.

The ability to recognise an unknown person's characteristics using their typing behaviour offers significant benefits. These include targeted advertising, security and privacy, online customer service, etc. But beyond all that, the development of such approaches is associated with ethical concerns that need to be addressed before any application outside the laboratory. One of the main implications is that unauthorised analysis of typing behaviour provides the possibility of using these techniques to gather private and sensitive information (e.g., gender, sexual preferences, personality, marital status, etc.) from the user without their consent. For such a reason, care must be taken to ensure that such technologies are built in accordance with the regulatory framework for data privacy (e.g., under the Data Protection Act 2018 in the UK).