A BCI Framework for Affection Recognition: Channel and Feature Selection, and Subjective Label Dichotomization

Abstract

Affective computing has become a vital component in the evolvement of artificial intelligence humanization. Compared to various sources of reading human emotions, brain signals are considered more objective and accurate in the perspective of brain-computer interaction. Brain imaging and recording solutions such as fMRI and EEG are feasible considering the massive number of neurons (i.e., quantified in billions) and the rapid and unanticipated interactivity among them. Yet, some challenges are multifaceted and complex. In particular, the dimensionality of the spatially distributed channels and the temporal resolution in EEG-based brain-computer interface (BCI) undermines the prediction power of human affections. Thus, prior to modeling such data, as the final stage of the learning pipeline, adequate preprocessed signals' features of the stimuli-related electrode channels are of the essence to predict underlying human emotions. This thesis investigates and proposes a framework that tackles two problems in EEG-related affective computing studies. The first pipeline is stimulus-dependent, in which we implement subject-specific unsupervised learning to select the most stimulus-subject-relevant EEG features and channels. Also, we embed unsupervised algorithms for feature extraction and selection in the time and frequency domains. The second pipeline is to solve the problem of subjective labeling and label imbalance in such experiments. In BCI applications, ground truth labels are expected to be precise - their existence is a vital component of supervised learning problems. In some instances, however, they can prove to be obstacles. They can lead to two possible issues: class imbalances (i.e., skewed label distribution) and unreliability due to the uncertainty of subjects' underlying emotional states. Since the labels are continuous, they need to be dichotomized for a classification task. Dichotomization is typically decided statistically or based on a subject matter expert. However, the subjectivity of participants and its impact is neglected. To improve the prediction pipeline, we investigate the effect of thresholding on EEG emotional self-assessment to minimize subjectivity, improve model outcomes, and alleviate the impact of label imbalance.