Can NLP Models Evaluate Social Situations?

Abstract

In our everyday lives, we go through situations, interact with others, and take actions on specific topics. Sometimes conflict arises as a result of our actions, and one of the parties involved in the situation might be held accountable for conflict. In this disserta- tion, we investigated whether natural language processing (NLP) classification models can be used to evaluate conflicting social situations and decide who should be blamed for the conflict. That is, given a text description of a conflicting social situation, can we NLP models judge (or classify) who should be blamed? To achieve this, we collected a dataset that contains text descriptions of conflicting social situations. Each description is labelled with a judgment that is either blame one of the parties involved in the situ- ation, both, or no one as all the parties made justifiable actions. First, we analyzed the dataset to get insight into the linguistic features that characterize each social situation judgment class and found that the differences between classes are statistically signif- icant in many linguistic and psychological dimensions. Then, we built classifiers to judge the conflicts in social situations. These include features-based classifiers, Hier- archical Attention Networks (HAN) and Bidirectional Encoder Representations from Transformers (BERT). We found these models perform poorly when we used the text description of the situations, but when we used the discussions that people generated when judging the situations, the performance of the models improved significantly.