Sentiment analysis of Saudi's Postal & logistical service providers based on Twitter feedback to improve the service

Abstract

Considering the constant and rapid development of social media platforms like Facebook and Twitter, it has become the norm for people to use these applications frequently during the day. Twitter refers to a social media website that enables users to send and receive short messages, known as tweets. Social media platforms have also become a valuable source for industry leaders to conduct their decision-making activities based on their clients’ opinions and feedbacks. Considering the current significance of social media platforms, this study aims to develop a machine learning (ML) framework that can analyse the tweets on Twitter, and identify the negative, positive, or neutral sentiments in the postal and logistical sector. This will aid in illustrating the relationship between the sentiments conveyed in Arabic tweets and customers’ experiences about the provided service. In this project, an Arabic text classification is designed and implemented for twitter content, using various algorithms such as Random Forest Classifier (RF), Logistic regression (LR), Support Vector Machine (SVM), multilayer perceptron (MLP), and multinominal Naive Bayes (MNB). The experiments are conducted in two major sections; the first one includes the use of negative, positive, and neutral classes, while the subsequent one includes only two classes, namely the positive and negative classes. The results show that in Arabic, sentiments classifiers achieved higher accuracies using negative and positive classes with Stratified-Kfold through TF-IDF as the dataset is imbalanced. The Random Forest Classifier F1 scored 82%, SVM and LR scored 81%. When the same experiment was conducted with the use of ngrams range (2,3), the results were also compatible. However, in the case of 3 classes, the same experiments yielded results showed that classifiers perform better with TF-IDF, where Random Forest, SVM, and LR classifiers scored the highest F1 scores with 76%, 75%, 75% respectively, while KNN and AdaBoost classifiers scored only 67%. However, while applying Ngrams (bigram and trigrams), SVM, LR outperform the other models with the highest F1 scores, with 73%.