Optimizing Hate Text Detection using Custom NLP Techniques and an Adapted DeBERTa-based Machine Learning Model

Abstract

The rapid expansion of social media has transformed online communication, providing platforms for public debate and community engagement. However, this openness has also facilitated the spread of harmful content, particularly hate speech, which poses significant risks to individual well-being, social cohesion, and digital trust. Detecting such content remains a major challenge due to the subtle, context-dependent, and evolving nature of hateful expressions. Traditional machine learning models, though useful as early baselines, often fail to capture linguistic nuance and contextual depth. Recent advances in natural language processing (NLP), particularly Transformer-based architectures, have significantly improved text classification tasks by enabling context-sensitive embeddings. This research investigates the effectiveness of DeBERTa (Decoding-enhanced BERT with Disentangled Attention) for hate speech detection. The study employs a systematic methodology consisting of four stages: data preparation and preprocessing, exploratory data analysis, model development, and evaluation. A curated dataset of 2,041 social media posts, derived from a larger corpus, was pre-processed to remove noise, normalise text, and correct class imbalance. The DeBERTa-v3-large model was fine-tuned using cross-entropy loss and AdamW optimisation. Performance was assessed with accuracy, precision, recall, F1-score, ROC, and PR curves, while error analysis and confusion matrices were used to identify common misclassifications. The findings demonstrate that DeBERTa can effectively capture indirect meaning and grammar connections. Additionally, outperforming traditional approaches and offering robust classification of hate and non- hate content. The study contributes to both NLP research and the wider cybersecurity domain by supporting the development of more reliable automated moderation tools that promote safer digital environments.