BERT-based ensemble learning for multi-aspect hate speech detection

Ahmed Cherif Mazari

doi:10.1007/s10586-022-03956-x

Return

BERT-based ensemble learning for multi-aspect hate speech detection

2023-01-03

PRE

Ahmed Cherif Mazari *

Nesrine Boudoukhani

Abdelhamid Djeffal

DOI：10.1007/s10586-022-03956-x

Original

Original request for help

Save

Abstract

En 中文

The social media world nowadays is overwhelmed with unfiltered content ranging from cyberbullying and cyberstalking to hate speech. Therefore, identifying and cleaning up such toxic language presents a big challenge and an active area of research. This study is dedicated to multi-aspect hate speech detection based on classifying text in multi-labels including 'identity hate', 'threat', 'insult', 'obscene', 'toxic' and 'severe toxic'. The proposed approach is based on the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model combined with Deep Learning (DL) models to compose several ensemble learning architectures. The DL models used are built by stacking Bidirectional Long-Short Term Memory (Bi-LSTM) and/or Bidirectional Gated Recurrent Unit (Bi-GRU) on GloVe and FastText word embeddings. Whereby, these models and BERT are trained individually on multi-label hateful dataset and used in combination for hate speech detection tasks on social media. Thus, we demonstrate that encoding texts by using recent word embedding techniques as FastText and GloVe alongside Bi-LSTM and Bi-GRU can create models that, when combined with BERT, can enhance the ROC-AUC score to 98.63%.

Keywords:

BERT model

Multi-aspect hate speech

Language toxicity

Deep learning

Ensemble learning