.png){kind=icon}
Return
Data augmentation using Heuristic Masked Language Modeling
.png){kind=icon}
PRE
AI
DOI：10.1007/s13042-023-01784-y.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Data augmentation has played an important role in generalization capability and performance improvement for data-driven deep learning models in recent years. However, most of the existing data augmentation methods in NLP suffer from high manpower consumption or low promotion, which limits the practical applications. To this end, we propose a simple yet effective approach named Heuristic Masked Language Modeling(HMLM) to obtain high-quality data by introducing mask language modeling embedded in pre-trained models. More specifically, the HMLM method first identifies the core words of the sentence and masks some non-core fragments in the sentence. Then, these masked fragments will be filled with words created by the pre-trained model to match the contextual semantics. Compared with the previous data augmentation approaches, the proposed method can create more grammatical and contextual augmented data without a heavy cost. We conducted experiments on typical text classification tasks e.g., intent recognition, news classification and sentiment analysis separately. Experimental results demonstrate that our proposed method is comparable to state-of-the-art data augmentation approaches.
Keywords:
Data augmentation
Mask language modeling
Pre-trained models
Text classification
Journal
IF:
2.7
Papers:
3.2K
Citations:
5.6K
Organization
Cited Papers
.png){kind=icon}
ShareSave
ShareSaveShareSaveCognitive Predictors of Obsessive–Compulsive Symptoms in Adolescence: A Preliminary Investigation
PREAI
ShareSaveShareSave
View more