.png){kind=icon}
返回
Multi-MCCR: Multiple models regularization for semi-supervised text classification with few labels
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2023.110588.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Semi-supervised learning has achieved impressive results and is commonly applied in text classifica-tions. However, in situations where labeled texts are exceedingly limited, neural networks are prone to over-fitting due to the non-negligible inconsistency between model training and inference caused by dropout mechanisms that randomly mask some neurons. To alleviate this inconsistency problem, we propose a simple Multiple Models Contrast learning based on Consistent Regularization, named Multi-MCCR, which consists of multiple models with the same structure and a C-BiKL loss strategy. Specifically, one sample first goes through multiple identical models to obtain multiple different output distributions, which enriches the sample output distributions and provides conditions for subsequent consistency approximation. Then, the C-BiKL loss strategy is proposed to minimize the combination of the bidirectional Kullback--Leibler (BiKL) divergence between the above multiple output distributions and the Cross-Entropy loss on labeled data, which provides consistency constraints (BiKL) for the model and simultaneously ensures correct classification (Cross-Entropy). Through the above setting of multi-model contrast learning, the inconsistency caused by the randomness of dropout between model training and inference is alleviated, thereby avoiding over-fitting and improving the classification ability in scenarios with limited labeled samples. We conducted experiments on six widely-used text classification datasets, including sentiment analysis, topic categorization, and reviews classification, and the experimental results show that our method is universally effective in semi-supervised text classification with limited labeled texts. (c) 2023 Elsevier B.V. All rights reserved.
Keyword:
Semi-supervised learning
Multiple models contrast learning
Consistent regularization
Text classification
期刊
K
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Prognostic role of FUT8 expression in relation to p53 status in stage II and III colorectal cancer.png)
PLOS ONE
OAAI
PLOS ONEIF0
OAAI分享收藏分享收藏
Distributed model predictive control of constrained nonlinear systems with communication delays
PREAI
分享收藏
查看更多内容