Question classification based on co-training style semi-supervised learning

Zhengtao Yu

doi:10.1016/j.patrec.2010.06.010

Question classification based on co-training style semi-supervised learning

2010-10-01

24

PRE

AI

DOI：10.1016/j.patrec.2010.06.010

Original

Original request for help

Save

Abstract

En 中文

In statistical question classification, semi-supervised learning that can exploit the abundant unlabeled samples has received substantial attention in recent years. In this paper, a novel question classification approach with the co-training style semi-supervised learning is proposed. In particular, the method extracts high-frequency keywords as classification features, and uses the word semantic similarity to adjust the feature weights. The classifiers are initially trained from labeled data and then the learned models are refined using unlabeled data which can get labeled if the classifiers agree on the labeling. Experiments on the Chinese question answering system in tourism domain were conducted by employing different feature selections, different supervised and semi-supervised algorithms, different feature dimensions and different unlabeled rates. The experimental results show the proposed method can effectively improve the classification accuracy. Specifically, under the 40% unlabeled rate of training set, the average accuracy rates reach 88.9% on coarse types and 78.2% on fine types, respectively, which get an improvement of around 2-4% points. (c) 2010 Elsevier B.V. All rights reserved.

Keywords:

Chinese question classification

Word semantic similarity

Semi-supervised learning

Co-training

Question classification based on co-training style semi-supervised learning

Abstract

Journal

Organization