.png){kind=icon}
.png){kind=icon}
Return
Neural architectures for aggregating sequence labels from multiple annotators
.png){kind=icon}
OA
AI
DOI：10.1016/j.neucom.2022.09.081.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Labelled data for training sequence labelling models can be collected from multiple annotators or work -ers in crowdsourcing. However, these labels could be noisy because of the varying expertise and reliabil-ity of annotators. In order to ensure high quality of data, it is crucial to infer the correct labels by aggregating noisy labels. Although label aggregation is a well-studied topic, only a number of studies have investigated how to aggregate sequence labels. Recently, neural network models have attracted research attention for this task. In this paper, we explore two neural network-based methods. The first method combines Hidden Markov Models with networks while also learning distributed representations of annotators (i.e., annotator embedding); the second method combines BiLSTM with autoencoders. The experimental results on three real-world datasets demonstrate the effectiveness of using neural networks for sequence label aggregation. Moreover, our analysis shows that annotators' embeddings not only make our model applicable to real-time applications, but also useful for studying the behaviour of annotators.(c) 2022 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http:// creativecommons.org/licenses/by/4.0/).
Keywords:
Natural language processing
Sequence labelling
Neural network
Annotation
Annotator reliability
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.