Deep Learning From Multiple Noisy Annotators as A Union

doi:10.1109/TNNLS.2022.3168696

Deep Learning From Multiple Noisy Annotators as A Union

2023-12-01

12

PRE

AI

DOI：10.1109/TNNLS.2022.3168696

Original

Original request for help

Save

Abstract

En 中文

Crowdsourcing is a popular solution for large-scale data annotations. So far, various end-to-end deep learning methods have been proposed to improve the practical performance of learning from crowds. Despite their practical effectiveness, most of them have two major limitations--they do not hold learning consistency and suffer from computational inefficiency. In this article, we propose a novel method named UnionNet, which is not only theoretically consistent but also experimentally effective and efficient. Specifically, unlike existing methods that either fit a given label from each annotator independently or fuse all the labels into a reliable one, we concatenate the one-hot encoded vectors of crowdsourced labels provided by all the annotators, which takes all the labeling information as a union and coordinates multiple annotators. In this way, we can directly train an end-to-end deep neural network by maximizing the likelihood of this union with only a parametric transition matrix. We theoretically prove the learning consistency and experimentally show the effectiveness and efficiency of our proposed method.

Keywords:

Training

Deep learning

Labeling

Noise measurement

Neural networks

Standards

Learning systems

Annotators

crowdsourcing

noisy labels

transition matrix

Deep Learning From Multiple Noisy Annotators as A Union

Abstract

Journal

Organization