.png){kind=icon}
Return
DLTA: A Framework for Dynamic Crowdsourcing Classification Tasks
.png){kind=icon}
PRE
AI
DOI：10.1109/TKDE.2018.2849385.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The increasing popularity of crowdsourcing markets enables the application of crowdsourcing classification tasks. How to conduct quality control in such an application to achieve accurate classification results from noisy workers is an important and challenging task, and has drawn broad research interests. However, most existing works do not exploit the label acquisition phase, which results in their disability of making a proper budget allocation. Moreover, some works impractically make the assumption of managing workers, which is not supported by common crowdsourcing platforms such as AMT or CrowdFlower. To overcome these drawbacks, in this paper, we devise a Dynamic Label Acquisition and Answer Aggregation (DLTA) framework for crowdsourcing classification tasks. The framework proceeds in a sequence of rounds, adaptively conducting label inference and label acquisition. In each round, it analyzes the collected answers of previous rounds to perform proper budget allocation, and then issues the resultant query to the crowd. To support DLTA, we propose a generative model for the collection of labels, and correspondingly strategies for label inference and budget allocation. Experimental results show that compared with existing methods, DLTA obtains competitive accuracy in the binary case. Besides, its extended version, which plugs in the state-of-the-art inference technique, achieves the highest accuracy.
Keywords:
Classification crowdsourcing
quality control
label inference
label acquisition
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
10.4
Papers:
6.9K
Citations:
3.2W
Organization
No organization information available
Cited Papers
.png){kind=icon}
ShareSaveEFFECT OF TESTOSTERONE ON PROTEIN SYNTHESIS AND CONTRACTILITY OF THE LEVATOR ANI MUSCLE OF THE RAT
PREAI
ShareSaveShareSave
View more