.png){kind=icon}
返回
Multicategory Crowdsourcing Accounting for Variable Task Difficulty, Worker Skill, and Worker Intention
.png){kind=icon}
PRE
AI
DOI：10.1109/TKDE.2014.2327026.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Crowdsourcing allows instant recruitment of workers on the web to annotate image, webpage, or document databases. However, worker unreliability prevents taking a worker's responses at face value. Thus, responses from multiple workers are typically aggregated to more reliably infer ground-truth answers. We study two approaches for crowd aggregation on multicategory answer spaces: stochastic modeling-based and deterministic objective function-based. Our stochastic model for answer generation plausibly captures the interplay between worker skills, intentions, and task difficulties and captures a broad range of worker types. Our deterministic objective-based approach aims to maximize the average aggregate confidence of weighted plurality crowd decision making. In both approaches, we explicitly model the skill and intention of individual workers, which is exploited for improved crowd aggregation. Our methods are applicable in both unsupervised and semi-supervised settings, and also when the batch of tasks is heterogeneous, i.e., from multiple domains, with task-dependent answer spaces. As observed experimentally, the proposed methods can defeat tyranny of the masses, i.e., they are especially advantageous when there is an (a priori unknown) minority of skilled workers amongst a large crowd of unskilled (and malicious) workers.
Keyword:
Crowdsourcing
expectation-maximization
ensemble classification
inference
multicategory
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
10.4
论文数:
6.8K
被引数:
3.2W
机构
引用论文
Development of Intelligence Data Analytics System for Quality Enhancement of Die-Casting Process
.png){kind=icon}
PREAI
分享收藏
分享收藏分享收藏分享收藏Gait video-based prediction of unified Parkinson’s disease rating scale score: a retrospective study
OAAI
OAAI分享收藏分享收藏
查看更多内容