arrow
返回

Inter-Coder Agreement for Computational Linguistics

delete2008-12-01
delete921
delete
OA
AI
R
Ron Artstein *
M
Massimo Poesio
DOI:10.1162/coli.07-034-R2delete
delete原文链接
delete原文求助
delete分享
delete收藏
摘要

摘要

En 中文
This article is a survey of methods for measuring agreement among corpus annotators. It exposes the mathematics and underlying assumptions of agreement coefficients, covering Krippendorff's alpha as well as Scott's pi and Cohen's kappa; discusses the use of coefficients in several annotation tasks; and argues that weighted, alpha-like coefficients, traditionally less used than kappa-like measures in computational linguistics, may be more appropriate for many corpus annotation tasks-but that their use makes the interpretation of the value of the coefficient even harder.
Keyword:
LOW KAPPA
RELIABILITY
TEXT
DISCOURSE

期刊

Computational Linguistics 封面图
Computational Linguistics
IF:
5.3
论文数:
837
被引数:
2.7K

机构

U
University of Essex
学者数:
4.0K
论文数: 4.8K
被引数: 5
引用论文

引用论文

Imaginal exposure alone and imaginal exposure with cognitive restructuring in treatment of posttraumatic stress disorder.
err2003-08-01
err0
PREAI
errRichard A. Bryant; Michelle L. Moulds; Rachel M. Guthrie; Suzanne T. Dang; Reginald D. V. Nixon
err分享
err收藏
Dialogue act modeling for automatic tagging and recognition of conversational speech
err2000-09-01
err635
errOAAI
errStolcke, A; Ries, K; Coccaro, N; Shriberg, E; Bates, R; Jurafsky, D; Taylor, P; Martin, R; Van Ess-Dykema, C; Meteer, M
err分享
err收藏
err分享
err收藏
学者 查看更多内容