.png){kind=icon}
.png){kind=icon}
Return
Inter-Coder Agreement for Computational Linguistics
.png){kind=icon}
OA
AI
DOI：10.1162/coli.07-034-R2.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This article is a survey of methods for measuring agreement among corpus annotators. It exposes the mathematics and underlying assumptions of agreement coefficients, covering Krippendorff's alpha as well as Scott's pi and Cohen's kappa; discusses the use of coefficients in several annotation tasks; and argues that weighted, alpha-like coefficients, traditionally less used than kappa-like measures in computational linguistics, may be more appropriate for many corpus annotation tasks-but that their use makes the interpretation of the value of the coefficient even harder.
Keywords:
LOW KAPPA
RELIABILITY
TEXT
DISCOURSE