.png){kind=icon}
返回
xcomet: Transparent Machine Translation Evaluation through Fine-grained Error Detection
.png){kind=icon}
OA
AI
DOI：10.1162/tacl_a_00683.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Widely used learned metrics for machine translation evaluation, such as Comet and Bleurt, estimate the quality of a translation hypothesis by providing a single sentence-level score. As such, they offer little insight into translation errors (e.g., what are the errors and what is their severity). On the other hand, generative large language models (LLMs) are amplifying the adoption of more granular strategies to evaluation, attempting to detail and categorize translation errors. In this work, we introduce xcomet, an open-source learned metric designed to bridge the gap between these approaches. xcomet integrates both sentence-level evaluation and error span detection capabilities, exhibiting state-of-the-art performance across all types of evaluation (sentence-level, system-level, and error span detection). Moreover, it does so while highlighting and categorizing error spans, thus enriching the quality assessment. We also provide a robustness analysis with stress tests, and show that xcomet is largely capable of identifying localized critical errors and hallucinations.
期刊
T
IF:
6.9
论文数:
486
被引数:
5.7K
机构
引用论文
Epidemiological Trends of Acute Chemical Poisoning among Children over a Recent Three-Year Period in Saudi Arabia.png)
Children
OAAI
ChildrenIF0
OAAI分享收藏Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation专家，错误和上下文: 对机器翻译的人类评估的大规模研究OAAI
OAAI分享收藏分享收藏
查看更多内容