.png){kind=icon}
返回
Machine Translation Testing via Syntactic Tree Pruning
.png){kind=icon}
OA
AI
DOI：10.1145/3640329.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Machine translation systems have been widely adopted in our daily life, making life easier and more convenient. Unfortunately, erroneous translations may result in severe consequences, such as financial losses. This requires to improve the accuracy and the reliability of machine translation systems. However, it is challenging to test machine translation systems because of the complexity and intractability of the underlying neural models. To tackle these challenges, we propose a novel metamorphic testing approach by syntactic tree pruning (STP) to validate machine translation systems. Our key insight is that a pruned sentence should have similar crucial semantics compared with the original sentence. Specifically, STP (1) proposes a core semantics-preserving pruning strategy by basic sentence structures and dependency relations on the level of syntactic tree representation, (2) generates source sentence pairs based on the metamorphic relation, and (3) reports suspicious issues whose translations break the consistency property by a bag-of-words model. We further evaluate STP on two state-of-the-art machine translation systems (i.e., Google Translate and Bing Microsoft Translator) with 1,200 source sentences as inputs. The results show that STP accurately finds 5,073 unique erroneous translations in Google Translate and 5,100 unique erroneous translations in Bing Microsoft Translator (400% more than state-of-the-art techniques), with 64.5% and 65.4% precision, respectively. The reported erroneous translations vary in types and more than 90% of them are not found by state-of-the-art techniques. There are 9,393 erroneous translations unique to STP, which is 711.9% more than state-of-the-art techniques. Moreover, STP is quite effective in detecting translation errors for the original sentences with a recall reaching 74.0%, improving state-of-the-art techniques by 55.1% on average.
Keyword:
Software testing
machine translation
metamorphic testing
期刊
A
IF:
6.2
论文数:
1.2K
被引数:
3.4K
机构
引用论文
Molecular motion in plastic phases of organometallic compounds: iron-57 Mössbauer and carbon-13 n.m.r. spectra of tricarbonyl(1–5-hapto-cyclohexadienyl)iron tetrafluoroborate and related compounds有机金属化合物的塑料相中的分子运动: 铁-57 m ö ssbauer和碳-13 n.m.r.三羰基 (1-5-半环己二烯基) 四氟硼酸铁和相关化合物的光谱
.png){kind=icon}
PREAI
分享收藏
分享收藏分享收藏
查看更多内容