.png){kind=icon}
返回
Is word order considered by foundation models? A comparative task-oriented analysis
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2023.122700.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Word order, a linguistic concept essential for conveying accurate meaning, is seemingly not that necessary in language models based on the existing works. Contrary to this prevailing notion, our paper delves into the impacts of word order by employing carefully selected tasks that demand distinct abilities. Using three large language model families (ChatGPT, Claude, LLaMA), three controllable word order perturbation strategies, one novel perturbation qualification metric, four well-chosen tasks, and three languages, we conduct experiments to shed light on this topic. Empirical findings demonstrate that Foundation models take word order into consideration during generation. Moreover, tasks emphasizing reasoning abilities exhibit a greater reliance on word order compared to those primarily based on world knowledge.
Keyword:
Foundation model
Word order
Order perturbation ratio
MGSM
WinoGrande
期刊
IF:
7.5
论文数:
3.0W
被引数:
10.2W
机构
查看更多机构
引用论文
.png){kind=icon}
分享收藏
分享收藏 没有更多内容