.png){kind=icon}
返回
Visual Spatial Reasoning
.png){kind=icon}
OA
AI
DOI：10.1162/tacl_a_00566.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Spatial relations are a basic part of human cognition. However, they are expressed in natural language in a variety of ways, and previous work has suggested that current vision-and-language models (VLMs) struggle to capture relational information. In this paper, we present Visual Spatial Reasoning (VSR), a dataset containing more than 10k natural text-image pairs with 66 types of spatial relations in English (e.g., under, in front of, facing). While using a seemingly simple annotation format, we show how the dataset includes challenging linguistic phenomena, such as varying reference frames. We demonstrate a large gap between human and model performance: The human ceiling is above 95%, while state-of-the-art models only achieve around 70%. We observe that VLMs' by-relation performances have little correlation with the number of training examples and the tested models are in general incapable of recognising relations concerning the orientations of objects.(1)
期刊
T
IF:
6.9
论文数:
486
被引数:
5.7K
机构
引用论文
.png){kind=icon}
Parasitological Evaluation of a Foodhandler Population Cohort in Panama: Risk Factors for Intestinal Parasitism巴拿马Foodhandler人群队列的寄生虫学评估: 肠道寄生虫的危险因素
PREAI
分享收藏Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
OAAI
OAAI分享收藏
查看更多内容