arrow
返回

Deep0Tag: Deep Multiple Instance Learning for Zero-Shot Image Tagging

delete2020-01-01
delete22
PRE
AI
S
Shafin Rahman *
S
Salman Khan
N
Nick Barnes
DOI:10.1109/TMM.2019.2924511delete
delete原文链接
delete分享
delete收藏
摘要

摘要

En 中文
Zero-shot learning aims to perform visual reasoning about unseen objects. In-line with the success of deep learning on object recognition problems, several end-to-end deep models for zero-shot recognition have been proposed in the literature. These models are successful in predicting a single unseen label given an input image but do not scale to cases where multiple unseen objects are present. Here, we focus on the challenging problem of zero-shot image tagging, where multiple labels are assigned to an image, that may relate to objects, attributes, actions, events, and scene type. Discovery of these scene concepts requires the ability to process multi-scale information. To encompass global as well as local image details, we propose an automatic approach to locate relevant image patches and model image tagging within the Multiple Instance Learning (MIL) framework. To the best of our knowledge, we propose the first end-to-end trainable deep MIL framework for the multi-label zero-shot tagging problem. We explore several alternatives for instance-level evidence aggregation and perform an extensive ablation study to identify the optimal pooling strategy. Due to its novel design, the proposed framework has several interesting features: 1) unlike previous deep MIL models, it does not use any off-line procedure (e.g., Selective Search or EdgeBoxes) for bag generation. 2) During test time, it can process any number of unseen labels given their semantic embedding vectors. 3) Using only image-level seen labels as weak annotation, it can produce a localized bounding box for each predicted label. We experiment with the large-scale NUS-WIDE and MS-COCO datasets and achieve superior performance across conventional, zero-shot, and generalized zero-shot tagging tasks.
Keyword:
Deep learning
Multiple instance learning
Feature pooling
Object detection
Zero-shot tagging
AI总结

AI总结

对已上传原文的论文进行重点信息的提取,主要内容包括:简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。

期刊

IEEE Transactions on Multimedia 封面图
IEEE Transactions on Multimedia
IF:
9.7
论文数:
5.3K
被引数:
3.4W

机构

A
Australian National University
学者数:
1.5K
论文数: 741
被引数: 2.0W
引用论文

引用论文

Selective Search for Object Recognition
err2013-04-02
err3.9K
PREAI
errUijlings, J. R. R.; van de Sande, K. E. A.; Gevers, T.; Smeulders, A. W. M.
err分享
err收藏
Determining the RAD51-DNA Nucleoprotein Filament Structure and Function by Cryo-Electron Microscopy
err2018-01-01
err0
PREAI
errLingyun Zhao; Jingfei Xu; Weixing Zhao; Patrick Sung; Hong-Wei Wang
err分享
err收藏
学者 查看更多内容