Augmenting Image Descriptions Using Structured Prediction Output

Yahong Han

doi:10.1109/TMM.2014.2321530

Augmenting Image Descriptions Using Structured Prediction Output

2014-10-01

15

PRE

AI

DOI：10.1109/TMM.2014.2321530

Original

Original request for help

Save

Abstract

En 中文

The need for richer descriptions of images arises in a wide spectrum of applications ranging from image understanding to image retrieval. While the Automatic Image Annotation (AIA) has been extensively studied, image descriptions with the output labels lack sufficient information. This paper proposes to augment image descriptions using structured prediction output. We define a hierarchical tree-structured semantic unit to describe images, from which we can obtain not only the class and subclass one image belongs to, but also the attributes one image has. After defining a new feature map function of structured SVM, we decompose the loss function into every node of the hierarchical tree-structured semantic unit and then predict the tree-structured semantic unit for testing images. In the experiments, we evaluate the performance of the proposed method on two open benchmark datasets and compare with the state-of-the-art methods. Experimental results show the better prediction performance of the proposed method and demonstrate the strength of augmenting image descriptions.

Keywords:

Image descriptions

image annotation

structured learning

tree-structured semantic unit

Augmenting Image Descriptions Using Structured Prediction Output

Abstract

AI Summary

Journal

Organization