.png){kind=icon}
Return
Visual Relationship Embedding Network for Image Paragraph Generation
.png){kind=icon}
PRE
AI
DOI：10.1109/TMM.2019.2954750.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Image paragraph generation aims to produce a complete description of a given image. This task is more challenging than image captioning, which only generates one sentence to describe the entire image. Traditional paragraph generation methods usually produce paragraph descriptions based on individual regions that are detected by a Region Proposal Network (RPN). However, relationships among visual objects are either ignored or utilized in an implicit manner in previous work. In this paper, we attempt to explore more visual information through a novel paragraph generation network that explicitly incorporates visual relationship semantics when producing descriptions. First, a novel Relation Pair Generative Adversarial Network (RP-GAN) is designed to locate regions that may cover subjective or objective elements. Then, their relationships are inferred through an attention-based network. Finally, the visual features and relationship semantics of valid relation pairs are taken as inputs by a Long Short-Term Memory (LSTM) network for generating sentences. The experimental results show that by explicitly utilizing the predicted relationship information, our proposed method obtains more accurate and informative paragraph descriptions than previous methods.
Keywords:
Visualization
Semantics
Task analysis
Proposals
Automobiles
Buildings
Gallium nitride
Paragraph generation
image caption
region localization
attention network
visual relationship
GAN
LSTM
Journal
IF:
9.7
Papers:
4.7K
Citations:
2.4W
Organization
Cited Papers
Acetylation of Lysine 382 and Phosphorylation of Serine 392 in p53 Modulate the Interaction between p53 and MDC1 In Vitro.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAIShareSaveShareSaveShareSave
View more