.png){kind=icon}
Return
Human-Centric Image Captioning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2022.108545.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In this paper, we propose a new topic, Human-Centric Captioning, to mainly describe the human behavior in an image. Human activities and relationships are the primary objectives of visual understanding in daily applications. However, existing image captioning systems cannot differently treat humans and other objects, which limits the ability to understand and describe diverse human activities. As the first explorer of this new task, we build a novel Human-Centric COCO dataset concentrating on humans. Accordingly, we propose a novel Human-Centric Captioning Model (HCCM) that focuses on human-centric feature hierarchization and sentence generation. Specifically, our model first utilizes human body part level knowledge to hierarchize the image features and then applies a novel three-branch captioning model to process these hierarchical features independently to calibrate the descriptions of human actions. Comprehensive experiments demonstrate that our HCCM achieves the state-of-the-art performance with BLEU-4, CIDEr and SPICE scores of 41.5, 127.3, 23.5 respectively. Dataset and code are publicly available at https://github.com/JohnDreamer/HCCM/. (c) 2022 Elsevier Ltd. All rights reserved.
Keywords:
Human-centric
Image captioning
Feature hierarchization
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
Cited Papers
ShareSaveEnhancing the alignment between target words and corresponding frames for video captioning.png)
PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
ShareSaveShareSaveLearning visual relationship and context-aware attention for image captioningPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
ShareSaveScript identification in natural scene image and video frames using an attention based Convolutional-LSTM networkPATTERN RECOGNITION
OAAI
PATTERN RECOGNITIONIF7.6
OAAIShareSaveShareSaveShareSave no more
no more