Aesthetic multi-attributes network for image captioning

doi:10.1016/j.compeleceng.2025.110103

Return

Aesthetic multi-attributes network for image captioning

2025-04-01

PRE

DOI：10.1016/j.compeleceng.2025.110103

Original

Original request for help

Save

Abstract

En 中文

Image aesthetic quality assessment has witnessed a remarkable rise in popularity in recent years. Aesthetic captioning has emerged as a novel approach to encapsulate the overall aesthetic impression of an image. However, the inherently challenging task of annotating aesthetic attributes has constrained the scale of existing datasets. To address this limitation, the DPChallenge Multi-Attributes Captions (DPC-MAC) dataset was developed by integrating semiautomatically generated annotations from small-scale, fully annotated datasets with extensive technical reviews sourced from a photography platform. The DPC-MAC dataset encompasses four key aesthetic attributes: composition, lighting, color, and subject. To effectively leverage this data, we introduce an innovative Aesthetic Multi-Attributes Captioning Network (AMACN), comprising the Bottom-Up and Top-Down Attention Network (BUTDAN) and the Object- Semantics Aligned Pretrained Network (OSAPN). Both networks are trained using a combination of small-scale, fully annotated datasets and the large-scale DPC-MAC dataset. The performance of the proposed AMACN model on DPC-MAC surpasses existing methods based on standard image captioning evaluation metrics, demonstrating its efficacy. This groundbreaking task of aesthetic attribute assessment represents a promising avenue for advancing research in this field. By innovatively integrating aesthetic attributes with descriptive commentary, the DPCMAC dataset provides a valuable resource for researchers to develop more precise and nuanced aesthetic models. This work not only paves the way for further exploration of image aesthetics but also holds the potential to enhance the quality and sophistication of aesthetic evaluations.

Keywords:

Image aesthetic quality assessment

Image captioning

Aesthetic attributes assessment

Semi-supervised learning