Privileged multi-task learning for attribute-aware aesthetic assessment

Yangyang Shu

doi:10.1016/j.patcog.2022.108921

Privileged multi-task learning for attribute-aware aesthetic assessment

2022-12-01

6

PRE

AI

DOI：10.1016/j.patcog.2022.108921

Original

Original request for help

Save

Abstract

En 中文

Aesthetic attributes are crucial for aesthetics because they explicitly present some photo quality cues that a human expert might use to evaluate a photo's aesthetic quality. However, the aesthetic attributes have not been largely and sufficiently exploited for photo aesthetic assessment. In this paper, we pro-pose a novel approach to photo aesthetic assessment with the help of aesthetic attributes. The aesthetic attributes are used as privileged information (PI), which is often available during training phase but un-available in prediction phase due to the high collection expense. The proposed framework consists of a deep multi-task network as generator and a fully connected network as discriminator. Deep multi-task network learns the aesthetic attributes and score simultaneously to capture their dependencies and ex-tract better feature representations. Specifically, we use ranking constraint in the label space, similarity constraint and prior probabilities loss in the privileged information space to make the output of multi-task network converge to that of ground truth. Adversarial loss is used to identify and distinguish the predicted privileged information of a deep multi-task network from the ground truth PI distribution. Ex-perimental results on two benchmark databases demonstrate the superiority of the proposed method to state-of-the-art.(c) 2022 Elsevier Ltd. All rights reserved.

Keywords:

Aesthetic assessment

Privileged information

Multi -task learning

Privileged multi-task learning for attribute-aware aesthetic assessment

Abstract

Journal

Organization