Partially Supervised Compatibility Modeling

Weili Guan

doi:10.1109/TIP.2022.3187290

Partially Supervised Compatibility Modeling

2022-01-01

19

PRE

AI

Xiaojun Chang (Xiaojun Chang) *

Liqiang Nie (Liqiang Nie)

DOI：10.1109/TIP.2022.3187290

Original

Original request for help

Save

Abstract

En 中文

Fashion Compatibility Modeling (FCM), which aims to automatically evaluate whether a given set of fashion items makes a compatible outfit, has attracted increasing research attention. Recent studies have demonstrated the benefits of conducting the item representation disentanglement towards FCM. Although these efforts have achieved prominent progress, they still perform unsatisfactorily, as they mainly investigate the visual content of fashion items, while overlooking the semantic attributes of items (e.g., color and pattern), which could largely boost the model performance and interpretability. To address this issue, we propose to comprehensively explore the visual content and attributes of fashion items towards FCM. This problem is non-trivial considering the following challenges: a) how to utilize the irregular attribute labels of items to partially supervise the attribute-level representation learning of fashion items; b) how to ensure the intact disentanglement of attribute-level representations; and c) how to effectively sew the multiple granulairites (i.e, coarse-grained item-level and fine-grained attribute-level) information to enable performance improvement and interpretability. To address these challenges, in this work, we present a partially supervised outfit compatibility modeling scheme (PS-OCM). In particular, we first devise a partially supervised attribute-level embedding learning component to disentangle the fine-grained attribute embeddings from the entire visual feature of each item. We then introduce a disentangled completeness regularizer to prevent the information loss during disentanglement. Thereafter, we design a hierarchical graph convolutional network, which seamlessly integrates the attribute- and item-level compatibility modeling, and enables the explainable compatibility reasoning. Extensive experiments on the real-world dataset demonstrate that our PS-OCM significantly outperforms the state-of-the-art baselines. We have released our source codes and well-trained models to benefit other researchers (https://site2750.wixsite.com/ps-ocm).

Keywords:

Visualization

Representation learning

Graph neural networks

Task analysis

Semantics

Image color analysis

Data models

Partial supervision

disentangled representation

fashion compatibility estimation

graph convolutional network

Partially Supervised Compatibility Modeling

Abstract

Journal

Organization

Cited Papers