Multi-label zero-shot learning with graph convolutional networks

doi:10.1016/j.neunet.2020.09.010

Multi-label zero-shot learning with graph convolutional networks

2020-12-01

30

PRE

AI

DOI：10.1016/j.neunet.2020.09.010

Original

Original request for help

Save

Abstract

En 中文

The goal of zero-shot learning (ZSL) is to build a classifier that recognizes novel categories with no corresponding annotated training data. The typical routine is to transfer knowledge from seen classes to unseen ones by learning a visual-semantic embedding. Existing multi-label zero-shot learning approaches either ignore correlations among labels, suffer from large label combinations, or learn the embedding using only local or global visual features. In this paper, we propose a Graph Convolution Networks based Multi-label Zero-Shot Learning model, abbreviated as MZSL-GCN. Our model first constructs a label relation graph using label co-occurrences and compensates the absence of unseen labels in the training phase by semantic similarity. It then takes the graph and the word embedding of each seen (unseen) label as inputs to the GCN to learn the label semantic embedding, and to obtain a set of inter-dependent object classifiers. MZSL-GCN simultaneously trains another attention network to learn compatible local and global visual features of objects with respect to the classifiers, and thus makes the whole network end-to-end trainable. In addition, the use of unlabeled training data can reduce the bias toward seen labels and boost the generalization ability. Experimental results on benchmark datasets show that our MZSL-GCN competes with state-of-the-art approaches. (c) 2020 Elsevier Ltd. All rights reserved.

Keywords:

Zero-shot learning

Graph Convolutional Networks

Multi-label classification

Label correlations

Multi-label zero-shot learning with graph convolutional networks

Abstract

AI Summary

Journal

Organization