Adversarial Attacks on Scene Graph Generation

doi:10.1109/TIFS.2024.3360880

Adversarial Attacks on Scene Graph Generation

2024-01-01

1

PRE

AI

DOI：10.1109/TIFS.2024.3360880

Original

Original request for help

Save

Abstract

En 中文

Scene graph generation (SGG) effectively improves semantic understanding of the visual world. However, the recent interest of researchers focuses on enhancing SGG in non-adversarial settings, which raises our curiosity about the adversarial robustness of SGG models. To bridge this gap, we perform adversarial attacks on two typical SGG tasks, Scene Graph Detection (SGDet) and Scene Graph Classification (SGCls). Specifically, we initially propose a bounding box relabeling method to reconstruct reasonable attack targets for SGCls. It solves the inconsistency between the specified bounding boxes and the scene graphs selected as attack targets. Subsequently, we introduce a two-step weighted attack by removing the predicted objects and relational triples that affect attack performance, which significantly increases the success rate of adversarial attacks on two SGG tasks. Extensive experiments demonstrate the effectiveness of our methods on five popular SGG models and four adversarial attacks.

Keywords:

Task analysis

Object detection

Windows

Visualization

Mirrors

Predictive models

Perturbation methods

Scene graph generation

adversarial attack

bounding box relabeling

two-step weighted attack

Adversarial Attacks on Scene Graph Generation

Abstract

Journal

Organization