Interaction-Aware Transformer Network for Human-Object Interaction Detection

doi:10.1109/TCYB.2025.3587037

Interaction-Aware Transformer Network for Human-Object Interaction Detection

0

PRE

AI

DOI：10.1109/TCYB.2025.3587037

Original

Original request for help

Save

Abstract

En 中文

human-object interaction (HOI) detection tackles the problem of joint localization and classification of HOIs. Recent HOI detection methods are mainly based on transformer networks, where the explicit priors at the object level (e.g., scene layout, object appearance, or category) are usually fed into the transformer to improve the object query ability. Though these methods have achieved remarkable results, they did not pay enough attention to the implicit action-level information, which is the fundamental element of HOI. In this work, we propose an interaction-aware transformer network (IATN) to obtain the interaction-aware query, by jointly utilizing implicit action-level priors and explicit object-level priors. Specifically, we design an action-aware module (AAM) to aggregate implicit action priors from the scene level and instance level, respectively. Then, we design an action-oriented graph (AOG), where human feature and object feature are graph nodes and action semantics represent graph edges, to aggregate priors jointly from action level and object level. Afterwards, the interaction-aware query is acquired and finally adopted to obtain the HOI predictions. Besides, we leverage knowledge distillation to enhance the action-level priors by transferring the final HOI predictions to the intermediate features. Extensive experiments on HICO-DET and V-COCO datasets verify the effectiveness of our proposed interaction-aware model.

Keywords:

Action recognition

graph network

human-object interaction (HOI)

knowledge distillation

Interaction-Aware Transformer Network for Human-Object Interaction Detection

Abstract

Journal

Organization