.png){kind=icon}
返回
Geometric Features Enhanced Human-Object Interaction Detection
.png){kind=icon}
OA
AI
DOI：10.1109/TIM.2024.3427800.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Cameras are essential vision instruments to capture images for pattern detection and measurement. Human-object interaction (HOI) detection is one of the most popular pattern detection approaches for captured human-centric visual scenes. Recently, Transformer-based models have become the dominant approach for HOI detection due to their advanced network architectures and, thus, promising results. However, most of them follow the one-stage design of vanilla Transformer, leaving rich geometric priors underexploited and leading to compromised performance, especially when occlusion occurs. Given that geometric features tend to outperform visual ones in occluded scenarios and offer information that complements visual cues, we propose a novel end-to-end Transformer-style HOI detection model, i.e., geometric features enhanced HOI detector (GeoHOI). One key part of the model is a new unified self-supervised keypoint learning method named UniPointNet that bridges the gap of consistent keypoint representation across diverse object categories, including humans. GeoHOI effectively upgrades a Transformer-based HOI detector benefiting from the keypoints similarities measuring the likelihood of HOIs and local keypoint patches to enhance interaction query representation, so as to boost HOI predictions. Extensive experiments show that the proposed method outperforms the state-of-the-art models on V-COCO and achieves competitive performance on HICO-DET. Case study results on the postdisaster rescue with vision-based instruments showcase the applicability of the proposed GeoHOI in real-world applications.
Keyword:
Transformers
Feature extraction
Visualization
Detectors
Semantics
Instruments
Shape
Attention mechanism
graph convolutional network (GCN)
human-object interaction (HOI)
interactiveness learning
object keypoints
期刊
IF:
5.9
论文数:
2.0W
被引数:
5.8W
机构
引用论文
分享收藏Urbanisation at Multiple Scales Is Associated with Larger Size and Higher Fecundity of an Orb-Weaving Spider.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏Safe and Efficient Robot Manipulation: Task-Oriented Environment Modeling and Object Pose Estimation安全高效的机器人操纵: 面向任务的环境建模和对象姿态估计
PREAI
分享收藏IPGN: Interactiveness Proposal Graph Network for Human-Object Interaction DetectionIPGN: 人机交互检测的交互性建议图网络
PREAI
分享收藏A Local-Global Estimator Based on Large Kernel CNN and Transformer for Human Pose Estimation and Running Pose Measurement基于大核CNN和Transformer的局部-全局估计器，用于人体姿态估计和运行姿态测量
PREAI
分享收藏
查看更多内容