.png){kind=icon}
返回
GLOCAL: A self-supervised learning framework for global and local motion estimation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patrec.2023.12.024.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Motions in videos are typically a mixture of local dynamic object motions and global camera motion, which are inconsistent in some cases, and even interfere with each other, causing difficulties in various downstream applications, such as video stabilization that requires the global motion, and action recognition that consumes local motions. Therefore, it is crucial to estimate them separately. Existing methods separate two motions from the mixed motion fields, such as optical flow. However, the quality of mixed motion determines the higher bounds of the performance. In this work, we propose a framework, GLOCAL, to directly estimate global and local motions simultaneously from adjacent frames in a self-supervised manner. Our GLOCAL consists of a Global Motion Estimation (GME) module and a Local Motion Estimation (LME) module. The GME module involves a mixed motion estimation backbone, an implicit bottleneck structure for feature dimension reduction, and an explicit bottleneck for global motion recovery based on the global motion bases with foreground mask under the training guidance of proposed global reconstruction loss. An attention U-Net is adopted for LME which produces local motions while excluding motion of irrelevant regions under the guidance of proposed local reconstruction loss. Our method can achieve better performance than the existing methods on the homography estimation dataset DHE and the action recognition dataset NCAA and UCF-101.
Keyword:
Video understanding
motion pattern
optical flow
motion estimation
期刊
IF:
3.3
论文数:
8.0K
被引数:
1.6W
机构
引用论文
分享收藏Forced acoustical response of a cavity coupled with a semi-infinite space using coupled mode theory.png)
Wave Motion
Wave MotionIF0
PREAI
分享收藏Wireless Underground Communications in Sewer and Stormwater Overflow Monitoring: Radio Waves through Soil and Asphalt MediumInformation
OAAI
InformationIF0
OAAI分享收藏Global motion estimation with iterative optimization-based independent univariate model for action recognitionPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏
查看更多内容