.png){kind=icon}
返回
SplatFlow: Learning Multi-frame Optical Flow via Splatting
.png){kind=icon}
PRE
AI
DOI：10.1007/s11263-024-01993-0.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The occlusion problem remains a crucial challenge in optical flow estimation (OFE). Despite the recent significant progress brought about by deep learning, most existing deep learning OFE methods still struggle to handle occlusions; in particular, those based on two frames cannot correctly handle occlusions because occluded regions have no visual correspondences. However, there is still hope in multi-frame settings, which can potentially mitigate the occlusion issue in OFE. Unfortunately, multi-frame OFE (MOFE) remains underexplored, and the limited studies on it are mainly specially designed for pyramid backbones or else obtain the aligned previous frame's features, such as correlation volume and optical flow, through time-consuming backward flow calculation or non-differentiable forward warping transformation. This study proposes an efficient MOFE framework named SplatFlow to address these shortcomings. SplatFlow introduces the differentiable splatting transformation to align the previous frame's motion feature and designs a Final-to-All embedding method to input the aligned motion feature into the current frame's estimation, thus remodeling the existing two-frame backbones. The proposed SplatFlow is efficient yet more accurate, as it can handle occlusions properly. Extensive experimental evaluations show that SplatFlow substantially outperforms all published methods on the KITTI2015 and Sintel benchmarks. Especially on the Sintel benchmark, SplatFlow achieves errors of 1.12 (clean pass) and 2.07 (final pass), with surprisingly significant 19.4% and 16.2% error reductions, respectively, from the previous best results submitted. The code for SplatFlow is available at https://github.com/wwsource/SplatFlow.
Keyword:
Optical flow estimation
Multi-frame optical flow estimation
Splatting
Occlusion
期刊
IF:
9.3
论文数:
4.0K
被引数:
2.8W
机构
引用论文
Judgment Capacity, Fear of Falling, and the Risk of Falls in Community-Dwelling Older Adults: The Progetto Veneto Anziani Longitudinal Study社区居住的老年人的判断能力，对跌倒的恐惧和跌倒的风险: Progetto Veneto Anziani纵向研究
OAAI
OAAI分享收藏First case report of post-operative infection due to Francisella tularensis after cardiac surgery
OAAI
OAAI分享收藏The role of BRCA status on prognosis in patients with triple-negative breast cancer.png)
OncotargetOAAI
OncotargetIF0
OAAI分享收藏Innate and Learned Predator Recognition Mediated by Chemical Signals in Eurycea nanaEthology
EthologyIF0
PREAI
分享收藏Role of fecal calprotection in predicting endoscopic recurrence in postoperative Crohn’s disease
PREAI
分享收藏
查看更多内容