.png){kind=icon}
.png){kind=icon}
Return
A Motion Distillation Framework for Video Frame Interpolation
.png){kind=icon}
PRE
AI
DOI：10.1109/TMM.2023.3314971.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In recent years, we have seen the success of deep video enhancement models. However, the performance improvement of new methods has gradually entered a bottleneck period. Optimizing model structures or increasing training data brings less and less improvement. We argue that existing models with advanced structures have not fully demonstrated their performance and demand further exploration. In this study, we statistically analyze the relationship between motion estimation accuracy and video interpolation quality of existing video frame interpolation methods, and find that only supervising the final output leads to inaccurate motion and further affects the interpolation performance. Based on this important observation, we propose a general motion distillation framework that can be widely applied to flow-based and kernel-based video frame interpolation methods. Specifically, we begin by training a teacher model, which uses the ground-truth target frame and adjacent frames to estimate motion. These motion estimates then guide the training of a student model for video frame interpolation. Our experimental results demonstrate the effectiveness of this approach in enhancing performance across diverse advanced video interpolation model structures. For example, after applying our motion distillation framework, the CtxSyn model achieves a PSNR gain of 3.047 dB.
Keywords:
Training
Computational modeling
Optical flow
Kernel
Interpolation
Motion estimation
Correlation
Deep learning
frame interpolation
knowledge distillation
optical flow
Journal
IF:
9.7
Papers:
4.5K
Citations:
2.4W