.png){kind=icon}
返回
Multi-scale deep feature fusion based sparse dictionary selection for video summarization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.image.2023.117006.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The explosive growth of video data constitutes a series of new challenges in computer vision, and the function of video summarization (VS) is becoming more and more prominent. Recent works have shown the effectiveness of sparse dictionary selection (SDS) based VS, which selects a representative frame set to sufficiently reconstruct a given video. Existing SDS based VS methods use conventional handcrafted features or single-scale deep features, which could diminish their summarization performance due to the underutilization of frame feature representation. Deep learning techniques based on convolutional neural networks (CNNs) exhibit powerful capabilities among various vision tasks, as the CNN provides excellent feature representation. Therefore, in this paper, a multi-scale deep feature fusion based sparse dictionary selection (MSDFF-SDS) is proposed for VS. Specifically, multi-scale features include the directly extracted features from the last fully connected layer and the global average pooling (GAP) processed features from intermediate layers, then VS is formulated as a problem of minimizing the reconstruction error using the multi-scale deep feature fusion. In our formulation, the contribution of each scale of features can be adjusted by a balance parameter, and the row-sparsity consistency of the simultaneous reconstruction coefficient is used to select as few keyframes as possible. The resulting MSDFF-SDS model is solved by using an efficient greedy pursuit algorithm. Experimental results on two benchmark datasets demonstrate that the proposed MSDFF-SDS improves the F-score of keyframe based summarization more than 3% compared with the existing SDS methods, and performs better than most deep-learning methods for skimming based summarization.
Keyword:
Video summarization
Keyframe
Sparse coding
Dictionary selection
Multi-scale
期刊
S
IF:
2.7
论文数:
2.8K
被引数:
4.2K
机构
引用论文
分享收藏
Preparation and microwave dielectric properties of BPO4 ceramics with ultra-low dielectric constant
PREAI
分享收藏Effects of Novel Isoform-Selective Phosphoinositide 3-Kinase Inhibitors on Natural Killer Cell Function.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏
查看更多内容