.png){kind=icon}
返回
Action Segmentation through Self-Supervised Video Features and Positional-Encoded Embeddings
.png){kind=icon}
OA
AI
DOI：10.1145/3649465.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Action segmentation consists of temporally segmenting a video and labeling each segmented interval with a specific action label. In this work, we propose a novel action segmentation method that requires no initial video analysis and no annotated data. Our proposal involves extracting features from videos using several pre-trained deep-learning models, including spatiotemporal and self-supervised methods. Data is then transformed using a positional encoder, and finally, a clustering algorithm is applied, where each produced cluster presumably corresponds to a different single and distinguishable action. For self-supervised features, we explored DINO, and for spatiotemporal features, we investigated I3D and SlowFast methods. Moreover, two different clustering algorithms (FINCH and KMeans) were investigated, and we also explored how varying the length of video snippets that generate the feature vectors affected the quality of the segmentation. Experiments show that our method produces competitive results on the Breakfast and INRIA Instructional Videos dataset benchmarks. Our best result was produced using a composition of self-supervised features generated by DINO, FINCH clustering, and positional encoding.
Keyword:
Neural networks
video understanding
action segmentation
clustering
video understanding
action segmentation
clustering
期刊
IF:
6
论文数:
2.0K
被引数:
5.4K
机构
暂无机构信息
引用论文
Metabolism of an oxysterol, 7‐ketocholesterol, by sterol 27‐hydroxylase in hepG2 cells.png)
Lipids
LipidsIF0
PREAI
分享收藏分享收藏Digital image forensic approach based on the second-order statistical analysis of CFA artifacts
PREAI
分享收藏
查看更多内容