.png){kind=icon}
返回
U-Transformer-based multi-levels refinement for weakly supervised action segmentation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2023.110199.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Action segmentation is a research hotspot in human action analysis, which aims to split videos into segments of different actions. Recent algorithms have achieved great success in modeling based on temporal convolution, but these methods weight local or global timing information through additional modules, ignoring the existing long-term and short-term information connections between actions. This paper proposes a U-Transformer structure based on multi-level refinement, introduces neighborhood attention to learn the neighborhood information of adjacent frames, and aggregates video frame features to effectively process long-term sequence information. Then a loss optimization strategy is proposed to smooth the original classification effect and generate a more accurate calibration sequence by introducing a pairing similarity optimization method based on deep feature learning. In addition, we propose a timestamp supervised training method to generate complete information for actions based on pseudo-label predictions for action boundary predictions. Experiments on three challenging action segmentation datasets, 50Salads, GTEA, and Breakfast, show that our model performs state-of-the-art models, and our weakly supervised model also performs comparably to fully supervised performance.
Keyword:
Action segmentation
U-Transformer
Timestamp supervision
Multi-stages refinement
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Maximization and restoration: Action segmentation through dilation passing and temporal reconstruction最大化和恢复: 通过扩张传递和时间重建进行动作分割.png)
PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏
查看更多内容