.png){kind=icon}
返回
Sentiment-Oriented Transformer-Based Variational Autoencoder Network for Live Video Commenting
.png){kind=icon}
OA
AI
DOI：10.1145/3633334.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Automatic live video commenting is getting increasing attention due to its significance in narration generation, topic explanation, etc. However, the diverse sentiment consideration of the generated comments is missing from current methods. Sentimental factors are critical in interactive commenting, and there has been lack of research so far. Thus, in this article, we propose a Sentiment-oriented Transformer-based Variational Autoencoder (So-TVAE) network, which consists of a sentiment-oriented diversity encoder module and a batch attention module, to achieve diverse video commenting with multiple sentiments and multiple semantics. Specifically, our sentiment-oriented diversity encoder elegantly combines a VAE and random mask mechanism to achieve semantic diversity under sentiment guidance, which is then fused with cross-modal features to generate live video comments. A batch attention module is also proposed in this article to alleviate the problem of missing sentimental samples, caused by the data imbalance that is common in live videos as the popularity of videos varies. Extensive experiments on Livebot and VideoIC datasets demonstrate that the proposed So-TVAE outperforms the state-of-the-art methods in terms of the quality and diversity of generated comments. Related code is available at https://github.com/fufy1024/So-TVAE.
Keyword:
Automatic live video commenting
multi-modal learning
variational autoencoder
batch attention mechanism
期刊
IF:
6
论文数:
2.0K
被引数:
5.4K
机构
引用论文
.png){kind=icon}
THE RESPONSE OF NATIVE STANDS OF LOWBUSH BLUEBERRY IN NEWFOUNDLAND TO NITROGEN, PHOSPHORUS, AND POTASSIUM FERTILIZERS纽芬兰地区野生低丛蓝莓对氮、磷和钾肥的响应
PREAI
分享收藏分享收藏
查看更多内容