.png){kind=icon}
返回
Automated Audio Captioning With Topic Modeling
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3235733.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Automatic audio captioning (AAC) is an important area of research aimed at generating meaningful descriptions for audio clips. Most existing methods use relevant semantic information to improve AAC performance and have demonstrated the feasibility of semantic information extraction. Audio events and keywords are commonly used for this purpose. Unlike previous studies, this study proposes a framework that uses topic modeling to obtain relevant semantic content since topic models explore the main themes of the documents. To this end, we present a framework that integrates audio embeddings with audio topics in a transformer-based encoder-decoder architecture. First, we represent each audio clip with a set of topics using a pre-trained topic model, BERTopic. Then, we design a multilayer perceptron (MLP)-based multi-label classifier to predict the topics of audio clips in the testing phase. Finally, in the proposed framework, we input audio embedding and extracted topics into the transformer model to generate captions. The results show that the proposed model improves performance and competes with the most advanced methods that utilize additional external data for training. We believe that the topic modeling can be used to extract semantic content in the AAC task.
Keyword:
Semantics
Bit error rate
Feature extraction
Transformers
Event detection
Audio systems
Predictive models
Audio captioning
audio event detection
PANNs
topic modeling
BERTopic
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
.png){kind=icon}
分享收藏分享收藏分享收藏Role of Arg82 in the Early Steps of the Bacteriorhodopsin Proton-Pumping CycleArg82在细菌视紫红质质子泵循环的早期步骤中的作用
OAAI
OAAI分享收藏
查看更多内容