.png){kind=icon}
返回
Graph Attention for Automated Audio Captioning
.png){kind=icon}
OA
AI
DOI：10.1109/LSP.2023.3266114.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
State-of-the-art audio captioning methods typically use the encoder-decoder structure with pretrained audio neural networks (PANNs) as encoders for feature extraction. However, the convolution operation used in PANNs is limited in capturing the long-time dependencies within an audio signal, thereby leading to potential performance degradation in audio captioning. This letter presents a novel method using graph attention (GraphAC) for encoder-decoder based audio captioning. In the encoder, a graph attention module is introduced after the PANNs to learn contextual association (i.e. the dependency among the audio features over different time frames) through an adjacency graph, and a top-k mask is used to mitigate the interference from noisy nodes. The learnt contextual association leads to a more effective feature representation with feature node aggregation. As a result, the decoder can predict important semantic information about the acoustic scene and events based on the contextual associations learned from the audio signal. Experimental results show that GraphAC outperforms the state-of-the-art methods with PANNs as the encoders, thanks to the incorporation of the graph attention module into the encoder for capturing the long-time dependencies within the audio signal.
Keyword:
Feature extraction
Decoding
Transformers
Semantics
Acoustics
Noise measurement
Matrix converters
Audio modelling
automated audio captioning
graph attention network
temporal information
期刊
IF:
9.6
论文数:
1.1W
被引数:
1.7W
机构
引用论文
.png){kind=icon}
分享收藏分享收藏Research on the Efficiency of Wireless Power Transfer System Based on Multi-Auxiliary Transmitting Coils基于多辅助发射线圈的无线电能传输系统效率研究
PREAI
分享收藏Role of Arg82 in the Early Steps of the Bacteriorhodopsin Proton-Pumping CycleArg82在细菌视紫红质质子泵循环的早期步骤中的作用
OAAI
OAAI分享收藏分享收藏
查看更多内容