.png){kind=icon}
返回
Local Information Assisted Attention-Free Decoder for Audio Captioning
.png){kind=icon}
OA
AI
DOI：10.1109/LSP.2022.3189536.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Automated audio captioning aims to describe audio data with captions using natural language. Existing methods often employ an encoder-decoder structure, where the attention-based decoder (e.g., Transformer decoder) is widely used and achieves state-of-the-art performance. Although this method effectively captures global information within audio data via the self-attention mechanism, it may ignore the event with short time duration, due to its limitation in capturing local information in an audio signal, leading to inaccurate prediction of captions. To address this issue, we propose a method using the pretrained audio neural networks (PANNs) as the encoder and local information assisted attention-free Transformer (LocalAFT) as the decoder. The novelty of our method is in the proposal of the LocalAFT decoder, which allows local information within an audio signal to be captured while retaining the global information. This enables the events of different duration, including short duration, to be captured for more precise caption generation. Experiments show that our method outperforms the state-of-the-art methods in Task 6 of the DCASE 2021 Challenge with the standard attention-based decoder for caption generation.
Keyword:
Decoding
Feature extraction
Wind forecasting
Interference
Convolution
Transformers
Task analysis
Automated audio captioning
local information
attention-free transformer
期刊
IF:
9.6
论文数:
1.1W
被引数:
1.7W
机构
引用论文
Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
.png){kind=icon}
PREAI
分享收藏分享收藏Role of Arg82 in the Early Steps of the Bacteriorhodopsin Proton-Pumping CycleArg82在细菌视紫红质质子泵循环的早期步骤中的作用
OAAI
OAAI分享收藏分享收藏Li6P6O18: X-ray powder structure determination of lithium cyclohexaphosphateLi6P6O18: 环磷酸锂的x射线粉末结构测定
PREAI
分享收藏
查看更多内容