.png){kind=icon}
.png){kind=icon}
Return
Automatic Audio Description: A Training-Free Approach Using Foundation Models
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-032-05060-1_15.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In this paper, we propose a training-free framework for generating audio descriptions (ADs) by leveraging large pretrained Video-Language Models (VLMs) and Large Language Models (LLMs) without task-specific fine-tuning. Our method enhances video understanding through a semantic-constrained prompting strategy that incorporates temporally coherent context into VLM inputs, while an adaptive character recognition module ensures consistent identity tracking across frames. By explicitly linking visual character observations to narrative elements, the system produces contextually rich and coherent visual descriptions. Finally, the video captions are then refined into a single, concise audio description sentence through a LLM operating exclusively on text inputs, ensuring clarity, brevity, and narrative cohesion. The experimental evaluation performed on the MAD-eval-Named and TVAD benchmarks, validates the approach achieving CIDEr scores of 23.2 and 23.4, respectively. Compared to state-of-the-art training-free baselines, our framework consistently yields relative improvements ranging from 3.6% to 8% across multiple evaluation metrics.
Keywords:
Audio description generation
video-language models
training-free methods
semantic-constrained prompting
Journal
C
IF:
0
Papers:
32
Citations:
0