.png){kind=icon}
返回
Integrated feature-enhanced multimodal intent detection method
.png){kind=icon}
PRE
AI
DOI：10.1007/s13042-025-02876-7.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multimodal intent detection is an important task for understanding human language in real-world multimodal scenarios. Previous intent detection methods rely on textual information, ignoring the potential connections between different modalities, and face the challenge of efficiently extracting semantic features from non-linguistic modalities. In response to these challenges, we propose the MIMO model, which employs a multimodal feature extraction and processing approach to address the complexity of text, video, and audio data. In our model, we employ a feature selection flow (SwinVTRSFlow) to process video features, reduce feature dimensionality, and improve model performance. To establish an optimal multimodal semantic environment for text modalities, we introduce Perceptual Fusion Representation (PFR). PFR efficiently aligns and fuses the features of different modalities and utilizes intent tags to obtain optimal textual semantic insights, guiding the learning process of other modalities. Experiments show that our video and audio feature enhancement and transformation approach significantly outperforms the baseline, improving the processing efficiency and performance of multimodal data. These improvements are crucial for multimodal intent detection tasks.
Keyword:
Intent detection
Multimodal
Feature-enhanced
Perceptual fusion representation
期刊
IF:
2.7
论文数:
3.2K
被引数:
5.6K
机构
引用论文
Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition基于模态感知提示的Token级对比学习用于多模态意图识别
OAAI
OAAI分享收藏Multitask learning for multilingual intent detection and slot filling in dialogue systems.png)
INFORMATION FUSION
OAAI
INFORMATION FUSIONIF15.5
OAAI分享收藏分享收藏
查看更多内容