Learning Dynamic Frame Semantics for Multimodal Sentiment Analysis

Zhuang, Jian

doi:10.1109/taffc.2026.3706491

Learning Dynamic Frame Semantics for Multimodal Sentiment Analysis

2026-07-01

0

PRE

AI

DOI：10.1109/taffc.2026.3706491

Original

Original request for help

Save

Abstract

En 中文

In this paper, we investigate how the mathematization of frame semantics can enhance multimodal sentiment analysis (MSA). Existing deep learning approaches to MSA typically focus on single semantic frames. In contrast, frame semantics, a well-recognized theory from linguistics, posits that language cognition and interpretation essentially rely on the coherence of multiple frames divided according to lexical units and encapsulated with diverse background knowledge and conceptual structures. To facilitate MSA with frame semantics, we propose a new model, namely Dynamic-Aware Frame Semantic Analysis (DAFSA). DAFSA first models multimodal sequential features (e.g., text, audio, and vision) as multimodal semantic frames. Then, DAFSA can extract the representations that capture the subtle emotional expressions by dynamically learning and summarizing the coherent structures from intra- and cross-modal frames. Experimental results demonstrate that DAFSA significantly outperforms state-of-the-art approaches to MSA and foundation models whose size is larger than DAFSA up to 100 & times;, showcasing the effectiveness of modeling frame semantics in MSA.

Keywords:

Modeling

Sentiment analysis

Visualization

Measurement units

Modules (abstract algebra)

Technology

Labeling

Conferences

Educational institutions

Learning (artificial intelligence)

Multimodal sentiment analysis

frame semantics

mathematical modeling

Learning Dynamic Frame Semantics for Multimodal Sentiment Analysis

Abstract

Journal

Organization

Cited Papers