AU-Guided Neural Prototype Trees for Interpretable Micro-Expression Recognition

Xuewei Sun

doi:10.1109/taffc.2026.3718575

AU-Guided Neural Prototype Trees for Interpretable Micro-Expression Recognition

2026-07-30

0

PRE

AI

DOI：10.1109/taffc.2026.3718575

Original

Original request for help

Save

Abstract

En 中文

Micro-expressions (MEs) are unconscious, brief facial movements that can reveal concealed emotions. However, automatic ME recognition faces two fundamental bottlenecks: the tendency of standard black-box models to overfit to domain-specific artifacts under extreme data scarcity, and the limited transparency of their reasoning. To address the critical necessity for generalized and robust interpretability, this paper proposes a unified, end-to-end framework: the Action Unit-Guided Prototypical Tree (AU-ProtoTree). The framework solves these issues by integrating AU-guided motion and spatial priors across three levels. First, at the signal level, an AU-guided weakly-supervised strategy refines an optical flow network to filter domain noise and amplify diagnostically relevant motion. Second, at the feature level, a parallel dual-attention Vision Transformer (ViT) balances data-driven learning with motion-saliency priors for robust feature extraction. Third, at the decision level, the AU-anchored Deformable Prototype Tree (ADPT) eschews black-box mappings, modeling recognition as an interpretable hierarchical path where each node compares the input with a deformable prototype spatially regularized toward an AU-associated facial region. The framework achieves state-of-the-art cross-domain accuracies of 86.11%, 83.43%, and 87.86% on the CASME II, SAMM, and SMIC-HS datasets, respectively.

Keywords:

Action unit

interpretability

micro-expression recognition

prototypical tree

AU-Guided Neural Prototype Trees for Interpretable Micro-Expression Recognition

Abstract

Journal

Organization

Cited Papers