CPI-Parser: Integrating Causal Properties Into Multiple Human Parsing

doi:10.1109/TIP.2024.3469579

Return

CPI-Parser: Integrating Causal Properties Into Multiple Human Parsing

2024-01-01

PRE

Xuanhan Wang

Xiaojia Chen

Lianli Gao *

Jingkuan Song (Jingkuan Song)

Hengtao Shen

DOI：10.1109/TIP.2024.3469579

Original

Original request for help

Save

Abstract

En 中文

Existing methods of multiple human parsing (MHP) apply deep models to learn instance-level representations for segmenting each person into non-overlapped body parts. However, learned representations often contain many spurious correlations that degrade model generalization, leading learned models to be vulnerable to visually contextual variations in images (e.g., unseen image styles/external interventions). To tackle this, we present a causal property integrated parsing model termed CPI-Parser, which is driven by fundamental causal principles involving two causal properties for human parsing (i.e., the causal diversity and the causal invariance). Specifically, we assume that an image is constructed by a mix of causal factors (the characteristics of body parts) and non-causal factors (external contexts), where only the former ones decide the essence of human parsing. Since causal/non-causal factors are unobservable, the proposed CPI-Parser is required to separate key factors that satisfy the causal properties from an image. In this way, the parser is able to rely on causal factors w.r.t relevant evidence rather than non-causal factors w.r.t spurious correlations, thus alleviating model degradation and yielding improved parsing ability. Notably, the CPI-Parser is designed in a flexible way and can be integrated into any existing MHP frameworks. Extensive experiments conducted on three widely used benchmarks demonstrate the effectiveness and generalizability of our method. Code and models are released (https://github.com/HAG-uestc/CPI-Parser) for research purpose.

Keywords:

Correlation

Semantics

Context modeling

Representation learning

Image segmentation

Degradation

Visualization

Pipelines

Image reconstruction

Data models

Multiple human parsing