Context-Aware Weakly Supervised Image Manipulation Localization With SAM Refinement

Xinghao Wang

doi:10.1109/LSP.2025.3592110

Return

Context-Aware Weakly Supervised Image Manipulation Localization With SAM Refinement

2025-01-01

PRE

DOI：10.1109/LSP.2025.3592110

Original

Original request for help

Save

Abstract

En 中文

Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods. Recent approaches in image manipulation detection have largely been driven by fully supervised approaches, which require labor-intensive pixel-level annotations. Thus, it is essential to explore weakly supervised image manipulation localization methods that only require image-level binary labels for training. However, existing weakly supervised image manipulation methods overlook the importance of edge information for accurate localization, leading to suboptimal localization performance. To address this, we propose a Context-Aware Boundary Localization (CABL) module to aggregate boundary features and learn context-inconsistency for localizing manipulated areas. Furthermore, by leveraging Class Activation Mapping (CAM) and Segment Anything Model (SAM), we introduce the CAM-Guided SAM Refinement (CGSR) module to generate more accurate manipulation localization maps. By integrating two modules, we present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture. Our method achieves outstanding localization performance across multiple datasets.

Keywords:

Context-aware boundary localization

image manipulation localization

weakly supervised