Contrastive Learning based Multi-task Network for Image Manipulation Detection

Qilin Yin

doi:10.1016/j.sigpro.2022.108709

2022-12-01

PRE

DOI：10.1016/j.sigpro.2022.108709

原文链接

原文求助

摘要

En 中文

The popularity of image editing techniques and user-friendly editing software have seriously reduced the authenticity of the images. Detection and localization of image manipulations are becoming urgent problems to be solved. Although many existing solutions attempt to address these problems, most works can only solve one specific type of manipulations. Furthermore, some methods need heavy, time-consuming preprocessings and/or postprocessings to localize tampered region, resulting in disconnection and under-optimization of the model. In this paper, a contrastive learning based multi-task network is proposed for the localization of multiple image manipulations. Multi-scale tampered patch classifications and pixelwise tampered region semantic segmentation are integrated into an end-to-end multi-task network. The consistency of different region statistical properties is measured by contrastive learning to enhance the feature representation ability of the proposed network, improving the performance of tampered patch detection. Various scale tampered patch detections cooperate to localize the tampered region boundaries from coarse to fine. Prediction Pyramid composed of different scale patch detection results provides comprehensive guidance for pixel-wise semantic segmentation of the tampered region. Experimental results on four standard image manipulation datasets demonstrate the better performance of the proposed model. (C) 2022 Elsevier B.V. All rights reserved.

Keyword:

Image manipulation forensics

Multi-task network

Contrastive learning

Prediction pyramid

Semantic segmentation