CaPTURe: Cartoon Pose Transfer Using Reverse Attention

doi:10.1016/j.neucom.2023.126619

2023-10-01

PRE

Chang-Hyeon An

Hyun‐Chul Choi *

DOI：10.1016/j.neucom.2023.126619

原文链接

原文求助

摘要

En 中文

Most of the previous pose transfer methods require additional input data such as joint keypoints of the target pose extracted by a pre-trained network in a human domain. However, in the fields where cartoon characters are used, such as animation or webtoons, the body proportions and structures of the characters often deviate from those of real-life humans. Therefore, it is not appropriate to utilize a pre-trained network designed for the human domain to extract additional data from these cartoon characters. Even if the network is newly trained in the cartoon domain, expensive data labeling is necessary. As a result, most of the previous pose transfer methods are not suitable for application in the cartoon domain. To address these issues, we propose a cartoon pose transfer network named CaPTURe that can successfully perform pose transfer in the cartoon domain with only target images and no other input data. Here, we incorporate the attention mechanism to accurately generate the desired identity. Additionally, we employ reverse attention to enhance the precision of generating the target pose. This approach eliminates the influence of identity information in the pose feature, enabling our network to focus solely on utilizing pose information. Consequently, through comparative experiments using a cartoon domain dataset, CaPTURe shows significant improvements in quantitative results over the previous state-of-the-art methods. Specifically, it achieves a 4.7% reduction in L1 Distance, a 0.1% improvement in SSIM, and a 0.8% enhancement in LPIPS. Moreover, CaPTURe exhibits superior qualitative performance than the previous state-of-the-art methods. In addition, we demonstrate that CaPTURe is capable of achieving effective pose transfer not just in the cartoon domains, but also in the human domains, as evidenced by our experiments on a human domain dataset.

Keyword:

Generative Adversarial Network

Cartoon pose transfer

Attention

Reverse attention