.png){kind=icon}
返回
LVCD: Reference-based Lineart Video Colorization with Diffusion Models
.png){kind=icon}
OA
AI
DOI：10.1145/3687910.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
We propose the first video diffusion framework for reference-based lineart video colorization. Unlike previous works that rely solely on image generative models to colorize lineart frame by frame, our approach leverages a large-scale pretrained video diffusion model to generate colorized animation videos. This approach leads to more temporally consistent results and is better equipped to handle large motions. Firstly, we introduce Sketch-guided ControlNet which provides additional control to finetune an image-to-video diffusion model for controllable video synthesis, enabling the generation of animation videos conditioned on lineart. We then propose Reference Attention to facilitate the transfer of colors from the reference frame to other frames containing fast and expansive motions. Finally, we present a novel scheme for sequential sampling, incorporating the Overlapped Blending Module and Prev-Reference Attention, to extend the video diffusion model beyond its original fixed-length limitation for long video colorization. Both qualitative and quantitative results demonstrate that our method significantly outperforms state-of-the-art techniques in terms of frame and video quality, as well as temporal consistency. Moreover, our method is capable of generating high-quality, long temporal-consistent animation videos with large motions, which is not achievable in previous works. Our code and model are available at https://luckyhzt.github.io/lvcd.
Keyword:
Lineart video colorization
Diffusion Models
Animation
期刊
IF:
9.5
论文数:
4.7K
被引数:
3.6W
机构
引用论文
First case report of post-operative infection due to Francisella tularensis after cardiac surgeryOAAI
OAAI分享收藏Role of Sodium Ions in the Response of the Frequency of Miniature End-plate Potentials to Osmotic Changes in the Neuromuscular Junction.png)
Nature
NatureIF0
PREAI
分享收藏Sensorless Control of Z Source Inverter fed BLDC Motor Drive by FOC - DTC Hybrid Control Strategy Using Fuzzy Logic Controller采用模糊逻辑控制器的foc-dtc混合控制策略的Z源逆变器馈电BLDC电机驱动的无传感器控制
PREAI
分享收藏
分享收藏
查看更多内容