Compression artifact-aware incremental learning for neural network-based video coding

doi:10.1016/j.neucom.2026.133175

Return

Compression artifact-aware incremental learning for neural network-based video coding

2026-02-27

PRE

Qipu Qin

Cheolkon Jung *

DOI：10.1016/j.neucom.2026.133175

Original

Original request for help

Save

Abstract

En 中文

Compression artifacts are inherent in video coding and can significantly influence the training dynamics and performance of neural network-based video coding (NNVC) tools. In this paper, we propose compression artifact-aware incremental learning (CAIL) for NNVC tools based on consistent quantization parameter gap (QPG), called QPG-CAIL. Unlike conventional training approach for NNVC tools that employ raw uncompressed data as labels, QPG-CAIL takes data with higher compression artifacts as inputs and data with lower compression artifacts as labels, while keeping a consistent QPG between them. To this end, we construct multiple training datasets with varying distortion levels, including high, low, and no compression artifacts, to train the NNVC tool incrementally from small to large QPGs and comprehensively analyze their effects on the coding efficiency. To verify the effectiveness of QPG-CAIL in NNVC, we introduce a unified reference frame synthesis network (URFS-Net), which is integrated into the VVC between the decoded picture buffer (DPB) and reference picture lists (RPL) to generate neural reference frames for VVC inter prediction. Extensive experiments demonstrate that URFS-Net trained by the proposed QPG-CAIL achieves state-of-the-art performance in reference frame generation. For random access (RA)/low-delay B (LB) configurations, it provides average Bjøntegaard Delta rate (BD-rate) reductions of {4.64%/3.21% (Y), 5.15%/4.80% (U), 6.01%/6.17% (V)} over NNVC-12.0, and {6.44%/6.07% (Y), 12.80%/12.01% (U), 12.31%/11.49% (V)} over VTM-15.0.

Keywords:

compression artifacts

incremental learning

neural network-based video coding

quantization parameter gap

reference frame generation