arrow
Return

Task–Adapted Learnable Embedded Quantization for Scalable Human-Machine Image Compression

delete2025-01-01
delete0
PRE
AI
S
Shaohui Li
S
Shuoyu Ma
W
Wenrui Dai
N
Nuowen Kan
F
Fan Cheng
李
李成林 (Chenglin Li)
邹
邹君妮 (Junni Zou)
熊
熊红凯 (Hongkai Xiong)
DOI:10.1109/tcsvt.2025.3525664delete
deleteOriginal
deleteShare
deleteSave
Abstract

Abstract

En 中文
Image compression for both human and machine vision has become prevailing to accommodate to rising demands for machine-machine and human-machine communications. Scalable human-machine image compression is recently emerging as an efficient alternative to simultaneously achieve high accuracy for machine vision in the base layer and obtain high-fidelity reconstruction for human vision in the enhancement layer. However, existing methods achieve scalable coding with heuristic mechanisms, which cannot fully exploit the inter-layer correlations and evidently sacrifice rate-distortion performance. In this paper, we propose task-adapted learnable embedded quantization to address this problem in an analytically optimized fashion. We first reveal the relationship between the latent representations for machine and human vision and demonstrate that optimal representation for machine vision can be approximated with post-training optimization on the learned representation for human vision. On such basis, we propose task-adapted learnable embedded quantization that leverages learnable step predictor to adaptively determine the optimal quantization step for diverse machine vision tasks such that inter-layer correlations between representations for human and machine vision are sufficiently exploited using embedded quantization. Furthermore, we develop a human-machine scalable coding framework by incorporating the proposed embedded quantization into pre-trained learned image compression models. Experimental results demonstrate that the proposed framework achieves state-of-the-art performance on machine vision tasks like object detection, instance segmentation, and panoptic segmentation with negligible loss in rate-distortion performance for human vision.

Journal

IEEE Transactions on Circuits and Systems for Video Technology cover
IEEE Transactions on Circuits and Systems for Video Technology
IF:
11.1
Papers:
2.3K
Citations:
3.6W

Organization

Shanghai Jiao Tong University cover
Shanghai Jiao Tong University
Scholars:
1.3W
Papers: 3.9K
Citations: 8.7W
T
tsinghua shenzhen international graduate school
Scholars:
373
Papers: 159
Citations: 6.5K
Cited Papers

Cited Papers

End-to-End Learning of Compressible Features
err2020-10-01
err0
errOAAI
errSaurabh Singh; Sami Abu-El-Haija; Nick Johnston; Johannes Balle; Abhinav Shrivastava; George Toderici
errShare
errSave
Learned Progressive Image Compression With Spatial Autoregression
err2023-12-04
err0
PREAI
errTian,Wenxin; Li,Shaohui; Dai,Wenrui; Lu,Cewu; Hu,Weisheng; Zhang,Lin; Du,Junfeng; Xiong,Hongkai
errShare
errSave
Towards Coding for Human and Machine Vision: Scalable Face Image Coding
err2021-01-01
err39
PREAI
errYang, Shuai; Hu, Yueyu; Yang, Wenhan; Duan, Ling-Yu; Liu, Jiaying
errShare
errSave
An overhead-free region-based JPEG framework for task-driven image compression
err2023-01-01
err8
PREAI
errJeong, Seonghye; Jeong, Seongmoon; Woo, Simon S.; Ko, Jong Hwan
errShare
errSave
Nonlinear Transform Coding
err2021-02-01
err77
errOAAI
errBalle, Johannes; Chou, Philip A.; Minnen, David; Singh, Saurabh; Johnston, Nick; Agustsson, Eirikur; Hwang, Sung Jin; Toderici, George
errShare
errSave
Overview of the High Efficiency Video Coding (HEVC) Standard
err2012-01-01
err0
PREAI
errGary J. Sullivan; Jens-Rainer Ohm; Woo-Jin Han; Thomas Wiegand
errShare
errSave
Overview of the Versatile Video Coding (VVC) Standard and its Applications
err2021-01-01
err0
errOAAI
errBenjamin Bross; Ye-Kui Wang; Yan Ye; Shan Liu; Jianle Chen; Gary J. Sullivan; Jens-Rainer Ohm
errShare
errSave
researcher View more