"An informative research poster titled 'Generative Latent Coding for Ultra-Low Bitrate Image Compression' is displayed at the CVPR conference in Seattle, WA, from June 17-23, 2024. The poster, authored by Zhaoyang Jia, Jiahao Li, Bin Li, Houqiang Li, and Yan Lu from the University of Science and Technology of China and Microsoft Research Asia, outlines an innovative approach to image coding that leverages Generative Latent Coding (GLC) for effective image compression. The primary sections of the poster elaborate on the Introduction, Methods, and Results. The Introduction section discusses the significance of GLC in achieving higher texture realism and semantic consistency in image compression. The Methods section details the overall framework and progressive learning techniques for generative latent space and code-prediction-based loss. The Results section presents a visual comparison and rate-distortion curves, demonstrating the superior performance of GLC in saving 45% bits under the same FID compared to previous state-of-the-art methods. The poster also features illustrative diagrams and charts that elucidate the technical aspects and effectiveness of the proposed method."
Text transcribed from the image:
***
1958
ty of Science and Technology of
Introduction
Generative Latent Coding for Ultra-Low Bitrate Image Compression
Zhaoyang jia¹* Jiahao Li²
1University of Science and Technology of China 2Microsoft Research Asia
Bin Li² Houqiang Li¹ Yan Lu²
Compression
Space Mapping
Original
Pixel-Space
Compressed Results
⭑
Latent-Space
Compressed Results
Nearest Space
of Code 2
Code +1
DISTS = 0.20
SNR=20.3
DISTS = 0.09
SNR 21.8
Code +2
Pixel Space
Human Perception Space
Original Image
Generative Latent Space
Latent Space
Compression
Pixel Space
Compression
Methods
Analysis
Transform
Analysis
Analysis
Transform
Transformy
Pixel
Loss
Synthesis
Transform
Synthesis
Transform
Synthesis
Transform
Generative Latent Coding
Training Stage I
Training Stage II
Overall Framework & Progressive Learning
Original
Training Stage III
Text Sketch
0.029 bpp
Ours
0.023 bpp
Ours
0.036 bpp
Lower
Texture Realism?
Semantic Consistency?
Higher
We propose Generative Latent Coding (GLC)
for ultra-low bitrate image coding
•
Transform coding in VQVAE latent space
Using categorical hyper module
Code-prediction-based supervision
GLC achieves 45% bits saving under same
FID compared with previous SOTA
This work was done when Zhaoyang Jia was an intern at MSRA
ga
9s
ha
Code-Pred-based Latent Loss (Stage)
Code-Pred-based Supervision
Code-red-based
Supervision
MISE
Hyper
AE
VO-E Codebook C
Code-Pred-based Pixel Loss (Stage
Codebock
Aux Code
Predictor
hvao
Transform Coding in Latent Space
Other features:
.
Code-Pred-based
Supervision
Cress Entropy
Code-Prediction-Based loss
220
Variable bitrate in a single model
Restoration and style transfer applications
*
Results
CVPR
SEATTLE, WA JUNE 17-21, 2024
HAC
0.041 bpp
MS-ILLM
0.037 bpp
Original
Ours
0.038 bpp
WC/H.266
FCC (ICPR2021)
CelebAHO
Visual Comparison
--EVC (ICLR2023)
-- TCM (CVPR2023)
MS-ILLM (ICML20231
HIFIC (NeuriP52020)
CUC 2020
Text Sketch OCMW2023)
--GLC (Proposed)
Rate-Distortion Curves
029
411
188