# GroupedMixer: An Entropy Model with
Group-wise Token-Mixers
for Learned Image Compression

Daxin Li, Yuanchao Bai, Kai Wang, Junjun Jiang,
Xianming Liu, Wen Gao This work was supported in part by National Key Research
and Development Program of China under Grant 2022YFF1202104, in part by
National Natural Science Foundation of China under Grants 62301188, 92270116,
62071155 and U23B2009, in part by the Strategic Research, and Consulting
Project by the Chinese Academy of Engineering under Grant 2023-XY-39, in part
by China Postdoctoral Science Foundation under Grant 2022M710958, and in part
by Heilongjiang Postdoctoral Science Foundation under Grant LBH-Z22156. Daxin
Li, Yuanchao Bai, Kai Wang, Junjun Jiang and Xianming Liu are with the School
of Computer Science and Technology, Harbin Institute of Technology, Harbin,
150001, China, E-mail: {hahalidaxin<EMAIL_ADDRESS>{yuanchao.bai,
csxm<EMAIL_ADDRESS>Wen Gao is with the School of Electronics
Engineering and Computer Science, Peking University, Beijing, 100871, China,
E-mail<EMAIL_ADDRESS>
(Corresponding author: Xianming Liu)

###### Abstract

Transformer-based entropy models have gained prominence in recent years due to
their superior ability to capture long-range dependencies in probability
distribution estimation compared to convolution-based methods. However,
previous transformer-based entropy models suffer from a sluggish coding
process due to pixel-wise autoregression or duplicated computation during
inference. In this paper, we propose a novel transformer-based entropy model
called GroupedMixer, which enjoys both faster coding speed and better
compression performance than previous transformer-based methods. Specifically,
our approach builds upon group-wise autoregression by first partitioning the
latent variables into groups along spatial-channel dimensions, and then
entropy coding the groups with the proposed transformer-based entropy model.
The global causal self-attention is decomposed into more efficient group-wise
interactions, implemented using inner-group and cross-group token-mixers. The
inner-group token-mixer incorporates contextual elements within a group while
the cross-group token-mixer interacts with previously decoded groups.
Alternate arrangement of two token-mixers enables global contextual reference.
To further expedite the network inference, we introduce context cache
optimization to GroupedMixer, which caches attention activation values in
cross-group token-mixers and avoids complex and duplicated computation.
Experimental results demonstrate that the proposed GroupedMixer yields the
state-of-the-art rate-distortion performance with fast compression speed.

###### Index Terms:

Entropy Model, Transformer, Lossy Image Compression

††publicationid: pubid: Copyright © 2024 IEEE. Personal use of this material
is permitted. However, permission to use this material for any other purposes
must be obtained from the IEEE by sending an email to pubs-
<EMAIL_ADDRESS>
##  I Introduction

Lossy image compression is a long-standing and vital research topic in signal
processing and computer vision fields, which has been widely used to reduce
the cost of storage and transmission. Due to the rapidly increasing user
demand for online sharing and communication via prevalent applications such as
WeChat, Twitter, etc., the number of images on the Internet has grown
exponentially. Considering the enormous amount of image data nowadays, better
compression methods are constantly sought to preserve higher fidelity under
the desired bitrate constraint. On the other hand, a better compression method
should also be computationally efficient, which enables high-throughput
processing.

With the rapid advancement of deep learning, end-to-end optimized lossy image
compression has made remarkable progress over the last few years [1, 2, 3, 4,
5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22]. Currently,
state-of-the-art (SOTA) learning-based methods [16, 17, 7, 8, 23, 10] have
achieved superior performance to that of traditional compression methods, such
as VVC [24], which implies a promising future for learned image compression.
Most learned image compression methods are based on an auto-encoder framework.
The original input image is mapped to latent representations using nonlinear
transforms and then quantized to discrete values. The discrete latent
variables are encoded by arithmetic coding tools with the probability
distribution estimated by an entropy model. As shown in [2], the final bitrate
to losslessly compress the latent variables is the cross entropy between the
prior and estimated distribution of the latent variables. It indicates that
more accurate estimation leads to fewer bits required to compress an image,
which motivates researchers to design more powerful entropy models.

Recently, in light of the success of Transformer in natural language
processing [25], many researchers [23, 26, 9, 10] have adopted transformer
architectures in image compression field. Transformer-based entropy models,
such as Entroformer [9] and Contextformer [10], have gained prominence due to
their superior ability to capture long-range dependencies in probability
distribution estimation compared to CNNs. Initially, the latent variables are
rearranged into a token sequence. Then, it is proposed to aggregate contexts
with a stack of masked transformer decoder blocks, yielding large receptive
fields, as well as content-adaptive transform. However, previous transformer-
based entropy models still have limitations. As illustrated in Table I,
Entroformer [9] only considers spatial correlation and suffers from a spatial
serial decoding process. Its parallel variant Entroformer-P reduces decoding
steps to 2, but the compression performance drops significantly. Compared to
Entroformer, Contextformer [10] further takes channel-wise correlation into
account, but suffers from a even more complex spatial-channel serial coding
process. Moreover, both Entroformer and Contextformer overlook redundant
context computation in the attention layer during inference. All these factors
make previous transformer-based entropy models time-consuming.

TABLE I: Comparing GroupedMixer with other entropy models: “Conv.” and “Trans.” refer to the use of convolutional modules and transformer modules in entropy model. “Sp.” and “ch.” denote spatial and channel contexts. “G” is the group count, and “Dec.” represents average encoding/decoding time for a $768\times 512$ image, “Share” denotes that building different autoregressive steps and mining different types of correlations with a set of shared parameters.  Method | Type | Context | $G$ | Dec.(<1s) | Share
---|---|---|---|---|---
Autoregressive [5] | Conv. | sp. | $HW$ | ✗ | ✔
Group-based Entropy Model |
Checkerboard [6] | Conv. | sp. | 2 | ✔ | ✔
ChARM [7] | Conv. | ch. | 10 | ✔ | ✗
ELIC [8] | Conv. | sp.+ch. | 10 | ✔ | ✗
TCM [11] | Trans. | sp.+ch. | 10 | ✔ | ✗
MLIC [12, 13] | Trans. | sp.+ch. | 20 | ✔ | ✗
Autoregressive Transformer |
Entroformer [9] | Trans. | sp. | $HW$ | ✗ | ✔
Entroformer-P [9] | Trans. | sp. | 2 | ✗ | ✔
Contextformer [10] | Trans. | sp.+ch. | $4HW$ | ✗ | ✔
Group-wise Autoregressive Transformer |
GroupedMixer | Trans. | sp.+ch. | 40 | ✔ | ✔
GroupedMixer-Fast | Trans. | sp.+ch. | 10 | ✔ | ✔

In this paper, we introduce a novel transformer-based entropy model with
group-wise token-mixers, dubbed GroupedMixer, to surmount the aforementioned
challenges. In GroupedMixer, we first partition the latent variables into
groups along spatial and channel dimensions. Then, we employ two group-wise
token-mixers, _i.e._ , inner-group and cross-group token-mixers to integrate
contextual information within each group and across previously decoded groups
respectively. Moreover, we carefully design position encodings for the
proposed group-wise token-mixers to provide spatial-channel positional
information. Alternate arrangement of these two token-mixers enables global
contextual reference. We compare our approach with previous entropy models [5,
6, 7, 8, 9, 10, 11, 12, 13] to distinguish it from various aspects in Table I.
Firstly, our approach explores spatial-channel correlations existing in latent
variables, as contrast to models focusing solely on spatial or channel
dimensions [6, 7, 9]. Secondly, our method employs a transformer architecture
for global referencing instead of convolution [5, 6, 7, 8], implemented with
decomposed attention mechanism with lower computation rather than full
attention [9, 10]. Thirdly, relying on group-wise autoregression instead of
pixel-wise autoregression [5, 9, 10], our approach only requires a constant
number of inferences during coding. Fourthly, in contrast to previous group-
based entropy models [7, 8, 11, 12, 13], our method shares weights across
different group predictions. This approach enables more effective modeling of
complex correlations. Additionally, we introduce context cache optimization to
GroupedMixer to further accelerate the network inference, which caches
attention activation values in cross-group token-mixer and avoids duplicated
computation of contextual information. The context cache optimization not only
enables acceleration of coding speed to under 1 second, but also effectively
reduces multi-step fine-tuning time, which eliminates train-test gap and
further boosts our performance. Experimental results demonstrate that our
proposed GroupedMixer achieves SOTA RD performance, while maintaining fast
coding speed.

The contributions of our work are summarized as follows:

  * •

We present an entropy model named as GroupedMixer, a group-wise autoregressive
transformer, which globally explores spatial-channel redundancies
simultaneously. To lower the computation complexity, we propose to decompose
the causal self-attention into two group-wise token-mixers, namely, inner-
group and cross-group token-mixers.

  * •

To further accelerate the network inference, we introduce context cache
optimization to GroupedMixer, which enables faster coding speed and also can
be applied to reduce multi-step fine-tuning time.

  * •

With the help of the proposed entropy model and accelerating technique, we can
achieve SOTA compression performance with fast compression speed.
Specifically, we achieve 17.84%, 19.77% and 22.56% BD-Rate savings on Kodak,
CLIC’21 Test and Tecnick datasets when compared to VVC.

The rest of this paper is organized as follows. Section II reviews related
works. Section III presents the proposed transformer-based entropy model.
Experimental results are shown in Section IV and concluding remarks are given
in Section V.

##  II Related Work

###  II-A Learned Image Compression

Learning-based image compression has witnessed remarkable advances in recent
years, including lossless image compression [27, 28, 29, 30], near-lossless
image compression [31] and lossy image compression [1, 2, 5]. Learning-based
lossy image compression originated from the work of Toderici _et al._ [1].
Ballé _et al._ presented an end-to-end optimized framework based on CNN in
[2], which can be formulated as variational auto-encoder (VAE) [32]. Jiang _et
al._ [4] innovate with a convolutional neural network based end-to-end
compression framework. Guo _et al._ [20] present a novel causal context model
based on CNNs to capture spatial-channel correlations in latent spaces. Wang
_et al._ [22] explore rate-distortion optimization in image compression
through ensemble learning. Wu _et al._ [17] introduce a block-based hybrid
image compression framework, enhancing efficiency and reducing memory issues
through novel prediction and post-processing techniques. Finally, Bao _et al._
[21] reveal a novel perspective on nonlinear transforms in learned image
compression, proposing a nonlinear modulation-like transform inspired by
communication techniques. In contrast to the papers listed, this study
introduces GroupedMixer, a novel transformer-based entropy model for learned
image compression. This model leverages group-wise autoregression and
decomposes full self-attention into more efficient group-wise token-mixers.
Additionally, it incorporates context cache optimization for faster inference.
Our compression model adapts VAE framework as in [2]. In this framework, given
an input natural image $\bm{x}$ in distribution $p_{x}$, an analysis transform
$g_{a}$ is applied to decorrelate it into the compact latent representations
$\bm{y}$. The latent variables $\bm{y}$ are then quantized to discrete values
$\bm{\hat{y}}$ and further losslessly compressed using entropy coding
techniques with an estimated distribution $p_{\hat{\bm{y}}}(\hat{\bm{y}})$. At
the decoder side, the received quantized latent variables $\bm{\hat{y}}$ are
recovered to a reconstruction $\bm{\hat{x}}$ using a synthesis transform
$g_{s}$. We formulate the above process as:

$\displaystyle\bm{y}$ $\displaystyle=g_{a}(\bm{x};\bm{\phi}),$ (1)
$\displaystyle\bm{\hat{y}}$ $\displaystyle=Q(\bm{y}),$
$\displaystyle\bm{\hat{x}}$ $\displaystyle=g_{s}(\bm{\hat{y}};\bm{\theta}),$

where $\phi$ and $\theta$ denote the parameters of analysis transform and
synthesis transform respectively. In learned image compression, the network
can be trained with a loss which is a trade-off between rate and distortion as
follows:

$\displaystyle R+\lambda\cdot D$ $\displaystyle=\mathbb{E}_{\bm{x}\sim
p_{\bm{x}}}\left[-\log_{2}p_{\hat{\bm{y}}}(\hat{\bm{y}})\right]+\lambda\cdot\mathbb{E}_{\bm{x}\sim
p_{\bm{x}}}[d(\bm{x},\hat{\bm{x}})],$ (2)

where the first term denotes the rate of the latent variables, the second term
measures the reconstruction quality with a predefined distortion metric $d$
(_e.g._ , PSNR and MS-SSIM), and $\lambda$ is a Lagrange multiplier which
controls the RD trade-off during training.

###  II-B Entropy Model

Entropy model plays a crucial role in estimating the prior distribution of
quantized latent variables $p_{\hat{\bm{y}}}(\hat{\bm{y}})$. According to [2],
more accurate estimation leads to lower bitrate. To capture general structural
information in latent variables, Ballé _et al._ [2] introduced hyperprior
$\bm{z}$, which is additionally encoded as side information to estimate
distribution of $\hat{\bm{y}}$. Minnen _et al._ [5] proposed a context model
based on masked convolution layer to exploit spatial redundancies in latent
variables. Thus, equipped with hyperprior and context model, rate term in (2)
can be reformulated as:

$\displaystyle R=\mathbb{E}_{\bm{x}\sim
p_{\bm{x}}}\left[\sum_{i}-\log_{2}p_{\hat{y}_{i}\mid\hat{\bm{z}},\hat{\bm{y}}_{<i}}(\hat{y}_{i}\mid\hat{\bm{z}},\hat{\bm{y}}_{<i};\bm{\psi})\right],$
(3)

where $\psi$ is the parameter of entropy model. Since then, many works have
been proposed to improve the context model. We summarize prior efforts related
to ours as following two main categories:

####  II-B1 Global Prediction

Several studies have expanded the context model’s receptive field beyond local
areas. Guo _et al._ [20] split latent representations along the channel
dimension, using global prediction for the second group based on similarity
calculations with the first group’s elements. Entroformer [9] developed a
transformer-based context model leveraging global spatial correlations and
accelerated it using a checkerboard-like parallelization. Contextformer [10]
also considered channel-wise correlations. In these transformer-based methods,
latent variables are initially rearranged into token sequences. The
introduction of stacked masked transformer decoder blocks in place of masked
convolution layers offers global receptive fields and context-adaptive
transformation. Stacking transformer blocks enhances feature collection and
improves prediction accuracy.

Figure 1:  (a) GroupedMixer overview. Preprocessed latent representations
$\bm{\hat{y}}$ are then passed through GroupedMixer modules to aggregate
group-wise context, and are finally projected as distribution parameters. (b)
Illustration of token-mixers. Cross-group token-mixer mixes the information
between previously decoded groups, while inner-group token-mixer mixes the
information within groups. (c) Detailed network architectures of two token-
mixers, where MSA represents multi-head self-attention, and PEG denotes
position embedding generator.

####  II-B2 Group-based Autoregression

Spatial autoregression, traditionally requiring serial coding, is accelerated
through group-based methods. These methods divide latent variables into k
groups across spatial-channel dimensions for parallelized coding within
groups, leading to k-step inference. He _et al._ [6] spatially split latent
variables into two groups, employing checkerboard-like masked convolution for
a two-step parallel model. ChARM [7] segmented latent variables into k channel
slices, introducing a channel-wise context model. ELIC [8] developed an
unevenly grouped channel-conditional model combined with a checkerboard
spatial context, effectively creating spatial-channel group-based models.
However, these methods, often reliant on convolutional layers, are limited to
local receptive fields and struggle with long-range dependencies. Recent
studies have concentrated on enhancing group-wise autoregression approaches
with transformer modules. TCM [11] improved the predictive power of the ChARM
network by integrating Swin-Transformer blocks. MLIC [12] was designed to
capture channel-wise features using CNNs and to address both local and global
spatial correlations through masked transformer modules. An advancement over
MLIC [12], MLIC++ [13] further improved upon this by altering the network
architecture and incorporating linear attention computation.

It’s important to note that previous group-based models, including TCM [11]
and MLIC [12, 13], used various sets of CNNs and transformer modules to model
each conditional distribution in the autoregressive model, differing
fundamentally from our approach. Our entropy model, GroupedMixer, is a group-
wise autoregressive transformer. This means that GroupedMixer shares weights
across predictions of different groups by exploiting common predictive
patterns. This shared-weight approach not only streamlines the model but also
allows GroupedMixer to more effectively model complex correlations.

Figure 2: Grouping scheme for modeling spatial-channel context. The latent
representations are separated along channel and spatial dimensions into
$G=k_{c}\cdot k_{h}\cdot k_{w}$ groups sequentially, and number indicates
order of autoregression. In this figure, we use $(k_{c},k_{h},k_{w})=(2,2,2)$
as an example.

##  III Proposed Method

Given an input image $\bm{x}$, we employ an analysis transform $g_{a}$ to
decorrelate it into the latent variables $\bm{y}$. The latent variables
$\bm{y}$ are then quantized to discrete values $\bm{\hat{y}}$ and losslessly
compressed using entropy coding techniques, such as arithmetic coding, based
on an estimated distribution. At the decoder side, the received quantized
latent variables $\bm{\hat{y}}$ are reconstructed to $\bm{\hat{x}}$ using a
synthesis transform $g_{s}$. The latent variables are assumed to follow a
single Gaussian distribution $\mathcal{N}(\bm{\mu},\bm{\sigma})$ and the
parameters $\bm{\mu},\bm{\sigma}$ are jointly estimated with a hyperprior
model and our proposed entropy model, GroupedMixer. In the following, we
overview the network architecture of the proposed GroupedMixer and specify the
details of the group-wise token-mixers.

###  III-A Overview of GroupedMixer

We first introduce the network architecture of the proposed entropy model
GroupedMixer, which is illustrated in Figure 1a. Given a quantized latent
representation $\hat{\bm{y}}\in\mathbb{R}^{H\times W\times C}$, we first
separate it into a constant number of groups along both spatial and channel
dimensions. We evenly split the latent variables into $k_{c}$ slices along the
channel dimension, and then each channel slice is spatially segmented into
$k_{h}\times k_{w}$ groups using a spatial splitting scheme, as illustrated in
Figure 2. We define the group partition function $f$ and its inverse ungroup
function $f^{-1}$ as follows:

$\displaystyle\begin{aligned} f&:\mathbb{R}^{H\times W\times
C}\rightarrow\mathbb{R}^{G\times hw\times c},\\\ f^{-1}&:\mathbb{R}^{G\times
hw\times c}\rightarrow\mathbb{R}^{H\times W\times C},\end{aligned}$ (4)

where the number of group is $G=k_{c}\cdot k_{h}\cdot k_{w}$ and the size of
each group is $hw\times
c,h=\frac{H}{k_{h}},w=\frac{W}{k_{w}},c=\frac{C}{k_{c}}$. We introduce two
kinds of spatial splitting schemes, enabling 2 and 4 steps in the spatial
autoregression respectively. For the 2-step splitting scheme, we adopt [6] and
partition the latent variables into a checkerboard pattern, which implies that
$k_{h}=1,k_{w}=2$. For the 4-step splitting scheme, we select
$k_{h}=2,k_{w}=2$ and divide latent variables as shown in Figure 2. With these
spatial splitting schemes, our context model can leverage bidirectional
spatial contextual information. In the following sections, the notation
$k_{c}\times k_{h}k_{w}$ is consistently employed to represent the method of
latent variables division. This denotes that the division process initially
splits along the channel into $k_{c}$ slices, followed by a $k_{h}k_{w}$-step
spatial division to further group each slice. In our study, two configurations
of latent variables division are employed: $10\times 4$ for the base model and
$5\times 2$ for the fast model, respectively.

After the grouping operation, the latent variables $\hat{\bm{y}}$ are divided
into groups
$\hat{\bm{y}}_{\mathcal{G}_{1}},\hat{\bm{y}}_{\mathcal{G}_{2}},\cdots,\hat{\bm{y}}_{\mathcal{G}_{G}}$,
where $\mathcal{G}$ denotes the set of all indices of symbols in the latent
variables and it satisfies
$\mathcal{G}=\mathcal{G}_{1}\cup\mathcal{G}_{2}\cdots\cup\mathcal{G}_{G}$.
Each group of representation satisfies that
$y_{\mathcal{G}_{i}}\in\mathbb{R}^{hw\times c}$. Our entropy model factorizes
the joint probability of the latent variables as a group-wise autoregression
as follows:

$p(\hat{\bm{y}}\mid\hat{\bm{z}})=\prod_{i=1}^{G}p(\hat{\bm{y}}_{\mathcal{G}_{i}}\mid\hat{\bm{y}}_{\mathcal{G}_{<i}},\hat{\bm{z}}).$
(5)

Compared with previous group-based methods [7, 8], instead of modeling each
conditional distribution with a separate network, we decouple the number of
groups with network design and parameterize group-wise autoregressive model
with a shared transformer architecture, GroupedMixer.

In GroupedMixer, instead of directly attending to all decoded values in
previous groups, which is computationally intensive and usually impractical,
we decompose the interactions with previous values into two token-mixers:
cross-group and inner-group token-mixers, to model global dependency across
groups and within groups respectively. Specifically, we first map the
$c$-channel representation into $D$ dimensions using an input embedding layer.
Then the grouped representation is fed to our transformer model, which
consists of multiple GroupedMixer modules, as detailed in Figure 1a. Each
module comprises cross-group and inner-group token-mixers. Given the grouped
and mapped input feature $X_{1}\in\mathbb{R}^{G\times hw\times D}$, a
GroupedMixer module can be expressed as:

$\displaystyle\bm{Y}_{l}=\text{Cross-Group Token-Mixer}_{l}(\bm{X}_{l}),$ (6)
$\displaystyle\bm{Z}_{l}=\text{Inner-Group Token-Mixer}_{l}(\bm{Y}_{l}),$

where $\bm{Y}_{l}$ and $\bm{Z}_{l}$ are intermediate feature and output
feature in l-th layer module. The whole GroupedMixer architecture is
constructed by repeatedly stacking the inner-group and cross-group token-
mixers.

The output feature $\bm{Z}_{L}\in\mathbb{R}^{G\times hw\times D}$ from the
transformer blocks is shifted rightward along the second dimension, thus
discarding the last group of features. A learnable parameter
$\bm{\theta}_{h}\in\mathbb{R}^{hw\times 1\times D}$ is then inserted at the
first group’s position, which is formulated as:

$\bm{Z}_{L}^{s}=[\bm{\theta}_{h},\bm{Z}_{L}(:,:G-1,:)],$ (7)

where $[\cdot]$ denotes concatenation operation along the second dimension.
The motivation behind the shift operation is to ensure that the values of the
current group is not accessible while predicting its parameters, aligning with
the objectives of previous studies [9, 10]. Then the shifted feature undergoes
an output embedding layer to map it back to the original dimension $c$ from
$D$. The representation is further concatenated with grouped hyper prior along
the channel dimension and sent to parameter net. In parameter net, we first
unsqueeze features to recover spatial dimension, from $\mathbb{R}^{G\times
hw\times c}$ to $\mathbb{R}^{G\times c\times h\times w}$, and then aggregate
hyperprior and context information within the same group using a stack of
multiple convolution layers, and finally squeeze feature to original shape, as
detailed in Figure 3. At the end, the predicted parameters are rearranged to
the same shape as the latent variables $\hat{\bm{y}}$ using ungroup operation
$f^{-1}$ and split into $\bm{\mu},\bm{\sigma}$.

Figure 3: Detailed structure of Parameters Net. $c$ is the number of channels
in each group.

###  III-B Group-wise Token Mixers with Acceleration

We next specify the details of two token-mixers, and further propose a context
cache optimization scheme to accelerate the network inference:

####  III-B1 Inner-Group Token-Mixer

Inner-group token-mixer is designed to model the dependency within each group.
As shown in the top subfigure of Figure 1b, given the grouped representation
$\bm{Y}\in\mathbb{R}^{G\times hw\times D}$, we apply an attention process
along the spatial dimension and share the weights across the group dimension.
Specifically, $\bm{Y}$ is sent to a multi-head self-attention module to mix
information along the spatial dimension and produce the output feature
$\bm{Y}^{o}\in\mathbb{R}^{G\times hw\times D}$. In the attention mechanism, we
necessitate position embedding to provide the following attention process with
the information of the current position. Considering discrepancy of image
resolution in training and testing phases, instead of using diamond-shape
position embedding in [9], we introduce a position embedding generator (PEG)
[33] using depth-wise convolution, which introduces inductive bias of
translation invariance in training phase. We define our position encoding
scheme as follows:

$\bm{Y}^{P}=\operatorname{DWConv}(\bm{Y})+\bm{Y},$ (8)

where the weights are initialized with zeros. We empirically find that the
proposed position encoding scheme generalizes better on large resolution
datasets in Section IV-F3. We then linearly project position-aware input
$\bm{Y}^{P}$ to queries, keys and values, and then split them into $m$
representations of each head, yielding
$\bm{Q}_{i}^{P},\bm{K}_{i}^{P},\bm{V}_{i}^{P}\in\mathbb{R}^{G\times hw\times
d_{h}},\forall i=1,\cdots,m$, where $d_{h}$ is the number of head dimensions.
For spatial dimension, Then we can formulate multi-head attention process as
follows:

$\displaystyle\bm{A}_{i}$
$\displaystyle=\sigma\left(\frac{\bm{Q}_{i}^{P}\bm{K}_{i}^{P\top}}{\sqrt{d_{h}}}\right)\bm{V}_{i}^{P},$
(9) $\displaystyle\bm{Y}^{o}$
$\displaystyle=\bm{W}\left(\bm{A}_{1}\oplus\cdots\oplus\bm{A}_{m}\right)+\bm{Y},$

where $\sigma$ denotes the softmax function. Then $\bm{Y}^{o}$ is sent to a
feed-forward network to mix information along the channel dimension using two
linear layers with expansion ratio $4$ and produce the output feature
$\bm{Z}\in\mathbb{R}^{G\times hw\times D}$. In this way, different tokens in
each group can be adequately mixed along spatial and channel dimensions for
generating output features.

Figure 4: Context cache optimization at inference time. Each block denotes a
group of attention activation values.

####  III-B2 Cross-Group Token-Mixer

Cross-group token-mixer exchanges information across different groups, and
thus makes the model to aggregate global spatial-channel information from
previous decoded values. As shown in the bottom subfigure of Figure 1b, the
cross-group token-mixer is implemented by applying masked attention mechanism
on the group dimension. For implementation, given the input feature
$\bm{X}\in\mathbb{R}^{G\times hw\times D}$, we first permute the group and
spatial dimensions, and get the rearranged intermediate feature
$\bm{X}^{r}\in\mathbb{R}^{hw\times G\times D}$. The input feature $\bm{X}^{r}$
undergoes a masked attention process to integrate information across the group
dimension while maintaining the same weights on the spatial dimension. We
reformulate (9) as:

$\bm{A}_{i}=\sigma\left(\frac{\bm{Q}_{i}\bm{K}_{i}^{\top}+\bm{Q}_{i}\bm{P}^{C}}{\sqrt{d_{h}}}+\bm{M}\right)\bm{V}_{i}$
(10)

where $\bm{P}^{C}\in\mathbb{R}^{G\times G\times d}$ denotes relative position
embedding, and $\bm{M}\in\mathbb{R}^{G\times G}$ signifies the mask matrix, in
which the lower triangle is zero and the rest is $-\infty$. To encode position
information for distinguishing each group, consider the group dimension as a
3D space with size $(k_{c},k_{h},k_{w})$. Each group index $i$ is thus
assigned a 3D coordinate $(x_{i},y_{i},z_{i})$. For indexes $u,v$, the
relative position vector $\mathbf{r}_{u,v}$ is computed as
$\mathbf{r}_{u,v}=(x_{u}-x_{v},y_{u}-y_{v},z_{u}-z_{v})$, where each component
of the vector $(x_{u}-x_{v},y_{u}-y_{v},z_{u}-z_{v})$ falls within the range
$[-k_{h}+1,k_{h}-1],[-k_{w}+1,k_{w}-1],[-k_{c}+1,k_{c}-1]$ respectively, and
is then mapped to a positive scalar $s$:

$\displaystyle s=(z_{u}-z_{v}+k_{c}-1)$
$\displaystyle\cdot(2k_{h}+1)\cdot(2k_{w}+1)$ (11)
$\displaystyle+(x_{u}-x_{v}+$ $\displaystyle k_{h}-1)\cdot(2k_{w}+1)$
$\displaystyle+y_{u}-y_{v}$ $\displaystyle+k_{w}-1.$

The relative position embedding $\bm{P}^{C}_{u,v}\in\mathbb{R}^{d}$ between
group $u$ and $v$ can be indexed as:

$\bm{P}^{C}_{u,v}=\bm{W}_{s,:},$ (12)

where $\bm{W}\in\mathbb{R}^{n_{p}\times d}$ is a 2D learnable parameter and
initialized with truncated normal distribution and
$n_{p}=(2k_{c}-1)\cdot(2k_{h}-1)\cdot(2k_{w}-1)$. Similar to inner-group
token-mixer, the output feature is sent to a MLP module to further mix
information along the channel dimension and produce the output feature
$\bm{Y}\in\mathbb{R}^{G\times hw\times D}$. Note that this cross-group process
implies that information is aggregated across accessible groups but restricted
to the same spatial location, as illustrated in the bottom subfigure of Figure
1b. By alternately stacking inner-group and cross-group token-mixers, the
model can adequately mix information along the spatial, group and channel
dimensions for capturing global information.

####  III-B3 Context Cache Optimization for Acceleration

At inference time, we estimate the distribution of the latent variables using
group-wise autoregression, which requires $G$ times inference of the context
model. However, the inference speed remains a challenge due to the large model
size and intensive computation involved in the autoregressive process. One
critical factor overlooked by previous works [9, 10] is the repeated context
computation during inference, which implies that the network inference inputs
a gradually enlarging context from $0$ to $G-1$ groups of tokens at each of
the G steps. Inspired by the attention cache optimization to expedite sequence
generation [34] in NLP, we propose context cache optimization for faster
group-wise autoregressive distribution prediction, as illustrated in Figure 4.
Considering that only cross-group token-mixer performs interactions with
previously decoded groups, the corresponding activations need to be cached and
are utilized for being attended in later inference stages. Therefore, we cache
activations including each group of keys and values in every cross-group
token-mixers at each inference time. Before performing attention process, key
and value produced by the current group are concatenated with cached
activation values. Note that mask is canceled in attention because all
concatenated decoded keys and values can be accessed at current step. The
self-attention with context cache optimization in the cross-group token-mixer
at step $t$ is implemented in (13):

$\displaystyle\bm{Q}^{t}_{i}$ $\displaystyle=\bm{X}^{t}_{i}\bm{W}^{Q},$ (13)
$\displaystyle\bm{K}^{t}_{i}$
$\displaystyle=[\text{Cache\\_K}_{t-1},\bm{X}^{t}_{i}\bm{W}^{K}],$
$\displaystyle\bm{V}^{t}_{i}$
$\displaystyle=[\text{Cache\\_V}_{t-1},\bm{X}^{t}_{i}\bm{W}^{V}],$
$\displaystyle\bm{A}^{t}_{i}$
$\displaystyle=\sigma(\frac{\bm{Q}^{t}_{i}\bm{K}^{t\top}_{i}+\bm{Q}^{t}_{i}\bm{P}_{c}^{\top}}{\sqrt{D}})\bm{V}^{t}_{i},$

where concatenated keys and values satisfy
$\bm{Q}_{i}^{t}\in\mathbb{R}^{hw\times 1\times
D},\bm{K}_{i}^{t}\in\mathbb{R}^{hw\times t\times
D},\bm{V}_{i}^{t}\in\mathbb{R}^{hw\times t\times D}$. Our context cache
optimization diverges from [34] in the following aspects: (1)  Unlike the
approach in [34] where attention results from all blocks are stored, we cache
attention activation values specific to group-wise autoregression in cross-
group token-mixers.  (2) Our strategy operates at the group level, caching and
concatenating intermediate activation values $\mathbb{R}^{hw\times 1\times D}$
for a group of tokens rather than individual tokens. These modifications not
only tailor the attention activation cache concept to our GroupedMixer’s
architecture but also represent a novel application of this strategy. Equipped
with context cache optimization, we only need to feed a single group of latent
variables $\hat{\bm{y}}_{\mathcal{G}_{t}}$ to GroupedMixer each time instead
of all decoded groups $\hat{\bm{y}}_{\mathcal{G}_{<t}}$. Thus, the inference
process of GroupedMixer is significantly accelerated, leading to faster image
coding.

###  III-C Analysis

####  III-C1 Computation Complexity

In the GroupedMixer module, instead of directly applying full self-attention
process on all tokens which entails computational complexity:

$\mathcal{O}(Attn)=4(Ghw)d_{h}^{2}+2(Ghw)^{2}d_{h}.$ (14)

We decompose the global attention into two parts: inner-group and cross-group
token-mixers. The computational complexity of the inner-group token-mixer is
$\mathcal{O}(4(hw)d_{h}^{2}+2(hw)^{2}d_{h})$, and the computational complexity
of the cross-group token-mixer is $\mathcal{O}(4Gd_{h}^{2}+2G^{2}d_{h})$. The
overall complexity of this decomposition is reduced to follows:

$\mathcal{O}(GM)=4(hw+G)d_{h}^{2}+2((hw)^{2}+G^{2})d_{h},$ (15)

where the major computation and memory cost comes from the self-attention
process in inner-group token-mixer. This decomposition also enables us to
scale to larger $G$, which is configured to $10,40$ in our implementation. To
better understand the computational complexity, consider an image size of
$768\times 512$, downscaled to $32\times 48$ latent codes and then grouped
into 40 groups ($10\times 4$ configuration), retrieving $h=16,w=24$. The
overall computational complexity of our GroupedMixer module is:

$\displaystyle\mathcal{O}(GM)$ $\displaystyle=4(16\times
24+40)d_{h}^{2}+2((16\times 24)^{2}+40^{2})d_{h}$ (16)
$\displaystyle=1,696d_{h}^{2}+298,112d_{h}$

This demonstrates the efficiency of our method compared to directly applying
full attention process, whose complexity is

$\displaystyle\mathcal{O}(Attn)$ $\displaystyle=4(40\times 16\times
24)d_{h}^{2}+2(40\times 16\times 24)^{2}d_{h}$ (17)
$\displaystyle=61,440d_{h}^{2}+471,859,200d_{h}$

####  III-C2 Insights for Enhanced Performance

In this section, we delve deeper into the underlying reasons for the enhanced
compression performance, increased coding speed, and reduced parameter count
achieved by GroupedMixer. Contrasting with existing group-based entropy
models, such as ChARM [7], ELIC [8], which employ convolutional layers to
model dependencies, our approach leverages the transformer architecture to
capture long-range dependencies, outperforming conventional methods in terms
of performance. Moreover, our model significantly reduces the number of
parameters by utilizing a shared transformer architecture, as opposed to
modeling each group with distinct network parameters as in ChARM [7], ELIC
[8], TCM [11], MLIC [12, 13]. This design choice enables more effctive
modeling complex correlations with the same amount of aprameters and enhances
its scalability to larger group sizes. Different from previous transformer-
based entropy models Entroformer [9], Contextformer [10] that utilize pixel-
wise autoregression and full self-attention across all tokens, our model
employs group-wise autoregression and innovatively splits the attention
mechanism into inner-group and cross-group token-mixers. These designs
decrease the number of autoregression steps and further lower computational
complexity of each inference. Alongside with context cache optimization, they
speed up inference, making a notable advancement over Entroformer [9],
Contextformer [10]. These also facilitate efficient multi-step fine-tuning,
addressing train-test mismatch for improved performance, as shown in Section
IV-F4. Our model addresses redundancies across the spatial and channel
dimensions, rather than solely the spatial dimension as in Entroformer [9].
Due to reducing computation complexity, our approach can segment latent
variables into more slices across the channel dimension. This, coupled with
carefully designed position embeddings, results in enhanced performance
compared to Contextformer [10]. Consequently, our model attains superior
performance to earlier transformer-based entropy models with even less complex
analysis and synthesis transform network, coupled with the benefits of faster
inference speeds. This represents a significant advancement in the field of
efficient learned image compression.

Figure 5:  Performance evaluation on the Kodak dataset.

Figure 6:  Performance evaluation on the CLIC’21 Test dataset.

Figure 7:  Performance evaluation on the Tecnick dataset.

TABLE II: Evaluation of different methods on the Kodak dataset. “BD-Rate” is computed with VTM as anchor method. “Tot. Enc.” and “Tot. Dec.” indicate total encoding and decoding times in seconds (s), “T. Dec.” covers network transformation time during decoding, and “E. Dec.” includes entropy coding time (hyper prior, context model, arithmetic coding). Total parameter (“Tot. Param.”) are in millions (M), with “T. Param.” and “E. Param.” for transformation and entropy model parameters, respectively. ∗ marks results from original papers, † denotes our reimplemented model tests. “CCO” refers to context cache optimization. Model | BD-Rate(%) | Latency(s) | Params(M)
---|---|---|---
Tot. Enc. | Tot. Dec. | T. Dec. | E. Dec. | Tot. Param. | T. Param. | E. Param.
VVC [24] | 0.00 | 129.21 | 0.14 | / | / | / | / | /
Minnen [5] | 10.90 | 1.98 | 4.60 | 6e-3 | 4.59 | 25.5 | 7.0 | 18.5
Cheng [3] | 5.44 | 1.98 | 4.69 | 0.03 | 4.66 | 29.6 | 19.0 | 10.6
Xie [18] | -0.78 | 2.93 | 6.00 | 1.92 | 4.08 | 50.0 | 39.3 | 10.7
Group-based Entropy Model |  |  |  |
ChARM [7] | 1.38 | 0.08† | 0.09† | 6e-3† | 0.08† | 126.7† | 7.0† | 119.7†
STF [23] | -4.31 | 0.14 | 0.13 | 0.03 | 0.10 | 99.9 | 13.8 | 86.1
ELIC-SM [8] | -1.07 | 0.06† | 0.08† | 6e-3 † | 0.07† | 29.8† | 7.6† | 22.2†
ELIC [8] | -7.24 | 0.07† | 0.09† | 0.02† | 0.07† | 36.9† | 14.6† | 22.2†
TCM [11] | -11.74 | 0.16 | 0.15 | 0.07 | 0.08 | 76.6 | 28.9 | 47.7
MLIC+ [12] | -13.11 | / | / | / | / | / | / | /
MLIC++ [13] | -15.07 | 0.16 | 0.18 | 0.02 | 0.16 | 116.7 | 16.4 | 100.3
Autoregressive Transformer |  |  |  |
Entroformer [9] | 2.73 | 3.86 | 121.19 | 0.03 | 121.16 | 45.0 | 7.6 | 37.4
Entroformer-P [9] | 5.50 | 4.28 | 7.95 | 0.03 | 7.93 | 45.0 | 7.6 | 37.4
Contextformer [10] | -7.44 | 40∗ | 44∗ | / | / | 37.4∗ | 17.5∗ | 19.9∗
Group-wise Autoregressive Transformer |  |  |  |
GroupedMixer | -8.31 | 0.27 | 0.26 | 6e-3 | 0.25 | 36.4 | 7.6 | 28.8
\- w/o CCO | - | 1.06 | 1.01 | 6e-3 | 1.01 | - | - | -
GroupedMixer-Fast | -6.03 | 0.07 | 0.06 | 6e-3 | 0.05 | 36.8 | 7.6 | 29.2
GroupedMixer-Large | -17.84 | 0.17 | 0.17 | 0.07 | 0.10 | 72.8 | 28.9 | 43.9
TABLE III: Speed performance analysis of two models at varying resolutions. This comparison focuses on the latency metrics in milliseconds (ms) for both encoding and decoding processes across different image resolutions.  Model | Size | Latency(ms)
---|---|---
Tot. Enc. | Tot. Dec. | T. Dec. | E. Dec.
GM | $256\times 256$ | 242.40 | 241.15 | 1.46 | 239.69
$512\times 512$ | 256.52 | 251.34 | 4.27 | 247.07
$1024\times 1024$ | 363.20 | 356.63 | 15.50 | 341.12
$2048\times 2048$ | 1814.28 | 1755.38 | 59.31 | 1696.07
GM-Fast | $256\times 256$ | 60.90 | 59.39 | 1.71 | 57.68
$512\times 512$ | 68.67 | 63.26 | 4.43 | 58.82
$1024\times 1024$ | 172.44 | 158.91 | 15.64 | 143.28
$2048\times 2048$ | 1218.46 | 1147.26 | 59.23 | 1088.03

##  IV Experiments

###  IV-A Settings

####  IV-A1 Implementation Details

As our main contribution is to design a novel entropy model, we directly adopt
the transform network design in [8, 11]. We share the same analysis transform
$g_{a}$, synthesis transform $g_{s}$, hyper analysis transform $h_{a}$ and
hyper synthesis transform $h_{s}$ with ELIC-SM [8] and TCM [11]. We set the
number of channels $N=192$ and $M=320$ for all models. For architecture of
transformer, we employ depth $L=6$, the embedding channels $d_{e}=384$ and the
number of heads $m=12$ as our default configuration. We initialize the weights
of GroupedMixer with a truncated normal distribution as used in ViT [35]. In
our study, for rapid comparison, we employ a transform network identical to
that in ELIC-SM and opt for the $10\times 4$ partitioning method as the
foundational variant (GroupedMixer). We also use the $5\times 2$ method for a
faster variant, termed GroupedMixer-Fast. To present the best performance on
compression and speed, we substitute the transform network in GroupedMixer-
Fast with a mixed Transformer-CNN architecture, as proposed in TCM [11]. This
modified version is designated as GroupedMixer-Large. For training, we use the
Adam [36] optimizer with $\beta_{1}=0.9,\beta_{2}=0.999$. All models are
trained for over 2M iterations. Gradient clipping technique is also employed
for stable training and the clip value is set to 1.0. We optimize our models
with rate-distortion loss as Equation (2). During training, a Lagrange
multiplier $\lambda$ balances the rate-distortion trade-off. When optimizing
for MSE, we train different models with values of
$\lambda\in\\{4,8,16,32,75,150,300,450,900,1500\\}\times 10^{-4}$, where
$D=\operatorname{MSE}(\hat{x},x)$. When optimizing for MS-SSIM, we utilize
$\lambda\in\\{2,4,8,16,32,64\\}$, where $D=1-\operatorname{MS-
SSIM}(\hat{x},x)$.

Due to the repeated invocation of the entropy coder during decoding in the
group-based entropy model, we have achieved speed acceleration by optimizing
the calling interface, based on CompressAI [37]. Specifically, we modified the
data transfer method used when calling the entropy coder, changing it from
list transfer to tensor transfer, which speeds up entropy coding process. In
our method, tensor reorganization is a crucial step to align data for
subsequent processing. This is achieved through a rearrangement operation,
which we implement using the einops.rearrange function [38]. This function
allows us to efficiently transform the multi-dimensional structure of our data
tensors into the desired format. Our model is implemented based on Pytorch 2.0
[39], utilizing torch.nn.functional.scaled_dot_product_attention to accelerate
the computation process of attention.

####  IV-A2 Datasets

Our compression model is trained on randomly selected $5\times 10^{4}$ images
from DIV2K [40], FLICKR 2K [41], ImageNet [42] with resolution larger than
$480\times 480$ as our training set. We evaluate the rate-distortion
performance on three publicly available datasets: Kodak dataset [43] with 24
images with a size of $768\times 512$, Tecnick dataset [44] with 100 images
with a size of $1200\times 1200$, and CLIC’21 Test dataset [45] with 60 images
with a size of 2k resolution. We measure the distortion of reconstruction
using PSNR and MS-SSIM, and convert MS-SSIM to $-\log_{10}(1-\text{MS-SSIM})$
for clearer comparison. For CLIC’21 Test [45] and Tecnick [44], we adhere to
the testing protocol employed in CompressAI [37], where the images are padded
to multiples of $64$ beforehand, while the rate is derived from padded images
and distortion is evaluated on images without padding.

####  IV-A3 Three-Stage Training Scheme

Following previous works [6, 5, 8] and community discussion
111https://groups.google.com/g/tensorflow-compression/c/LQtTAo6l26U/m/mxP-
VWPdAgAJ , we encode $\lceil\bm{y}-\bm{\mu}\rfloor$ into bitstream which
enhances RD performance, and thus the decoder side receives the reconstructed
latent variables $\hat{\bm{y}}=\lceil\bm{y}-\bm{\mu}\rfloor+\bm{\mu}$. To
train a group-based transformer, we adapt a three-stage training strategy. In
the first stage, we employ one-pass training scheme for faster forward
process, and feed differentiable estimation
$\bm{y}+\mathcal{U}(-\frac{1}{2},\frac{1}{2})$ to the context model,
$\text{STE}(\bm{y})$ to the decoder, where
$\mathcal{U}(-\frac{1}{2},\frac{1}{2})$ is the uniform distribution and
$\text{STE}(\cdot)$ is the straight-through estimator [19]. The original
images are cropped to $256\times 256$ patches before being fed into the
network. Minibatches of 16 of these patches are used to update the network
parameters and the learning rate is set to $1\times 10^{-4}$. The first
training stage lasts for 1.2M-step iterations. From the second stage on, we
enlarge the size of patches from 256 to 384 to train our transformer for
better global context modeling ability and batch size is set to 8. The
learning rate drops to $5\times 10^{-5}$ at the beginning of this stage, and
drops to $1\times 10^{-5}$ at 1.5M steps. The second stage lasts for 0.6M
iterations. In the third stage, we apply multi-step fine-tuning to close the
train-test gap, which improves performance for group-based entropy models.
Specifically, we conduct $G$ times inference of context model to retrieve real
reconstruction $\hat{\bm{y}}=\text{STE}(\bm{y}-\bm{\mu})+\bm{\mu}$, and feed
it to both the context model and the decoder. The learning rate starts at
$4\times 10^{-5}$ and the learning rate is then divided by 2 when the
evaluation loss reaches a plateau (we use a patience of $1\times 10^{4}$
iterations). This stage finishes after at most 1M steps or when the learning
rate is decayed below a certain threshold value, which is set to $1\times
10^{-6}$. Thanks to the context cache optimization introduced in section
III-C1, our model can be trained with less computation and memory cost in this
stage.

###  IV-B Rate-Distortion Performance

We compare our method with other learned image compression methods, including
MLIC [12, 13], TCM [11], Contextformer [10], Entroformer [9], ELIC, ELIC-SM
[8], STF [23], ChARM [7], Xie _et al._[18], Cheng _et al._[3], Minnen _et
al._[5], as well as some traditional codecs, like VVC intra (VTM 17.0) [24]
and BPG [46]. For ChARM [7], we reproduce the method based on CompressAI [37]
for fair comparison. For ELIC, ELIC-SM [8], we use publicly available
unofficial implementation provided by [47], which is based on CompressAI [37].
For [3] and [5], we evaluate the codes provided by CompressAI [37]. The other
results are obtained from their official repositories or emails with authors.

Figure 5 shows the rate-distortion performance on the Kodak dataset with the
model optimized for MSE and MS-SSIM respectively. We also report BD-rates
reduction of our method and state-of-the-art learned methods in Table II. The
BD-rate is computed from the Rate-PSNR curve with VTM as the anchor method.
This demonstrates that our methods outperform all listed methods in terms of
PSNR and MS-SSIM. Our GroupedMixer, GroupedMixer-Fast, GroupedMixer-Large
reduce BD-rates by 8.31%, 6.03% and 17.81% on Kodak dataset over VVC when
measured in PSNR. From Figure 5 we can see, the performance gain is more
significant for higher bitrates. Compared to ELIC-SM [8], our GroupedMixer,
which employs the same transform network as ELIC-SM, achieves a 7.24% BD-Rate
saving. Additionally, it is capable of increasing the PSNR by up to
approximately 0.5dB at the same bitrate. Compared with TCM [11], our
GroupedMixer-Large, utilizing the same transform network as TCM, realizes a
6.07% BD-Rate saving. This demonstrates that our proposed entropy model
surpasses the convolution-based entropy model presented in ELIC [8] and the
Swin-Transformer Attention based entropy model used in TCM [11]. Compared with
previous SOTA MLIC++ [13], our GroupedMixer-Large also achieves 2.74% BD-Rate
saving. Compared with Entroformer, our GroupedMixer achieves 11.04% BD-rate
reduction. Even equipped with a lighter transform network than Contextformer
[10] and ELIC [8], our GroupedMixer still outperforms them by 1.07% and 0.87%
BD-rate reduction respectively, especially at higher bitrate regions.

The performance evaluation results on high-resolution image datasets CLIC’21
Test and Tecnick are respectively presented in Figure 7 and Figure 7. As
shown, our approach surpasses all other codecs. The GroupedMixer-Large model
demonstrates superior performance with BD-rate savings of 19.77% and 22.56% on
the CLIC’21 Test and Tecnick datasets, respectively. This marks a significant
improvement over the previous SOTA method, MLIC++ [13], which achieved BD-rate
savings of 15.60% and 18.36% on these datasets.

Figure 8: Visual comparisons of decompressed image of “kodim07.png” from Kodak
dataset based on different methods. The subfugure is titled as
“bitrate/PSNR/MS-SSIM”.

###  IV-C Model Complexity

We report encoding/decoding latency and number of parameters in Table II. For
evaluation of model speed, we limit the accessible computation resources to
one 3090 GPU card and AMD Ryzen 9 5900X 12-Core CPU. We use same testing
protocol as ELIC [8] to measure the latency. Note that entropy coding details
is not mentioned in ELIC [8], so we directly use the publicly accessible
reimplementation [47] to test coding latency. The table reports total and
individual latencies for transform and entropy coding time respectively,
averaged on all images in Kodak dataset. For computation of entropy coding
time, we count inference time of hyperprior, context model and entropy coder.

As shown in Table II, our method not only achieves SOTA RD performance but
also excels in efficiency. When compared with other transformer-based entropy
models, our approach demonstrates significantly faster speeds. For instance,
our base model records average encoding and decoding times of 0.27s and 0.26s,
respectively. This is 466 times faster than Entroformer [9] and 169 times
faster than Contextformer [10] in decoding. Our GroupedMixer-Fast variant
further improves these speeds to 0.07s for encoding and 0.06s for decoding. It
even surpasses ELIC-SM [8] in decoding speed while maintaining superior RD
performance. These enhancements are primarily due to the context cache
optimization and our refined entropy coding process, leading to a fivefold and
twofold increase in decoding speed respectively.

Additionally, our experiments on varying image sizes reveal a direct
correlation between resolution and coding time in GroupedMixer. For example,
decoding time for a $256\times 256$ image is 241.15 ms, increasing to 1755.38
ms for a $2048\times 2048$ image. In contrast, the GroupedMixer-Fast variant
shows a rise from 59.39 ms to 1147.26 ms for these resolutions. Notably, the
latency growth rate accelerates with higher resolutions. Since the time
complexity is predominantly influenced by the inner-group block complexity,
which is proportional to the square of the number of latent variables.

In Table II, we demonstrate that weight sharing across groups maintains a
nearly constant parameter count, even as the number of groups increases. In
fact, due to reductions in the input/output embedding layer size, the
parameter count may decrease. This represents an improvement over models like
ChARM [7], STF [23], TCM [11] and MLIC [12, 13], as our entropy model
conserves parameters effectively, even with more groups. Compared to previous
models like ELIC [8], Entroformer [9], and Contextformer [10], GroupedMixer
stands out for its superior RD performance, achieved with a comparable or
lower total parameter count and a lighter transform network. Detailed analysis
reveals that our GroupedMixer has slightly more parameters than Contextformer
[10] (18.3M vs. 15.9M), and the hyperprior network does carry more parameters
(10.5M vs. 4.0M). Although our context model and hyperprior network are
slightly more parameter-intensive compared to Contextformer [10], our
strategic choices enable us to maintain leading encoding speeds, a more
crucial aspect in practical applications. We can observe that enhancing the
transform network’s capacity increases inference latency marginally, but
significantly boosts performance. Our GroupedMixer-Large surpasses both TCM
[11] and MLIC [12, 13] in performance, even with fewer parameters.
Specifically, our model saves 43.9M parameters compared to MLIC++ [13] and
achieves better compression performance, demonstrating the efficient modeling
capabilities of our approach.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

Figure 9: Progressive decoding results of “kodim01.png”. The subfugure is
titled as number of received latent groups $k$ on decoder side and
“bitrate/PSNR”. It is recommended to zoom in and view these figures in full
screen.

###  IV-D Qualitative Results

To illustrate that our method can generate visually appealing results, we
visualize some reconstructed images for qualitative performance comparison. We
show some qualitative comparison of reconstructed images on the Kodak dataset
in Figure 8. This sample image “kodim07.png” is evaluated with an approximate
bpp of 0.1. We can deduce from the figure that our method can preserve more
details even at the lower bitrate, such as sharper texture of flower and
leaves in “kodim07.png”. It confirms that our MSE optimized method achieves
better fidelity with less bitrate compared with the traditional codecs, _e.g._
, VTM [24] and BPG [46], and surpasses the performance of other learned image
compression method Cheng _et al._[9] and Xie _et al._[48]. Moreover, we can
observe that our MS-SSIM optimized model achieve better visual quality than
other codecs. We conclude that better RD performance result benefits from our
proposed entropy model, which realizes more precise prediction and thus more
bitrate savings.

###  IV-E Progressive Decoding

In literature, learned image compression with group-based entropy model can
also be applied to implement progressive coding [7, 8]. In this section, we
also evaluate the ability of our method on progressive decoding. We directly
adapt our model ($10\times 4$ variant) trained with MSE distortion and
$\lambda=0.045$ for progressive decoding without any additional post-training.
Assuming that the decoder side already receives bitstreams of $k$ latent
groups, where $k=0$ denotes the case that no bitstream of $\hat{\bm{y}}$ but
only $\hat{\bm{z}}$ is received. We first reconstruct $k$ groups of latent
variables from bitstreams using entropy coding based on the predicted
distribution. For the remaining $G-k$ groups, we view the context model as a
generative model and sample latent variables from the predicted Gaussian
distribution
$p(\hat{\bm{y}}_{\mathcal{G}_{i}}\mid\hat{\bm{y}}_{\mathcal{G}_{<i}},\hat{\bm{z}};\psi)$
by ancestral sampling, where we first sample
$\hat{\bm{y}}_{\mathcal{G}_{k+1}}$ from
$p(\hat{\bm{y}}_{\mathcal{G}_{k+1}}\mid\hat{\bm{y}}_{\mathcal{G}_{<k+1}},\hat{\bm{z}};\psi)$,
then sample $\hat{\bm{y}}_{\mathcal{G}_{k+2}}$ from
$p(\hat{\bm{y}}_{\mathcal{G}_{k+2}}\mid\hat{\bm{y}}_{\mathcal{G}_{<k+2}},\hat{\bm{z}};\psi)$,
and so on. The progressive decoding results are shown in Figure 9. Compared
with other group-based methods [6, 7, 8], our method scales to a larger number
of groups and thus achieves finer progressive coding. From the figures,
sampling only from the hyperprior ($k=0$) renders the image content as color
blocks, indicating that the hyperprior encodes the blurry structure of the
image. Note that when only one group of latent variables is decoded ($k=1$),
the shape in sampled image for “kodim01.png” is still correlated and
recognizable, which is partly due to global spatial-channel-wise context
reference during sampling. As the received bitstream of latent variables
$\hat{\bm{y}}$ increases, the sampled image gradually contains more precise
structural information, such as the detail outline of windows and bricks in
“kodim01.png”. Then the image tone progressively changes, the details
gradually enrich, both approaching the real image. From the above phenomenon,
we can conclude that the initial latent groups tend to encode the structural
information of the image, while the later latent groups contain the color and
detail information.

###  IV-F Model Analysis

In this section, we conduct more experiments to further analyze GroupedMixer.
In the following experiments, for rapid validation, we adopt simpler network
architecture and training scheme. Specifically, we select the base
configuration of GroupedMixer as $\\{L=3,d_{e}=384,m=12\\}$ and use the
variant $5\times 2$ as default unless otherwise specified. For the training
strategy, we first train our model with $\lambda=0.045$ for 1M iterations with
base learning rate $1\times 10^{-4}$. Then, we fine-tune the model with
$\lambda=0.03,0.015$ respectively, covering $\lambda=0.015,0.03,0.045$
regions.

(a)

(b)

(c)

(d)

Figure 11: Comparative study of token-Mixer order, autoregression order,
position embedding, and multi-step fine-tuning. In this table, “i-c” vs. “c-i”
represent inner-group vs. cross-group token-mixer orders. “sfo” (spatial-
first-order) and “cfo” (channel-first-order) refer to different autoregression
orders. “Rel”, “Abs” and “DRel” is relative, absolute and diamond-shape
relative position embedding. “ms-ft” stands for multi-step fine-tuning.

####  IV-F1 Token-Mixer Order Study

We design two different token-mixers to construct the GroupedMixer Module. To
investigate the most efficient way to combine them, we evaluated the model
using 2 permutations. As depicted in Figure 11(a), it is beneficial to
position the cross-group token-mixer prior to the inner-group token-mixer, as
opposed to the converse arrangement. This preference may be attributed to the
fact that tokens in cross-group token-mixer, whether derived from their
proximate spatial positions or otherwise, demonstrate an increased capacity
for aggregating more complex information.

TABLE IV: Performance of different depths and dimensions for transformer blocks used in GroupedMixer.  Model | Depth | Dimension | BD-Rate(%) | Param.
---|---|---|---|---
$5\times 2$ | 3 | 96 | 11.92 | 12.5M
3 | 192 | 4.51 | 14.5M
1 | 384 | 8.98 | 15.4M
3 | 384 | 2.83 | 22.6M
6 | 384 | 0.00 | 36.8M

####  IV-F2 Spatial-Channel Order

Partitioning the latent variables into groups enables two different
autoregression orders, namely, spatial-first-order (sfo) and channel-first-
order (cfo), which can be expressed as follows respectively:

$\displaystyle f_{\text{sfo}}$ $\displaystyle:\mathbb{R}^{H\times W\times
C}\rightarrow\mathbb{R}^{{k_{c}k_{h}k_{w}}\times\frac{H}{k_{h}}\frac{W}{k_{w}}\times\frac{C}{k_{c}}}$
(18) $\displaystyle f_{\text{cfo}}$ $\displaystyle:\mathbb{R}^{H\times W\times
C}\rightarrow\mathbb{R}^{{k_{h}k_{w}k_{c}}\times\frac{H}{k_{h}}\frac{W}{k_{w}}\times\frac{C}{k_{c}}}$

Different coding orders prioritize contextual information from different
dimensions (spatial or channel), which may result in different compression
performance. The comparison result is illustrated in Figure 11(a). On average,
the channel-first-order (cfo) performs almost the same as the model in
spatial-first-order (sfo) configuration. We speculate that two different
partition methods provide two almost equivalent contextual information in the
amortized sense. We adopt the spatial-first-order (sfo) in our final model.

####  IV-F3 Position Embedding

We first examine the effects of position embedding used in cross-group token-
mixer. We build models with different position encoding methods, including
absolute position embedding, 3D relative position embedding and diamond-shape
3D relative position embedding, which extends diamond position embedding
proposed in [9] from 2D to 3D. The results are presented in Figure 11(b). We
observe that the 3D relative position embedding outperforms the others.
Moreover, we notice that introducing region constraint in cross-group token-
mixer is neither necessary nor effective. Hence, we adopt 3D relative position
embedding in our final model. In inner-group token-mixer, PEG plays a vital
role in generalizing model to high resolution images. We also conduct an
experiment to verify this, where configuration follows IV-A. We compare PEG
with previous proposed effective 2D position embedding, diamond position
embedding proposed in [9]. As shown in Figure 11(c), even though the two
models, one with PEG and the other without PEG, perform very closely on the
Kodak dataset, they show significant differences on the larger resolution
dataset CLIC’21 Test, with the model without PEG performing much worse than
the one with PEG. This is because PEG is implemented based on depth-wise
convolution, which allows for better generalization across images of varying
scales compared to diamond position embedding.

####  IV-F4 Multi-step Fine-tuning

To assess the impact of multi-step fine-tuning on training performance, an
ablation study on multi-step fine-tuning was conducted, where configuration
follows Section IV-A, as shown in Figure 11(d) In this comparison, “w. ms-ft”
represents the outcome with the final multi-step fine-tuning, while “w/o. ms-
ft” corresponds to the results where the third stage was replaced with a one-
pass training approach. It was observed that employing multi-step fine-tuning
in the training process effectively narrowed the gap between training and
testing performance, leading to an overall improvement. Quantitatively, this
approach resulted in a reduction of 1.39% in BD-Rate, underscoring the
effectiveness of multi-step fine-tuning in enhancing model performance.

####  IV-F5 Depth and Embedding Dimension

In Table IV we conduct architecture ablations to examine the relationships
between model architecture and compression performance. We explore the
following architectural changes: decreasing the number of hidden dimension,
holding model depth relatively constant $L=3$; decreasing model depth, holding
the hidden dimension constant $d_{e}=384$. The head dimension $d_{h}$ is
consistently set to a constant $32$. Our results show that decreasing the
hidden dimension size leads to performance deterioration, causing 1.68 % BD-
Rate increase from $d_{e}=384$ to $d_{e}=192$ and 9.09% BD-Rate increase from
$d_{e}=384$ to $d_{e}=96$. Likewise, reducing the model depth impairs the
performance, leading to 8.98% BD-Rate increase from $L=6$ to $L=1$ and 2.83%
BD-Rate increase from $L=6$ to $L=3$. This is because both the hidden
dimension and the depth are critical factors for the model, which influence
the model’s capacity to encode the contextual information.

##  V Conclusion

In this paper, we present an entropy model called GroupedMixer, a group-wise
autoregressive transformer for learned image compression. To construct a
spatial-channel context model that enables global reference and also entails
lower complexity, we propose to decompose self-attention into two group-wise
token-mixers, namely, inner-group and cross-group token-mixers. We then apply
context cache optimization to expedite inference speed, which allows faster
coding process and multi-step finetuning. We demonstrate that GroupedMixer
achieves SOTA compression performance with fast coding speed.

## References

  * [1] G. Toderici, S. M. O’Malley, S. J. Hwang, D. Vincent, D. Minnen, S. Baluja, M. Covell, and R. Sukthankar, “Variable rate image compression with recurrent neural networks,” _arXiv preprint arXiv:1511.06085_ , 2015.
  * [2] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” in _International Conference on Learning Representations_ , 2018.
  * [3] Z. Cheng, H. Sun, M. Takeuchi, and J. Katto, “Learned image compression with discretized gaussian mixture likelihoods and attention modules,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 7939–7948.
  * [4] F. Jiang, W. Tao, S. Liu, J. Ren, X. Guo, and D. Zhao, “An end-to-end compression framework based on convolutional neural networks,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 28, no. 10, pp. 3007–3018, 2018.
  * [5] D. Minnen, J. Ballé, and G. D. Toderici, “Joint autoregressive and hierarchical priors for learned image compression,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [6] D. He, Y. Zheng, B. Sun, Y. Wang, and H. Qin, “Checkerboard Context Model for Efficient Learned Image Compression,” in _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, pp. 14 766–14 775.
  * [7] D. Minnen and S. Singh, “Channel-Wise Autoregressive Entropy Models for Learned Image Compression,” in _2020 IEEE International Conference on Image Processing (ICIP)_ , pp. 3339–3343.
  * [8] D. He, Z. Yang, W. Peng, R. Ma, H. Qin, and Y. Wang, “ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding,” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, pp. 5708–5717.
  * [9] Y. Qian, X. Sun, M. Lin, Z. Tan, and R. Jin, “Entroformer: A transformer-based entropy model for learned image compression,” in _International Conference on Learning Representations_ , 2021.
  * [10] A. B. Koyuncu, H. Gao, A. Boev, G. Gaikov, E. Alshina, and E. Steinbach, “Contextformer: A transformer with spatio-channel attention for context modeling in learned image compression,” in _European Conference on Computer Vision_. Springer, 2022, pp. 447–463.
  * [11] J. Liu, H. Sun, and J. Katto, “Learned image compression with mixed transformer-cnn architectures,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 14 388–14 397.
  * [12] W. Jiang, J. Yang, Y. Zhai, P. Ning, F. Gao, and R. Wang, “Mlic: Multi-reference entropy model for learned image compression,” in _Proceedings of the 31st ACM International Conference on Multimedia_ , 2023, pp. 7618–7627.
  * [13] W. Jiang and R. Wang, “MLIC$^{++}$: Linear complexity multi-reference entropy modeling for learned image compression,” in _ICML 2023 Workshop Neural Compression: From Information Theory to Applications_ , 2023. [Online]. Available: https://openreview.net/forum?id=hxIpcSoz2t
  * [14] Y. Hu, W. Yang, Z. Ma, and J. Liu, “Learning end-to-end lossy image compression: A benchmark,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 8, pp. 4194–4211, 2021.
  * [15] Y. Qian, Z. Tan, X. Sun, M. Lin, D. Li, Z. Sun, L. Hao, and R. Jin, “Learning accurate entropy model with global reference for image compression,” in _International Conference on Learning Representations_ , 2020.
  * [16] G. Gao, P. You, R. Pan, S. Han, Y. Zhang, Y. Dai, and H. Lee, “Neural image compression via attentional multi-scale back projection and frequency decomposition,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 14 677–14 686.
  * [17] Y. Wu, X. Li, Z. Zhang, X. Jin, and Z. Chen, “Learned block-based hybrid image compression,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 32, no. 6, pp. 3978–3990, 2021.
  * [18] Y. Xie, K. L. Cheng, and Q. Chen, “Enhanced invertible encoding for learned image compression,” in _Proceedings of the 29th ACM international conference on multimedia_ , 2021, pp. 162–170.
  * [19] L. Theis, W. Shi, A. Cunningham, and F. Huszár, “Lossy image compression with compressive autoencoders,” in _International C/ballonference on Learning Representations_ , 2016.
  * [20] Z. Guo, Z. Zhang, R. Feng, and Z. Chen, “Causal contextual prediction for learned image compression,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 32, no. 4, pp. 2329–2341, 2021.
  * [21] Y. Bao, F. Meng, C. Li, S. Ma, Y. Tian, and Y. Liang, “Nonlinear Transforms in Learned Image Compression From a Communication Perspective,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 33, no. 4, pp. 1922–1936, Apr. 2023.
  * [22] Y. Wang, D. Liu, S. Ma, F. Wu, and W. Gao, “Ensemble Learning-Based Rate-Distortion Optimization for End-to-End Image Compression,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 31, no. 3, pp. 1193–1207, Mar. 2021.
  * [23] R. Zou, C. Song, and Z. Zhang, “The Devil Is in the Details: Window-based Attention for Image Compression,” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, pp. 17 471–17 480.
  * [24] B. Bross, Y.-K. Wang, Y. Ye, S. Liu, J. Chen, G. J. Sullivan, and J.-R. Ohm, “Overview of the Versatile Video Coding (VVC) Standard and its Applications,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 31, no. 10, pp. 3736–3764, Oct. 2021.
  * [25] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [26] Y. Bai, X. Yang, X. Liu, J. Jiang, Y. Wang, X. Ji, and W. Gao, “Towards end-to-end image compression and analysis with transformers,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 1, 2022, pp. 104–112.
  * [27] Y. Bai, X. Liu, K. Wang, X. Ji, X. Wu, and W. Gao, “Deep lossy plus residual coding for lossless and near-lossless image compression,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 46, no. 5, pp. 3577–3594, 2024.
  * [28] F. Kingma, P. Abbeel, and J. Ho, “Bit-swap: Recursive bits-back coding for lossless compression with hierarchical latent variables,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 3408–3417.
  * [29] J. Townsend, T. Bird, and D. Barber, “Practical lossless compression with latent variables using bits back coding,” in _7th International Conference on Learning Representations, ICLR 2019_ , vol. 7. International Conference on Learning Representations (ICLR), 2019.
  * [30] S. Zhang, N. Kang, T. Ryder, and Z. Li, “iflow: Numerically invertible flows for efficient lossless compression via a uniform coder,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 5822–5833, 2021.
  * [31] Y. Bai, X. Liu, W. Zuo, Y. Wang, and X. Ji, “Learning scalable $\ell_{\infty}$-constrained near-lossless image compression via joint lossy image and residual compression,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2021, pp. 11 946–11 955.
  * [32] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” _arXiv preprint arXiv:1312.6114_ , 2013.
  * [33] X. Chu, Z. Tian, B. Zhang, X. Wang, and C. Shen, “Conditional positional encodings for vision transformers,” in _The Eleventh International Conference on Learning Representations_ , 2022.
  * [34] Y. Yan, F. Hu, J. Chen, N. Bhendawade, T. Ye, Y. Gong, N. Duan, D. Cui, B. Chi, and R. Zhang, “Fastseq: Make sequence generation faster,” in _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations_ , 2021, pp. 218–226.
  * [35] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” in _International Conference on Learning Representations_ , 2020.
  * [36] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2015.
  * [37] J. Bégaint, F. Racapé, S. Feltman, and A. Pushparaja, “Compressai: a pytorch library and evaluation platform for end-to-end compression research,” _arXiv preprint arXiv:2011.03029_ , 2020.
  * [38] A. Rogozhnikov, “Einops: Clear and reliable tensor manipulations with einstein-like notation,” in _International Conference on Learning Representations_ , 2022. [Online]. Available: https://openreview.net/forum?id=oapKSVM2bcj
  * [39] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga _et al._ , “Pytorch: An imperative style, high-performance deep learning library,” _Advances in neural information processing systems_ , vol. 32, 2019.
  * [40] E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super-resolution: Dataset and study,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops_ , July 2017.
  * [41] M. Hodosh, P. Young, and J. Hockenmaier, “Framing image description as a ranking task: Data, models and evaluation metrics,” _The Journal of Artificial Intelligence Research_ , vol. 47, p. 853, 2013.
  * [42] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” _International Journal of Computer Vision (IJCV)_ , vol. 115, no. 3, pp. 211–252, 2015.
  * [43] E. Kodak, “Kodak lossless true color image suite (photocd pcd0992),” http://r0k.us/graphics/kodak, 1993.
  * [44] N. Asuni and A. Giachetti, “Testimages: a large-scale archive for testing visual devices and basic image processing algorithms.” in _STAG_ , 2014, pp. 63–70.
  * [45] CLIC Challenge on Learned Image Compression. [Online]. Available: http://compression.cc/
  * [46] F. Bellard, “Bpg image format,” https://bellard.org/bpg, 2015.
  * [47] W. Jiang, “Unofficial elic,” https://github.com/JiangWeibeta/ELIC, 2022.
  * [48] Y. Xie, K. L. Cheng, and Q. Chen, “Enhanced Invertible Encoding for Learned Image Compression,” in _Proceedings of the 29th ACM International Conference on Multimedia_. ACM, pp. 162–170.

|  Daxin Li received his B.S. degree in the Faculty of Computation from Harbin
Institute of Technology, Harbin, China, in 2021. He is currently pursuing his
Ph.D. degree in the Faculty of Computation from Harbin Institute of
Technology. His research interests lie primarily in the areas of image and
video compression, as well as deep learning applications.
---|---
|  Yuanchao Bai (Member, IEEE) received the B.S. degree in software
engineering from Dalian University of Technology, Liaoning, China, in 2013. He
received the Ph.D. degree in computer science from Peking University, Beijing,
China, in 2020. He was a postdoctoral fellow in Peng Cheng Laboratory,
Shenzhen, China, from 2020 to 2022. He is currently an assistant professor
with the School of Computer Science and Technology, Harbin Institute of
Technology, Harbin, China. His research interests include image/video
compression and processing, deep unsupervised learning, and graph signal
processing.
---|---
|  Kai Wang received the B.S. degree in software engineering from Harbin
Engineering University, Harbin, China, in 2020 and received the M.S. degree of
electronic information in software engineering from Harbin Institute of
Technology, Harbin, China, in 2022. He is currently pursuing the docter degree
in electronic information in Harbin Institute of Technology, Harbin, China.
His research interests include image/video compression and deep learning.
---|---
|  Junjun Jiang (Senior Member, IEEE) received the B.S. degree in mathematics
from Huaqiao University, Quanzhou, China, in 2009, and the Ph.D. degree in
computer science from Wuhan University, Wuhan, China, in 2014. From 2015 to
2018, he was an Associate Professor with the School of Computer Science, China
University of Geosciences, Wuhan. From 2016 to 2018, he was a Project
Researcher with the National Institute of Informatics (NII), Tokyo, Japan. He
is currently a Professor with the School of Computer Science and Technology,
Harbin Institute of Technology, Harbin, China. His research interests include
image processing and computer vision.
---|---
|  Xianming Liu (Member, IEEE) is a Professor with the School of Computer
Science and Technology, Harbin Institute of Technology (HIT), Harbin, China.
He received the B.S., M.S., and Ph.D degrees in computer science from HIT, in
2006, 2008 and 2012, respectively. In 2011, he spent half a year at the
Department of Electrical and Computer Engineering, McMaster University,
Canada, as a visiting student, where he then worked as a post-doctoral fellow
from December 2012 to December 2013. He worked as a project researcher at
National Institute of Informatics (NII), Tokyo, Japan, from 2014 to 2017. He
has published over 60 international conference and journal publications,
including top IEEE journals, such as T-IP, T-CSVT, T-IFS, T-MM, T-GRS; and top
conferences, such as CVPR, IJCAI and DCC. He is the receipt of IEEE ICME 2016
Best Student Paper Award.
---|---
|  Wen Gao (Fellow, IEEE) received the Ph.D. degree in electronics engineering
from The University of Tokyo, Japan, in 1991. He is currently a Boya Chair
Professor in computer science at Peking University. He is the Director of Peng
Cheng Laboratory, Shenzhen. Before joining Peking University, he was a
Professor with Harbin Institute of Technology from 1991 to 1995. From 1996 to
2006, he was a Professor at the Institute of Computing Technology, Chinese
Academy of Sciences. He has authored or coauthored five books and over 1000
technical articles in refereed journals and conference proceedings in the
areas of image processing, video coding and communication, computer vision,
multimedia retrieval, multimodal interface, and bioinformatics. He served on
the editorial boards for several journals, such as ACM CSUR, IEEE TRANSACTIONS
ON IMAGE PROCESSING (TIP), IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO
TECHNOLOGY (TCSVT), and IEEE TRANSACTIONS ON MULTIMEDIA (TMM). He served on
the advisory and technical committees for professional organizations. He was
the Vice President of the National Natural Science Foundation (NSFC) of China
from 2013 to 2018 and the President of China Computer Federation (CCF) from
2016 to 2020. He is the Deputy Director of China National Standardization
Technical Committees. He is an Academician of the Chinese Academy of
Engineering and a fellow of ACM. He chaired a number of international
conferences, such as IEEE ICME 2007, ACM Multimedia 2009, and IEEE ISCAS 2013.
---|---