Efficient Trimmed Convolutional Arithmetic Encoding
for Lossless Image Compression
Mu Li, Shuhang Gu, David Zhang
Department of Computing, Hong Kong Polytechnic University

arXiv:1801.04662v1 [cs.CV] 15 Jan 2018

csmuli@comp.polyu.edu.hk, shuhanggu@gmail.com, csdzhang@comp.polyu.edu.hk

Wangmeng Zuo∗
School of Computer Science and Technology, Harbin Institute of Technology
cswmzuo@gmail.com

Abstract

1. Introduction
Data compression, which aims to encode the information
using fewer bits, has been extensively studied for decades.
Compression methods can be classified into lossy and lossless ones. For lossless compression the reconstructed data
is an exact replica of the original data, while for lossy
compression the approximated reconstruction of the original data is allowed. Even though lossy and lossless compression algorithms are designed for different objectives,
they share similar compression pipeline, i.e., pre-processing
(e.g., analysis transform), entropy coding, entropy decoding, and post-processing (e.g., synthesis transform). Moreover, lossless compression can also be adopted as a component in lossy compression to encode the intermediate
codes generated by the lossy compression system, which
further indicates the indispensability of lossless compression in data compression.
Entropy encoding has been widely adopted for lossless
compression, where each input symbol is encoded with a
codeword with the length approximately proportional to the
negative logarithm of the probability. According to Shannon’s source coding theorem [11], the optimal code length
for a symbol should be − logb P , where b is the number of
symbols used to make output codes and P is the probability
of the input symbol. Following Shannon’s theorem, several
entropy encoding techniques, e.g., run length coding, Huffman coding, Golomb-Rice coding, and arithmetic coding,
have been developed.
One key issue of entropy encoding is to predict the probability of the current symbol to be encoded. In image compression, several approaches have been proposed to address
this issue. As for JPEG, the frequency of the symbols is
directly counted and the Huffman coding is used to compress the codes. In JPEG 2000, the EBCOT coder [9] is
employed to model the context and approximate the proba-

Arithmetic encoding is an essential class of coding techniques which have been widely used in various data compression systems and exhibited promising performance.
One key issue of arithmetic encoding method is to predict
the probability of the current symbol to be encoded from its
context, i.e., the preceding encoded symbols, which usually
can be executed by building a look-up table (LUT). However, the complexity of LUT increases exponentially with
the length of context. Thus, such solutions are limited in
modeling large context, which inevitably restricts the compression performance. Several recent convolutional neural
network (CNN) and recurrent neural network (RNN)-based
solutions have been developed to account for large context,
but are still costly in computation. The inefficiency of the
existing methods are mainly attributed to that probability
prediction is performed independently for the neighbouring symbols, which actually can be efficiently conducted by
shared computation. To this end, we propose a trimmed
convolutional network for arithmetic encoding (TCAE) to
model large context while maintaining computational efficiency. As for trimmed convolution, the convolutional kernels are specially trimmed to respect the compression order and context dependency of the input symbols. Benefited
from trimmed convolution, the probability prediction of all
symbols can be efficiently performed in one single forward
pass via a fully convolutional network. Experiments show
that our TCAE attains better compression ratio in lossless
gray image compression, and can be adopted in CNN-based
lossy image compression to achieve state-of-the-art ratedistortion performance with real time encoding speed.

∗ Corresponding

Author

1

bility, and the Binary Arithmetic Coding-MQ-Coder is utilized to compress the codes. JPEG-LS assumes that smaller
values are more likely to occur in the residual than larger
ones, and compress the codes with Golomb-Rice coding.
In H.264/AVC, the context-adaptive binary arithmetic coding (CABAC) [8] is introduced to model the context of
the preceding two encoded symbols for compressing the
codes with arithmetic coding. CABAC adopts look-up table (LUT) to predict the probability of the current symbol to
encode. However, the complexity of LUT increases exponentially along with the length of context, making CABAC
impractical in modeling large context.
Motivated by the powerful ability of deep neural networks in image representation, a number of deep models,
including PixCNN and PixelRNN, have been proposed to
model highly complex and long-range dependence between
pixels. As for image compression, Toderici et al. [13]
present a recurrent neural network (RNN) based model to
compress the image and further compress the binary codes
with arithmetic coding by estimating the probability of the
bit with a binary RNN network. Due to that one forward
pass is required to predict each bit, the RNN based encoder [13] is computationally very costly. Li et al. [7]
propose a content-weighted convolutional neural network
(CNN) architecture for lossy image compression, where a
shallow CNN is introduced to predict the probability of the
current bit to encode from its context cuboid. Even though
the CNN solution [7] speeds up the RNN based encoder, it
also performs probability prediction independently for the
neighbouring symbols and remains computationally expensive.
In this paper, we proposed a trimmed convolutional network for arithmetic encoding (TCAE) which can greatly
speed up the process of probability prediction. TCAE
adopts the fully convolutional network architecture, and further introduces a new class of trimmed convolution operations to make it feasible to efficiently perform probability
prediction to all bits by exploiting shared computation. In
contrast, the standard fully convolutional network should
take both the preceding encoded symbols and the neighboring non-encoded ones in context modeling, and thus cannot
be directly adopted. As for trimmed convolution, a binary
mask is incorporated with each convolution kernel to eliminate the interference of neighboring non-encoded symbols
in probability prediction, resulting in the trimmed convolution kernel. With the introduction of trimmed convolution,
we can respect the compression order and context dependency of the symbols while maintaining the efficiency of
fully convolutional network.
Generally, the settings of mask are different for the input
layer and hidden convolution layers in TCAE. As illustrated
in Fig. 1a, on the one hand, the bit at the centre location
from the input image denotes the current bit to be encoded,

and thus should be excluded from the context. On the other
hand, in the hidden convolution layer of TCAE, the bit at the
centre location only conveys its context information. Therefore, as shown in Fig. 1c, the centre location should also be
included in the context. Moreover, these mask settings can
also be extended to account for multiple filters and 3D data,
which will be explained in more details in Section 3.
Our TCAE can be directly applied to compress any 3D
binary cubiod (e.g., 8-bit representation of gray image)
without any preprocessing. In the experiments, we first
utilize TCAE to compress gray image, and our TCAE can
achieve higher compression ratio in comparison with PNG,
JPEG-LS and JPEG2000-LS. Furthermore, TCAE can also
be incorporated with the existing CNN-based lossy compression system to compress the intermediate codes. Using
the system [7] as an example, TCAE is deployed to both
the binary codes and importance map. Compared with the
convolutional entropy encoder [7], our TCAE can not only
improve the compression performance due to the consideration of large context, but also be significantly faster in
terms of encoding speed. Benefitted from trimmed convolution, the compression system [7] with TCAE can encode
the image in real time, and takes less than 1 second (s) to
obtain the codes of 24 images with the size of 752 × 496.
To sum up, the contribution of this work is three-fold:
• Trimmed convolution is incorporated with fully convolutional network to efficiently perform probability
prediction to all bits in one single forward pass.
• A trimmed convolutional arithmetic encoder (TCAE)
is developed to encode gray image. TCAE can also
be used to compress the intermediate codes in CNNbased lossy compression system.
• Experiments show that for lossless gray image compression TCAE can achieve higher compression ratio than PNG and JPEG2000-LS. Following the CNNbased lossy image compression system in [7], TCAE
can achieve real time encoding speed as well as improved rate-distortion performance.
The remainder of this paper is organized as follows. Section 2 briefly surveys some related works on lossless image
compression and entropy encoding. Section 3 describes the
proposed trimmed convolutional arithmetic encoder. Section 4 reports the results on lossless gray image compression and encoding intermediate codes of CNN-based compression system [7]. Finally, Section 5 ends the paper with
several concluding remarks.

2. Related Work
2.1. Lossless image compression standards
Lossless image compression has been investigated for
decades to compress image into a smaller size without los2

Figure 1: Illustration of the trimmed convolution operations used in the input layer and the hidden layers. The color blocks
represents codes or features has been calculated according to the encoding order. The red blocks represent the current code
or feature to be encoded. Each color corresponds to one layer of code maps and shares trimmed convolution kernels with the
same shape. The blank area in the code maps or feature maps represent the non-encoded codes or features along with the
encoding order.
ing any information. Most existing methods share similar pipeline i.e., building a statistical model and then using it to guide the mapping of input data stream to bit sequences by encoding the high frequency symbol with fewer
bits than low frequency symbol. JPEG-LS [16] takes use of
the LOCO-I algorithm which adopts the median edge detection predictor to predict the image and then compress the
residual with the context-based entropy coding. JPEG2000LS, the lossless version of JPEG 2000 standard [12], is
based on reversible integer wavelet transform (biorthogonal 3/5). PNG [2] first uses a lossless filter to transform the
images and then compresses the transformed data with the
DEFLATE algorithm which is a combination of LZ77 and
Huffman coding. Both TIFF and GIF apply Lemple-ZivWelch (LZW) algorithm [17] which is a dictionary based
method for lossless image compression.

sequence are stored as the symbol and its count. Huffman
coding is a variable-length coding scheme that encodes high
frequency symbols with short codeword and low frequency
symbols with long codeword. In Golomb-Rice code, the
input value is first divided into two parts, i.e., quotient and
remainder, and truncated binary encoding is then adopted to
encode the remainder. Arithmetic coding first predicts the
probability of the current symbol to be encoded and then
uses it to divide the current interval into sub-intervals for
encoding the updated sequence.

2.3. Deep networks based entropy encoding
PixelCNN [15] and PixelRNN [10] have been proposed for image generation, and have exhibited their ability in modeling highly complex and long-range dependence.
Therefore, it is also encouraging to utilize CNNs and RNNs
for predicting the probability of a bit from its context (i.e.,
the preceding encoded bits). Toderici et al. [14] learn a
RNN with residual learning for image compression and
adopt a binary RNN to predict the probability of the current
bit from its context, which is further utilized in arithmetic
encoding. However, the probability of each bit should be
estimated with one forward pass, making binary RNN estimator computational very inefficient. Li et al. [7] introduce a content-weighted CNN model for image compres-

2.2. Entropy encoding
Entropy encoding is a lossless data compression scheme
which has played an essential role in both lossy and lossless image compression systems. Run length encoding,
Huffman coding [5], Golomb-Rice coding [4] and arithmetic coding [18] are several representative entropy encoding techniques. Run length coding is a very simple lossless data compression scheme, where a consecutive symbol
3

H − 1, 0 ≤ k ≤ C − 1}. As illustrated in Fig. 4, beginning
at x0,0,0 , we follow the following order to encode X: (i)
xi+1,j,k is encoded after xi,j,k until i = W − 1; (ii) when
i = W − 1, x0,j+1,k is encoded after xi,j,k until j = H − 1;
(iii) when i = W − 1 and j = H − 1, x0,0,k+1 is encoded
after xi,j,k .
One key issue of arithmetic encoding is to predict the
probability of a symbol from its context, i.e., the preceding encoded symbols. Given the position (p, q, r) of 3D
cuboid, we denote its full context as CT X(xp,q,r ). Taking
the encoding order of X into account, CT X(xp,q,r ) can be
defined as CT X(xp,q,r ) = {xi,j,k |{k < r} ∨ {k = r, j <
q} ∨ {k = r, j = q, i < p}}. Unfortunately, the length
of the full context CT X(xp,q,r ) is not fixed and varies by
the position (p, q, r), making it difficult to learn probability
prediction model on CT X(xp,q,r ). Note that the context
close to the bit to be encoded plays more important role in
probability prediction. Thus we adopt a fixed length context defined as CT Xf (xp,q,r ) = {xi,j,k |{r − c ≤ k <
r, |i − p| ≤ w, |j − q| ≤ h} ∨ {k = r, q − h ≤ j <
q, |i − p| ≤ w} ∨ {k = r, j = q, p − w ≤ i < p}}. It is
natural to expect that large context (i.e., large h, w, c) benefits probability prediction, and in the following we suggest
to use trimmed convolutional networks for improved and
efficient arithmetic encoding.

sion as well as a convolutional entropy encoder to predict
the probability of the bit from a specially defined context
cuboid. Compared with the binary RNN estimator, the convolutional entropy encoder is more efficient, but repeating
calculation remains inevitable due to the overlap of the context of near neighbouring bits. Actually, the inefficiency of
the methods in [14, 7] is mainly attributed to that probability
prediction is performed independently for the neighbouring
symbols. In this work, we introduce a class of trimmed convolution operations to make that the probability prediction
can be efficiently performed with the help of shared computation. By stacking multiple trimmed convolution layers,
our TCAE can also account for large context to improve
compression performance.

3. Trimmed Convolutional Arithmetic Encoding
In this section, we present our trimmed convolutional
arithmetic encoding model (TCAE). As illustrated in
Fig. 2b, TCAE utilizes the trimmed convolutional networks
to predict the probability of codes from their context in one
single forward pass. Before introducing the trimmed convolutional network, we first describe arithmetic coding and
coding context in Sections 3.1 and 3.2. Then trimmed convolutional network is provided for context modeling and
probability prediction in Section 3.3, and the model objective is described in Section 3.4.

3.3. Trimmed convolutional networks
With the fixed length context defined above, we can extract a (2w + 1) × (2h + 1) × (c + 1) cuboid xp,q,r =
{xi,j,k |{r − c ≤ k ≤ r, p − w ≤ i ≤ p + w, q − h ≤ j ≤
q + h}. As shown in Fig. 2a, the blue voxels represent the
encoded bits before xp,q,r , the red voxel denotes the next bit
to be encoded, and the gray ones are the other non-encoded
bits. Then, for each location xp,q,r , Li et al. [7] adopt a
CNN architecture of three convolution layers followed by
three fully connected layers to predict xp,q,r from its context cuboid xp,q,r .
One main obstacle that restricts the share computation
of probability prediction is the introduction of non-encoded
bits (i.e., the red and gray voxels in Fig. 2a). In [7], Li et
al. suggest to assign a special default value to the current
non-encoded bits in the context cuboid xp,q,r . For example,
when predicting xp,q,r , xp,q,r should be replaced with default value [7]. When encoding the next bit after xp,q,r , the
original value of xp,q,r should be used in context modeling.
Therefore, fully convolutional network cannot be directly
utilized for context modeling, and the probability prediction is performed independently for each xp,q,r , leading to
repeating computation and encoding inefficiency [7].
Fortunately, the value of each location only have two
choices, i.e., the original value xp,q,r and the default value
for non-encoded bit. Moreover, given the voxel xp,q,r to
encode, the positions of all non-encoded bits are also fixed.

3.1. Arithmetic coding
Arithmetic encoding is an entropy encoding scheme for
lossless data compression. Given a string of symbols, entropy encoding assigns fewer bits for frequently occurring
symbols and more bits for not-so-frequently occurring ones.
Different from other entropy encoding algorithms such as
Huffman coding, arithmetic coding encodes the entire string
of symbols into a single number in the interval of [0, 1].
Denote by k the number of symbols occurred in the coding system. Given a new symbol and the current interval, arithmetic coding first predicts the probabilities that the
new symbol belongs to each value, and the current interval is further divided according to negative logarithm of the
predicted probabilities. In order to encode the updated sequence, the current interval is then updated based on the
ground truth of the new symbol as well as its predicted probability. For example, Fig. 3 illustrates the arithmetic coding
result of a sequence (0, 2, 3) for a coding system with the
symbols (0, 1, 2, 3) and the discrete distribution with probabilities (0.6, 0.2, 0.1, 0.1).

3.2. Coding schedule and context of 3D cuboid
In this work, we focus on the arithmetic encoding of 3D
binary code cuboid X = {xi,j,k |0 ≤ i ≤ W − 1, 0 ≤ j ≤
4

(a) Framework for traditional convolutional network for entropy probability estimator used in [7]. One bit is processed as a sample.

(b) Framework for trimmed convolutional network for entropy probability estimator. The whole code maps are processed as a sample.

Figure 2: Comparison between trimmed convolutional networks and traditional convolutional networks for entropy probability estimator. Different color means different coding planes.

Figure 3: Arithmetic encoding of a sequence (0,2,3) for a
coding system with the symbols (0,1,2,3) and the i.i.d distribution with probabilities (0.6,0.2,0.1,0.1)

Figure 4: Coding schedule for binary code cuboid

Without loss of generality, we let the default value be 0.
Thus, we can introduce a group of trimmed convolution operators to incorporate with fully convolutional network for
context modeling. Denote by w0 a 3D convolution kernel
w0 = {wt,i,j,k | − w0 ≤ i ≤ w0 , −h0 ≤ j ≤ h0 , −t ≤ k ≤
C − t − 1}. Then the convolution result (X ∗ w0 ) at the

location (p, q, r) can be written as,
X
(X ∗ w0 )(p, q, r) =

0
xl,m,n wt,i,j,k
.

l−i=p,m−j=q,n−k=r,t=r

(1)
5

However, both the context and non-encoded bits have effect
on the convolution result (X ∗ w0 )(p, q, r). We note that
the positions of non-encoded bits are fixed and pre-defined
with respect to w0 , and thus can introduce a mask m of
{0, 1} to exclude them in convolution. To this end, mi,j,k is
defined as 1 if xp+i,q+j,r+k is encoded before xp,q,r , and 0
otherwise. The trimmed convolution is thus defined as,
X1 = X ∗ (m ◦ w0 ),

(2)

where ∗ denotes the convolution operator, and ◦ denotes the
element-wise multiplication operator. With trimmed convolution, we can safely avoid the effect of non-encoded bits in
context modeling while maintaining the efficiency of fully
convolutional network for predicting probabilities of all bits
in one forward pass.
In the following, we first use single convolution kernel as
an example to explain the setting of m, and then extend it
to multi-kernel case. The settings of the mask m are different for the input layer and the hidden layers. For the input
layer, when predicting the probability of xp,q,r , both xp,q,r
and the bits encoded after xp,q,r should be masked out in
trimmed convolution. Following the definition of context in
Section 3.2, we define the mask m0 for the input layer as,
(
1, if(k < 0)∨(k = 0, j < 0)∨(k = 0, j = 0, i < 0)
0
mijk =
0, otherwise
(3)
When turning to the hidden layer Xd (d ≥ 1), we note
that xdp,q,r only conveys the context information of xp,q,r
and should not be excluded in the further context modeling.
Therefore, we modify the definition of the mask md (d ≥ 1)
for hidden layer as,
(
1, if(k < 0)∨(k = 0, j < 0)∨(k = 0, j = 0, i ≤ 0)
d
mijk =
0, otherwise.
(4)
Using a 3D convolution kernel with the size of 5 × 5 ×
C as an example, Fig. 5 illustrates the representative mask
planes with respect to k. As shown in Fig. 5ab, when k < 0
(k > 0), the k-th mask plane is a matrix of 1s (0s) for both
the input layer and the hidden layers. when k = 0, the
center position is masked out in the mask plane for the input
layer (see Fig. 5c) but are included for the hidden layers (see
Fig. 5d).
The trimmed convolution in Eqn. (2) only uses one convolution kernel in each layer, which is limited in probability prediction. Thus, we extend the trimmed convolution to the multiple convolution kernel form. Suppose there
are gin groups of feature maps X d = {Xd,1 , ..., Xd,gin }
in the d-th layer and gout groups of feature maps X d+1 =
{Xd+1,1 , ..., Xd+1,gout } in the (d + 1)-th layer. Each group
of feature map has the same size with the input cuboid X.

Figure 5: Mask planes with respect to k for trimmed convolution kernels with the size of 5 × 5. The gray value and red
value denotes 1 and the white value denotes 0. (a) k < 0,
(b) k > 0, (c) k = 0 for the input layer, (d) k = 0 for
the hidden layers, (e) 3D kernel mask for input layer, (f) 3D
kernel mask for hidden layer.

The group trimmed convolution is then defined as,
0

Xd+1,g =

gin
X

0

Xd,g ∗ (md ◦ wd,g,g ),

(5)

g=1

where Xd,g denotes the g-th group of feature map in X d ,
0
Xd+1,g denotes the g 0 -th group of feature map in X d+1 ,
0
md is the mask for the d-th layer, and wd,g,g is the convo0
lution kernel to connect Xd,g and Xd+1,g .

3.4. Model objective and learning
0

Given all the model parameters W = {wd,g,g } , the
output of the trimmed convolutional network can be written
t
as F (X; W). F (X; W) includes m parts, (F (X; W))p,q,v
denotes the predicted probability of xp,q,v = t with t =
0, . . . , m − 1. And m is the number of different code value
in the input code maps. We adopt the length of the codes
after arithmetic encoding as the model objective,
`(W; X) =

X m−1
X

t

−s(xp,q,v , t) log2 (F (X; W))p,q,v

p,q,v t=0

(6)
where s(xp,q,v , t) = 1 when xp,q,v = t, and s(xp,q,v , t) = 0
otherwise. According to the Shannon’s theorem, the compression ratio is defined as the ratio between uncompressed
size and compressed size, and can be written as
`(W; X)
1
1
1
=
=
[
r(e)
CHW log2 m
log2 m CHW
X m−1
X
p,q,v t=0

6

t

−s(xp,q,v , t) log2 (F (X; W))p,q,v .(7)

4.1. Network architecture and parameter setting

Thus, it is reasonable to use the objective in Eqn. (6) to
learn probability prediction model for entropy encoding.
By minimizing the model objective defined in Eqn.
(6), the trimmed convolutional network F (X; W) can be
learned from training data in an end-to-end manner. In this
work, we adopt the ADAM solver [6] to learn the model
parameters. The model is trained with the learning rate of
3 × 10−4 , 1 × 10−4 , 3.33 × 10−5 and 1.11 × 10−5 . The
smaller learning rate is adopted until the objective with the
larger one stops decreasing.

Tables 1 and 2 provide the architectures of the trimmed
convolutional networks for lossless gray image compression and the compression of intermediate codes generated
by CNN-based lossy image compression [7], respectively.
In TCAE, we also introduce the trimmed residual block,
which consists of two trimmed convolution layers with each
followed by the ReLU nonlinearity. The skip connection is
also added from the input to the output of the residual block.
For defining the model objective in Eqn (6), feature map rearrangement is adopted to reshape the input and output of
the trimmed convolutional network. Concretely, the input
is reshaped into a one dimensional vector with the size of
CHW × 1, while the output is reshaped into a 2D matrix
with the size of CHW × 2.
All the networks are trained on 10, 000 high quality images from the ImageNet [3] and tested on the Kodak PhotoCD image dataset. Due to that TCAE is a fully convolutional network, it can be trained and tested using images
with any size. In the experiments, we set the image size
as 128 × 128 for lossless gray image compression. Note
that the input of TCAE is required to be 3D binary cuboid.
We thus transform the discrete gray image to its 8-bit representation, resulting in a 128 × 128 × 8 3D binary cuboid.
With trimmed convolution, four residual blocks are stacked
upon two 3D convolution layers, and another 3D convolution layer is finally deployed for probability prediction.
When TCAE is incorporated with the lossy compression
system [7], there are four kinds of codes, i.e., binary codes
with 64 channels, binary codes with 128 channels, binary
importance map with 4 channels, and binary importance
map with 5 channels. For binary codes with 64 (128) channels, the parameter C in Table 2 is 64 (128) and the parameter p is 4, which means that 4 3D convolution kernels are deployed in this layer. Note that the binary codes are trimmed
with the importance map. To distinguish the trimmed codes
from the others, we reassign 0 to trimmed codes, and reassign 1(2) to untrimmed codes with 0(1). Consequently, the
probabilities for the three values (i.e., 0, 1, and 2) are required in the output layer, and we thus set q = 3. Due to
that the trimmed codes are not required to encode, we leave
out the value 0 in calculating the ratio loss. As for binary
importance maps with 4(5) channels, we set C = 4(5) and
p = 16. Due to the binary representation of importance
map, the probabilities for the two values (i.e., 0 and 1) are
required in the output layer, and we thus set q = 2.

4. Experiments
Two groups of experiments are conducted to test the proposed trimmed convolutional arithmetic encoding (TCAE).
The first is the lossless compression of gray image. To satisfy the requirement of 3D binary cuboid, we take the 8bit representation of gray image as the input, Our TCAE is
compared with several representative lossless image compression methods, i.e., PNG, TIFF, GIF, JPEG-LS, and
JPEG2000-LS. The second group of experiments is the incorporation with CNN-based lossy compression. We use
TCAE to compress the binary codes and importance map
generated by [7], and compare the rate-distortion performance with the baseline method [7], JPEG, JPEG-2000, and
Ballé et al. [1]. Moreover, the source code of our TCAE are
online available1 .
Table 1: Architecture of trimmed convolutional network for
lossless gray image compression.
Layer
Input
8 × 8 × 5 × 5 × 8 trimmed conv, pad 2, stride 1
64 × 8 × 5 × 5 × 8 trimmed conv, pad 2, stride 1
Trimmed residual block, 8 3D filters
Trimmed residual block, 8 3D filters
Trimmed residual block, 8 3D filters
Trimmed residual block, 8 3D filters
16 × 8 × 5 × 5 × 8 trimmed conv, pad 2, stride 1

Activation size
128 × 128 × 8
8 × 128 × 128 × 8
8 × 128 × 128 × 8
8 × 128 × 128 × 8
8 × 128 × 128 × 8
8 × 128 × 128 × 8
8 × 128 × 128 × 8
2 × 128 × 128 × 8

Table 2: Architecture of trimmed convolutional network for
compressing the binary codes and importance map generated by [7].
Layer
Input
p × C × 5 × 5 × C trimmed conv, pad 2, stride 1
pC × C × 5 × 5 × C trimmed conv, pad 2, stride 1
Trimmed residual block, p 3D filters
Trimmed residual block, p 3D filters
Trimmed residual block, p 3D filters
qC × 5 × 5 × C trimmed conv, pad 2, stride 1

Activation size
16 × 16 × C
p × 16 × 16 × C
p × 16 × 16 × C
p × 16 × 16 × C
p × 16 × 16 × C
p × 16 × 16 × C
q × 16 × 16 × C

4.2. Lossless gray image compression
For lossless gray image compression, our TCAE is simply deployed to the 8-bit representation of gray image without any transform. We compare TCAE with several popular lossless compression standards, including GIF, TIFF,
PNG, JPEG-LS and JPEG2000-LS. For PNG, JPEG2000-

1 https://github.com/limuhit/TrimmedConvolution

7

curves obtained using our TCAE and convolutional entropy
encoder [7]. Here, SSIM, MSE, and PSNR are adopted as
the distortion performance metrics, and bit per pixel (bpp)
is used as the indicator of compression rate. As shown in
Fig. 6, our TCAE achieves comparable performance with
convolutional entropy encoder [7]. When bpp ≥ 0.4, TCAE
performs better than convolutional entropy encoder [7],
which can be attributed to the ability of TCAE in modeling large context. Fig. 7 further compares the rate-distortion
curves obtained using Li et al. [7] with TCAE, JPEG, JPEG
2000, and Ballé [1]. When bpp ≥ 0.4, Li et al. [7] with
TCAE outperforms the competing methods, regardless of
any distortion metrics. The result indicates that TCAE can
be incorporated with the CNN-based lossy image compression to boost rate-distortion performance.

Figure 6: The effect of TCAE on the rate-distortion performance of the lossy image compression system [7]

Table 4: Run time (in seconds, s) used for encoding the
intermediate codes in [7].
Group
Group 1
Group 2
Group 3
Group 4
Group 5

Figure 7: Comparision of rate-distortion performance of the
TCAE extention of [7] and other lossy image compression
methods.

Binary
0.035
0.035
0.035
0.035
0.161

TCAE
Importance
0.003
0.003
0.003
0.003
0.005

Total
0.038
0.038
0.038
0.038
0.166

The most prominent merit of TCAE is its efficiency in
encoding due to the incorporation of trimmed convolution
and fully convolutional network. To illustrate this, Table 4
reports the run time for encoding the intermediate codes [7].
Here, Groups 1∼5 represent the five settings of model parameters [7] to obtain the compression models at different
bpps in the range of [0.118, 0.671]. All the experiments
are conducted on a computer with the GTX TitanX GPU of
12GB memory. The reported time is based on the encoding
of the importance map and binary codes of a 752 × 496 × 3
image, and we do not include the time for generating the
intermediate codes. In general, it takes about 0.051 s and
0.098 s to generate the intermediate codes with 64 and 128
channels, respectively. From Table 4, it can be seen that our
TCAE is 7∼20 times faster than the entropy encoder used
in [7]. With the introduction of trimmed convolution, our
TCAE can be performed at a near real time speed (e.g., ¿25
fps ) to encode the intermediate codes of images with the
size of 752 × 496 × 3.

LS, JPEG-LS, GIF and TIFF, all the compression ratio is
calculated with the results generated with the Matlab2015b.
The average compression ratio on 24 image from Kodak set
are given in Table 3. Our TCAE achieves the best compression ratio (2.00), which is much better than the second best
method (1.79 for JPEG2000-LS).
Table 3: Results for lossless gray image compression.
Method
TIFF
GIF
PNG
JPEG-LS
JPEG2000-LS
TCAE

Convolutional entropy encoder [7]
Binary
Importance
Total
0.181
0.092
0.274
0.249
0.092
0.342
0.468
0.092
0.561
0.710
0.092
0.802
1.199
0.115
1.313

Compression Ratio
1.01
1.13
1.61
1.57
1.79
2.00

5. Conclusion
4.3. CNN-based lossy image compression [7]

The paper presents a trimmed convolutional network for
arithmetic encoding (i.e., TCAE) of 3D binary cuboid. Benefited from trimmed convolution, we can utilize the fully
convolutional network architecture for performing probability prediction to all bits in one single forward pass, making
it possible to model large context while maintaining com-

Entropy encoding can also be used in the CNN-based
lossy image compression system to compress the intermediate codes. In our experiment, we take the system [7]
as an example, and replace the convolutional entropy encoder in [7] with our TCAE. Fig. 6 shows the rate-distortion
8

putational efficiency. In comparison with traditional methods such as PNG and JPEG2000-LS, our TCAE achieves
better compression ratio for lossless gray image compression. Moreover, it can also be incorporated with the CNNbased lossy image compression system (e.g., [7]) to compress the intermediate codes, and exhibits its superiority in
large context modeling and real time encoding speed. However, in [7] the entropy encoder and the encoder-decoder
network are separately trained with different objective functions. In future, we will improve the efficiency of the entropy decoding algorithms, and investigate appropriate network architecture and objectives for joint learning of these
two networks in an end-to-end manner.

[14] G. Toderici, D. Vincent, N. Johnston, S. J. Hwang, D. Minnen, J. Shor, and M. Covell. Full resolution image compression with recurrent neural networks. arXiv preprint
arXiv:1608.05148, 2016.
[15] A. van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals,
A. Graves, et al. Conditional image generation with pixelcnn decoders. In Advances in Neural Information Processing
Systems, pages 4790–4798, 2016.
[16] M. J. Weinberger, G. Seroussi, and G. Sapiro. The loco-i
lossless image compression algorithm: Principles and standardization into jpeg-ls. IEEE Transactions on Image processing, 9(8):1309–1324, 2000.
[17] T. A. Welch. A technique for high-performance data compression. Computer, 6(17):8–19, 1984.
[18] I. H. Witten, R. M. Neal, and J. G. Cleary. Arithmetic coding for data compression. Communications of the ACM,
30(6):520–540, 1987.

References
[1] J. Ballé, V. Laparra, and E. P. Simoncelli.
Endto-end optimized image compression.
arXiv preprint
arXiv:1611.01704, 2016.
[2] T. Boutell. Png (portable network graphics) specification
version 1.0. 1997.
[3] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. FeiFei. Imagenet: A large-scale hierarchical image database.
In Computer Vision and Pattern Recognition, 2009. CVPR
2009. IEEE Conference on, pages 248–255. IEEE, 2009.
[4] R. Gallager and D. Van Voorhis. Optimal source codes for
geometrically distributed integer alphabets (corresp.). IEEE
Transactions on Information theory, 21(2):228–230, 1975.
[5] D. A. Huffman. A method for the construction of minimumredundancy codes. Proceedings of the IRE, 40(9):1098–
1101, 1952.
[6] D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[7] M. Li, W. Zuo, S. Gu, D. Zhao, and D. Zhang. Learning convolutional networks for content-weighted image compression. arXiv preprint arXiv:1703.10553, 2017.
[8] D. Marpe, H. Schwarz, and T. Wiegand. Context-based adaptive binary arithmetic coding in the h. 264/avc video compression standard. IEEE Transactions on circuits and systems for video technology, 13(7):620–636, 2003.
[9] S. Medouakh and Z. Baarir. Entropy encoding ebcot (embedded block coding with optimized truncation in jpeg2000).
IJCSI, 8, 2011.
[10] A. v. d. Oord, N. Kalchbrenner, and K. Kavukcuoglu. Pixel
recurrent neural networks. arXiv preprint arXiv:1601.06759,
2016.
[11] C. E. Shannon. A mathematical theory of communication.
ACM SIGMOBILE Mobile Computing and Communications
Review, 5(1):3–55, 2001.
[12] A. Skodras, C. Christopoulos, and T. Ebrahimi. The jpeg
2000 still image compression standard. IEEE Signal processing magazine, 18(5):36–58, 2001.
[13] G. Toderici, S. M. O’Malley, S. J. Hwang, D. Vincent,
D. Minnen, S. Baluja, M. Covell, and R. Sukthankar. Variable rate image compression with recurrent neural networks.
arXiv preprint arXiv:1511.06085, 2015.

9

