Deep learning based image compression has recently witnessed exciting progress
and in some cases even managed to surpass transform coding based approaches
that have been established and refined over many decades.
However, state-of-the-art solutions for deep image compression typically
employ autoencoders which map the input to a lower dimensional latent space
and thus irreversibly discard information already before quantization.
Due to that, they inherently limit the range of quality levels that can
be covered. In contrast, traditional approaches in image compression allow
for a larger range of quality levels. Interestingly, they employ an invertible
transformation before performing the quantization step which explicitly discards
information. Inspired by this, we propose a deep image compression method
that is able to go from low bit-rates to near lossless quality
by leveraging normalizing flows to learn a bijective mapping from
the image space to a latent representation. In addition to this, we demonstrate
further advantages unique to our solution, such as the ability to maintain
constant quality results through re-encoding, even when performed multiple times.
To the best of our knowledge, this is the first work to explore the opportunities
for leveraging normalizing flows for lossy image compression.

§ INTRODUCTION

Using normalizing flows,
image quality and bit-rate are not affected by multiple
compression/decompression operations. With auto-encoder based
solutions [Ballé et al., 2018], we notice color shifts
and strong artifacts while the bit-rate increases.

§ BACKGROUND ON NORMALIZING FLOW

§ LOSSY IMAGE COMPRESSION WITH NORMALIZING FLOWS

§ RELATED WORK

§ EXPERIMENTS

§ CONCLUSION

In this paper, we explored an approach for lossy image compression based on normalizing flows. To the best of our knowledge, we are the first to evaluate the potential of normalizing flows in the task of lossy compression. Furthermore, we have showcased a number beneficial properties inherent to our solution:

1) We are able to cover the widest range of quality levels with a single trained model ranging from low bitrates to almost lossless quality thereby bridging an important gap that had remained between currently used autoencoder based methods and traditional transform codecs.
2) We show that we can reencode images multiple times in a quasi lossless way closely retaining rate distortion performance. This has been problematic with autoencoder based methods.
3) We describe different ways of achieving scalable coding or progressive reconstruction with our model.

These properties make normalizing flows a very exciting candidate for
lossy image compression models even though we do not yet outperform
existing autoencoder based approaches over the full range of bit rates.
We believe the results obtained in this work are encouraging
as there is still a lot of potential for improvement
and would push new research works to explore new bijective layers,
architecture designs and refined training procedures.

[Agustsson et al., 2019]
Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., and Gool, L. V.
Generative adversarial networks for extreme learned image
In The IEEE International Conference on Computer Vision
(ICCV), October 2019.

[Asuni & Giachetti, 2013]
Asuni, N. and Giachetti, A.
TESTIMAGES: A large data archive for display and algorithm
J. Graphics Tools, 170 (4):0 113–125, 2013.
URL <https://doi.org/10.1080/2165347X.2015.1024298>.

[Ballé et al., 2017]
Ballé, J., Laparra, V., and Simoncelli, E. P.
End-to-end optimized image compression.
ICLR, 2017.

[Ballé et al., 2018]
Ballé, J., Minnen, D., Singh, S., Hwang, S. J., and Johnston, N.
Variational image compression with a scale hyperprior.
ICLR, 2018.

[Bengio et al., 2013]
Bengio, Y., Léonard, N., and Courville, A. C.
Estimating or propagating gradients through stochastic neurons for
conditional computation.
CoRR, abs/1308.3432, 2013.
URL <http://arxiv.org/abs/1308.3432>.

[Choi et al., 2019]
Choi, Y., El-Khamy, M., and Lee, J.
Variable rate deep image compression with a conditional autoencoder.
In 2019 IEEE/CVF International Conference on Computer Vision,
ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pp.  3146–3154. IEEE, 2019.
URL <https://doi.org/10.1109/ICCV.2019.00324>.

[Chrabaszcz et al., 2017]
Chrabaszcz, P., Loshchilov, I., and Hutter, F.
A downsampled variant of imagenet as an alternative to the CIFAR
CoRR, abs/1707.08819, 2017.
URL <http://arxiv.org/abs/1707.08819>.

[Dinh et al., 2017]
Dinh, L., Sohl-Dickstein, J., and Bengio, S.
Density estimation using real NVP.
In 5th International Conference on Learning Representations,
ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track
Proceedings, 2017.
URL <https://openreview.net/forum?id=HkpbnH9lx>.

[Djelouah et al., 2019]
Djelouah, A., Campos, J., Schaub-Meyer, S., and Schroers, C.
Neural inter-frame compression for video coding.
In Proceedings of the IEEE International Conference on Computer
Vision, pp.  6421–6429, 2019.

[Fab, 2015]
BPG Specification.
Fabrice Bellard, 2015.
version 0.9.5.

[Goo, 2010]
Google, 2010.

[Hoogeboom et al., 2019]
Hoogeboom, E., Peters, J. W. T., van den Berg, R., and Welling, M.
Integer discrete flows and lossless compression.
CoRR, abs/1905.07376, 2019.
URL <http://arxiv.org/abs/1905.07376>.

[Kingma & Ba, 2015]
Kingma, D. P. and Ba, J.
Adam: A method for stochastic optimization.
In Bengio, Y. and LeCun, Y. (eds.), 3rd International
Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May
7-9, 2015, Conference Track Proceedings, 2015.
URL <http://arxiv.org/abs/1412.6980>.

[Kingma & Dhariwal, 2018]
Kingma, D. P. and Dhariwal, P.
Glow: Generative flow with invertible 1x1 convolutions.
In Advances in Neural Information Processing Systems 31: Annual
Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8
December 2018, Montréal, Canada., pp.  10236–10245, 2018.

[Lin et al., 2014]
Lin, T., Maire, M., Belongie, S. J., Bourdev, L. D., Girshick, R. B., Hays, J.,
Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L.
Microsoft COCO: common objects in context.
CoRR, abs/1405.0312, 2014.
URL <http://arxiv.org/abs/1405.0312>.

[Lu et al., 2019]
Lu, G., Ouyang, W., Xu, D., Zhang, X., Cai, C., and Gao, Z.
Dvc: An end-to-end deep video compression framework.
In CVPR, 2019.

[Mentzer et al., 2018]
Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., and Van Gool, L.
Conditional probability models for deep image compression.
In CVPR, 2018.

[Mentzer et al., 2019]
Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., and Gool, L. V.
Practical full resolution learned lossless image compression.
In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pp.  10629–10638, 2019.

[Minnen et al., 2018]
Minnen, D., Ballé, J., and Toderici, G. D.
Joint autoregressive and hierarchical priors for learned image
In NeurIPS. 2018.

[Rezende & Mohamed, 2015]
Rezende, D. J. and Mohamed, S.
Variational inference with normalizing flows.
In Proceedings of the 32nd International Conference on Machine
Learning, ICML 2015, Lille, France, 6-11 July 2015, pp.  1530–1538,
URL <http://proceedings.mlr.press/v37/rezende15.html>.

[Rippel & Bourdev, 2017]
Rippel, O. and Bourdev, L.
Real-time adaptive image compression.
In Proceedings of the 34th International Conference on Machine
Learning-Volume 70, pp.  2922–2930. JMLR. org, 2017a.

[Rippel & Bourdev, 2017]
Rippel, O. and Bourdev, L.
Real-time adaptive image compression.
In ICML, 2017b.

[Rippel et al., 2018]
Rippel, O., Nair, S., Lew, C., Branson, S., Anderson, A. G., and Bourdev, L.
Learned video compression.
arXiv, 2018.

[Schwarz et al., 2007]
Schwarz, H., Marpe, D., and Wiegand, T.
Overview of the scalable video coding extension of the H.264/AVC
IEEE Trans. Circuits Syst. Video Techn., 170
(9):0 1103–1120, 2007.
URL <https://doi.org/10.1109/TCSVT.2007.905532>.

[Taubman & Marcellin, 2002]
Taubman, D. S. and Marcellin, M. W.
Jpeg2000: Standard for interactive imaging.
Proceedings of the IEEE, 900 (8), 2002.

[Theis et al., 2017]
Theis, L., Shi, W., Cunningham, A., and Huszár, F.
Lossy image compression with compressive autoencoders.
arXiv preprint arXiv:1703.00395, 2017.

[Toderici et al., 2016]
Toderici, G., O'Malley, S. M., Hwang, S. J., Vincent, D., Minnen, D., Baluja,
S., Covell, M., and Sukthankar, R.
Variable rate image compression with recurrent neural networks.
ICLR, 2016.

[Toderici et al., 2017]
Toderici, G., Vincent, D., Johnston, N., Hwang, S. J., Minnen, D., Shor, J.,
and Covell, M.
Full resolution image compression with recurrent neural networks.
In CVPR, 2017.

[Wallace, 1991]
Wallace, G. K.
The JPEG still picture compression standard.
Commun. ACM, 340 (4):0 30–44, 1991.
URL <https://doi.org/10.1145/103085.103089>.