DMCNN: Dual-domain Multi-scale Convolutional Neural Network for Compression Artifacts Removal


Abstract

JPEG is one of the most commonly used standards among lossy image compression methods. However, JPEG compression inevitably introduces various kinds of artifacts, especially at high compression rates, which could greatly affect the Quality of Experience (QoE). Recently, convolutional neural network (CNN) based methods have shown excellent performance for removing the JPEG artifacts. Lots of efforts have been made to deepen the CNNs and extract deeper features, while relatively few works pay attention to the receptive field of the network. In this paper, we illustrate that the quality of output images can be significantly improved by enlarging the receptive fields in many cases. One step further, we propose a Dual-domain Multi-scale CNN (DMCNN) to take full advantage of redundancies on both the pixel and DCT domains. Experiments show that DMCNN sets a new state-of-the-art for the task of JPEG artifact removal.

fig1

Fig.1 The banding effects can still be clearly seen after the process of ARCNN(a) [1] and DnCNN(b) [2] due to their small receptive fields.

Network Architecture

The architecture of our proposed DMCNN is in Fig.2. The model is mainly composed of two similar auto-encoder style networks working on pixel and DCT domains, respectively. The input image is processed by DCT branch first, then passed into the pixel domain branch. The final restoration result is the weighted sum of the input, the DCT branch estimation and the pixel branch estimation.

fig2

Fig.2 The architecture of Dual-domain Multi-scale Convolutional Network (DMCNN).

Quantitative Results

QF = 10 QF = 20
Method PSNR (dB) SSIM PSNR-B (dB) PSNR (dB) SSIM PSNR-B (dB)
JPEG
27.77 0.791 25.33 30.07 0.868 27.57
ARCNN [1] 29.13 0.823 28.74 31.40 0.890 30.69
TNRD [3] 29.24 0.825 28.90 31.52 0.892 30.88
DnCNN-3 [2] 29.27 0.825 28.98 31.62 0.894 30.89
CAS-CNN [4] 29.44 0.833 29.19 31.70 0.895 30.88
DMCNN 29.73 0.842 29.55 32.09 0.905 31.32

Tab.1 The quantitative results on LIVE1.


QF = 10 QF = 20
Method PSNR (dB) SSIM PSNR-B (dB) PSNR (dB) SSIM PSNR-B (dB)
JPEG
27.80 0.788 25.10 30.05 0.867 27.22
ARCNN [1] 29.10 0.820 28.73 31.28 0.885 30.55
TNRD [3] 29.16 0.823 28.81 31.41 0.889 30.83
DnCNN-3 [2] 29.17 0.823 28.91 31.50 0.891 30.85
DDCN [5] 29.59 0.838 29.18 31.88 0.900 31.10
DMCNN 29.67 0.840 29.33 31.98 0.904 31.29

Tab.2 The quantitative results on BSDS500 testing set.

Selected Qualitative Results

label

Fig.3 Visual comparisons between different algorithms with QF=10. Zooming-in the figure will provide a better look at the restoration quality.

Resources

  • Paper: pdf
  • Citation

    @proceedings { zhang2018dmcnn, title={DMCNN: Dual-domain Multi-scale Convolutional Neural Network for Compression Artifacts Removal}, author={Zhang, Xiaoshuai and Yang, Wenhan and Hu, Yueyu and Liu, Jiaying}, journal={IEEE International Conference on Image Processing}, year={2018} }

    References

    [1] Chao Dong, Yubin Deng, Chen Change Loy, and Xiaoou Tang, “Compression artifacts reduction by a deep convolutional network,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 576–584.
    [2] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang, “Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017.
    [3] Yunjin Chen and Thomas Pock, “Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1256–1272, 2017.
    [4] Lukas Cavigelli, Pascal Hager, and Luca Benini, “CAS-CNN: A deep convolutional neural network for image compression artifact suppression,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN). IEEE, 2017, pp. 752–759.
    [5] Jun Guo and Hongyang Chao, “Building dual-domain representations for compression artifacts reduction,” in Proceedings of the European Conference on Computer Vision. Springer, 2016, pp. 628–644.