CSCNet - The Effect of the Stride

1. High CoherenceNatural images mainly consist of smooth and piecewise smooth regions.Hence, the dictionary must contain smooth or piecewise smooth filters.

Having smooth or piecewise smooth filters in the dictionary leads to highly correlated atoms in the global dictionary that results in high coherence.

Rethinking the Convolutional Sparse Coding (CSC) Model for Natural ImagesDror Simon and Michael Elad

Department of Computer Science, Technion – Israel Institute of Technology

{dror.simon@cs, elad@cs}.technion.ac.il

The CSC Model➢ A signal is a sum of m convolutions

of band limited filters with m sparse feature-maps

➢ By interlacing all the feature maps into , and defining a global dictionarythe CSC problem is defined by

where the dictionary is convolutional:

Main Contribution of this Work➢ Explain why the CSC model fails when used

for simple image restoration tasks➢ Show that local patch-based algorithms are

actually MMSE approximations for the global CSC model

➢ Propose novel MMSE approximation for the CSC model that provide superior results compared to patch based algorithms

➢ CSCNet - a discriminative recurrent model trained in a supervised fashion that is on par with current state of the art methods while using much fewer parameters

The CSC - Successful Applications➢ Cartoon-texture separation

➢ Image fusion

➢ Single Image Super Resolution

The CSC - Natural Image Denoising➢ Classic Patch-Averaging (PA) methods

denoise images very well➢ At first it seems that the CSC is a global

extension to patch-averaging methods➢ Unfortunately, CSC has led to poor natural

image denoising resultsWhy does it denoise so poorly?

1. Natural images lead to dictionary with poor properties, i.e. high coherence

2. A Bayesian explanation

2(a). Patch AveragingPatch averaging (PA) using a local dictionary solves N independent problems:

The final image is then obtained using patch-averaging

2(b). CSC MMSEThe MMSE estimator of the CSC model is

The MMSE estimator is not sparse! It is a sum of the oracle estimators of all possible supports

2(c). Connecting the CSC to PA➢ Assume results with non-overlapping

tangent patches➢ There are n such arrangements

➢ Each such shift can be described using a subsampled dictionary and respective

➢ The solution of each such shift is then:

Or equivalently:

CSC - Additional Definitions

PA is CSC MMSE

Generalizing the MMSE Approx.➢ Use strided convolutions with stride q.➢ In the non- overlapping case the stride size

equals the band of the filters.➢ Decrease the stride to allow overlaps

Contradiction!

q<nEach estimate introduces overlaps

q sufficiently largeLow mutual coherence can be preserved

CSCNet - Denoising Model➢ Use Lista [1] and its convolutional extension

[2] to obtain a sparse coding approximation➢ Introduce our strided concept to obtain an

MMSE approximation

CSCNet - Results➢ Denoising results on the BSD68 dataset

➢ Better than BM3D [3], WNNM [4], TNRD [5]

➢ On par with DnCNN [6], FFDNet [7]

BM3D WNNM TNRD DnCNN

FFDNet CSCNet

15 21.07 31.37 31.42 31.72 31.63 31.5725 28.57 28.83 28.92 29.22 29.19 29.1150 25.62 25.87 25.97 26.23 26.29 26.2475 24.21 24.40 — 24.64 24.79 24.77

References[1] Gregor, K., & LeCun, Y. Learning fast approximations of sparse coding. (ICML 2010).[2] Sreter, H., & Giryes, R. Learned convolutional sparse coding. (ICASSP 2018).[3] Dabov, K. et al. Image denoising with block-matching and 3D filtering. (2006).[4] Gu, S. et al. Weighted nuclear norm minimization with application to image denoising. (CVPR 2014)[5] Chen, Y., & Pock, T. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. (IEEE PAMI 2016).[6] Zhang, K. et al. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. (IEEE TIP 2017)[7] Zhang, K., Zuo, W., & Zhang, L. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. (IEEE TIP 2018).