Efficient, Interpretable Deep Blind Image Deblurring …gb l j d yl 1 gb i j=1 gbl j 2 +" 1 C C C A...
Transcript of Efficient, Interpretable Deep Blind Image Deblurring …gb l j d yl 1 gb i j=1 gbl j 2 +" 1 C C C A...
1
Efficient, Interpretable Deep Blind ImageDeblurring Via Unrolling
Supplementary Document
Yuelong Li, Student Member, IEEE, Mohammad Tofighi, Student Member, IEEE,Junyi Geng, Student Member, IEEE, Vishal Monga, Senior Member, IEEE, and Yonina C. Eldar, Fellow, IEEE
Let L be the cost function defined in (12). We derive its gradients w.r.t. its variables using the chain rule as follows:
∇wliL = ∇wl
iyli∇yl
iL = Rwl
iFdiag
(yl+1i
)F∗∇yl
iL,
∇ζliL = ∇ζlizl+1i ∇zl+1
iL
=
kl∗�(kl � gli − yli
)(∣∣∣kl∣∣∣2 + ζli
)2
T
F∗(I{|Pgg
l+1i |>bli} � ∇zl+1
iL),
∇bliL = ∇blizl+1i ∇zl+1
iL =
(I{gl+1
i <−bli} − I{gl+1i >bli}
)T∇zl+1
iL,
where Rwli
is the operator that extracts the components lying in the support of wli. Again using the chain rule,
∂L∂kl
=∂L∂zl+1
i
∂zl+1i
∂kl,
∂L∂zli
=∂Lzl+1i
∂zl+1i
∂zli+∂L∂kl
∂kl
∂zli,
∂L∂yli
=∂Lzl+1i
∂zl+1i
∂yli+
∂L∂kl+1
∂kl+1
∂yli+
∂L∂yl−1i
∂yl−1i
∂yli. (1)
We next derive each individual term in (1) as follows:
∂zl+1i
∂gl+1i
=∂zl+1
i
gl+1i
∂gl+1i
∂gl+1i
= diag(I{|Pgg
l+1i |>bli}
)F∗,
∂zl+1i
∂zli=∂zl+1
i
∂gl+1i
∂gl+1i
∂zli
∂zli∂zli
= diag(I{|Pgg
l+1i |>bli}
)F∗diag
ζli∣∣∣kl∣∣∣2 + ζli
F, (2)
∂zl+1i
∂yli=∂zl+1
i
∂gl+1i
∂gl+1i
∂yli
∂yli∂yli
= diag(I{|Pgg
l+1i |>bli}
)F∗diag
kl∗∣∣∣kl∣∣∣2 + ζli
F, (3)
and
∂zl+1i
∂kl=∂zl+1
i
∂gl+1i
(∂gl+1
i
∂kl
∂kli∂kli
+∂gl+1
i
∂kl∗∂kli∗
∂kli
)(4)
= diag(I{|Pgg
l+1i |>bli}
)F∗
diag ζli y
li(∣∣∣kl∣∣∣2 + ζli
)2
F∗ − diag
(kl∗)2� yli(∣∣∣kl∣∣∣2 + ζli
)2
F
,
∂kl+1
∂kl+13
=∂kl+1
∂kl+23
∂kl+23
∂kl+13
∂kl+13
∂kl+13
=I(1Tkl+
23
)− kl+
231T(
1Tkl+23
)2 diag
(I{
Pkkl+1
3>0})F∗,
2
∂kl+1
∂yli=∂kl+1
∂kl+13
kl+13
yli
yliyli
=I(1Tkl+
23
)− kl+
231T(
1Tkl+23
)2 · diag(I{
Pkkl+1
3>0})F∗diag
∑Ci=1 z
l+1i
∗
∑Ci=1
∣∣zl+1i
∣∣2 + ε
F, (5)
∂kl+1
∂zl+1i
=∂kl+1
∂kl+13
∂kl+ 13
∂zl+1i
∂zl+1i
∂zl+1i
+∂kl+
13
∂∂zl+1i
∗∂zl+1
i
∗
∂zl+1i
=I(1Tkl+
23
)− kl+
231T(
1Tkl+23
)2 diag
(I{
Pkkl+1
3>0})F∗· (6)
−diag(∑C
j=1 zl+1j
∗� ylj
)� zl+1
i
∗
(∑Cj=1
∣∣∣zl+1j
∣∣∣2 + ε
)2
F + diag
yli �
(∑Cj=1
∣∣∣zl+1j
∣∣∣2 + ε
)−(∑C
j=1 zl+1j
∗� ylj
)� zl+1
i(∑Cj=1
∣∣∣zl+1j
∣∣∣2 + ε
)2
F∗
,
∂yl−1i
∂yli=∂yl−1i
∂yl−1i
∂yl−1i
∂yli
∂yli∂yli
= F∗diag
(wl−1i
)F. (7)
Plugging (2) (3) (4) (5) (6) (7) into (1), we obtain
∇klL =
F∗diag ζli y
li(∣∣∣kl∣∣∣2 + ζli
)2
− Fdiag
(kl∗)2� yli(∣∣∣kl∣∣∣2 + ζli
)2
F∗
(I{|Pgg
l+1i |>bli} � ∇zl+1
iL)
∇gliL = Fdiag
ζli∣∣∣kl∣∣∣2 + ζli
F∗(I{|Pgg
l+1i |>bli} � ∇zl+1
iL)+
−Fdiag(∑L
j=1 glj
∗� yl−1j
)� gli
∗
(∑Lj=1
∣∣∣glj∣∣∣2 + ε
)2
+ F∗diag
yl−1i �
(∑Lj=1
∣∣∣glj∣∣∣2 + ε
)−(∑L
j=1 glj
∗� yl−1j
)� gli(∑L
j=1
∣∣∣glj∣∣∣2 + ε
)2
F∗
1
1Tkl−13
I{Pkk
l− 23>0
} �∇klC −I{
Pkkl− 2
3>0}kl− 1
3T
(1Tkl−
13
)2 ∇klL
∇yliL = Fdiag
kl∗∣∣∣kl∣∣∣2 + ζli
F∗(I{|Pgg
l+1i |>bli} � ∇zl+1
iL)+ Fdiag
∑Li=1 z
l+1i
∗
∑Li=1
∣∣zl+1i
∣∣2 + ε
F∗
1
1Tkl+23
I{Pkk
l+13>0
} �∇kl+1C −I{
Pkkl+1
3>0}kl+ 2
3T
(1Tkl+
23
)2 ∇kl+1C
+ Fdiag
(wl−1i
)F∗∇yl−1
iL
3
I. ADDITIONAL EXPERIMENTAL RESULTS
(a) Groundtruth (b) Perrone et al. [1] (c) Nah et al. [2] (d) Chakrabarti [3] (e) Xu et al. [4] (f) Kupyn et al. [5] (g) DUBLID
Fig. 1. Qualitative comparisons on the dataset from [6]. The blur kernels are placed at the right bottom corner.
REFERENCES
[1] D. Perrone and P. Favaro, “A Clearer Picture of Total Variation Blind Deconvolution,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 6, pp.1041–1055, Jun. 2016.
[2] S. Nah, T. H. Kim, and K. M. Lee, “Deep multi-scale convolutional neural network for dynamic scene deblurring,” in Proc. IEEE Conf. CVPR, vol. 1,2017, p. 3.
[3] A. Chakrabarti, “A Neural Approach to Blind Motion Deblurring,” in Proc. ECCV, Oct. 2016.[4] X. Xu, J. Pan, Y. J. Zhang, and M. H. Yang, “Motion Blur Kernel Estimation via Deep Learning,” IEEE Trans. Image Process., vol. 27, no. 1, pp.
194–205, Jan. 2018.[5] O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas, “Deblurgan: Blind motion deblurring using conditional adversarial networks,” in Proc.
IEEE Conf. CVPR, Jun. 2018.[6] L. Sun, S. Cho, J. Wang, and J. Hays, “Edge-based blur kernel estimation using patch priors,” in Proc. IEEE ICCP, Apr. 2013.