Image Deblurring - Department of Computing,...
-
Upload
trinhkhuong -
Category
Documents
-
view
218 -
download
2
Transcript of Image Deblurring - Department of Computing,...
![Page 1: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/1.jpg)
Seungyong Lee
POSTECH
Image Deblurring
1
![Page 2: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/2.jpg)
Contents
• Fast Motion Deblurring (Siggraph Asia 2009)
• Non-uniform Motion Deblurring for Camera Shakes using Image Registration (Siggraph 2011 Talks)
• Text Deblurring (an ongoing project)
2
![Page 3: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/3.jpg)
Sunghyun Cho Seungyong Lee
POSTECH POSTECH
Fast Motion Deblurring
3
![Page 4: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/4.jpg)
Motion blur
• Camera jitters
Blurred image Latent sharp image
4
![Page 5: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/5.jpg)
Image formation model
• Convolution
– Motion blur kernel • Trace of a sensor
Latent sharp image Blur kernel Blurred image
* : convolution operator
5
![Page 6: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/6.jpg)
Deblurring
• Non-blind deconvolution
– Ill-posed (Due to the loss of information caused by motion blur)
• Blind deconvolution
– Severely ill-posed
Latent image PSF Blurred image
Latent image PSF Blurred image 6
![Page 7: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/7.jpg)
Blind deconvolution
• Severely ill-posed problem
– No unique solution
Blurred image
Possible solutions
7
![Page 8: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/8.jpg)
Related work
• Parametric kernels
– Ex) 1D linear motion blur
– [Yitzhakey et al. 1998], [Rav-Acha and Peleg 2005], [Cho et al. 2007], [Money and Kang 2008], …
Blurred image Latent sharp image PSF
8
![Page 9: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/9.jpg)
Related work
• More complex motion blur
– [Fergus et al. 2006], [Jia 2007], [Shan et al. 2008]
– Excessive amount of computation
Blurred image Latent sharp image PSF
9
![Page 10: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/10.jpg)
Motivation
• Computation time Important for practical purpose
• Previous methods are slow
Image size: 640 x 480 kernel size: 25 x 25
[Fergus et al. 2006] took 1 hr 25 min. [Shan et al. 2008] took 4 min 48 sec.
Our method took 5.766 sec. in CPU and 0.734 sec. using GPU accel.
10
![Page 11: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/11.jpg)
Contributions
• Fast motion deblurring
– Only a few sec.
– Fast latent image estimation
– Fast blur kernel estimation
40x ~ 60x faster than [Shan et al. 2008]
– GPU acceleration
600x ~ 800x faster than [Shan et al. 2008]
11
![Page 12: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/12.jpg)
Motion deblurring: Common framework
• Iteratively solve
1. Estimate a PSF
2. Estimate a latent sharp image using a complex image prior
Latent image PSF Blurred image
Latent image PSF Blurred image
12
![Page 13: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/13.jpg)
Motion deblurring: Common framework
• Blur model
• Energy function
NKLB *
2( , ) * ( ) ( )f L K B L K q L r K
* : convolution operator
q(L), r(K) : regularization terms or priors for L, K
Latent image L Blur kernel K Blurred image B Noise N
13
![Page 14: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/14.jpg)
Motion deblurring: Common framework
Latent image estimation 2
arg min * )' (L
B L KL Lq
Kernel estimation 2
arg min * )' (K
B LK K Kr
Blurred image
Deblurred result
14
![Page 15: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/15.jpg)
Motion deblurring: Common framework
Blurred image Kernel estimation
15
![Page 16: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/16.jpg)
1st latent image estimation Kernel estimation
Motion deblurring: Common framework
16
![Page 17: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/17.jpg)
1st latent image estimation 1st kernel estimation
Motion deblurring: Common framework
17
![Page 18: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/18.jpg)
3rd latent image estimation 1st kernel estimation
Motion deblurring: Common framework
18
![Page 19: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/19.jpg)
3rd latent image estimation 3rd kernel estimation
Motion deblurring: Common framework
19
![Page 20: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/20.jpg)
5th latent image estimation 3rd kernel estimation
Motion deblurring: Common framework
20
![Page 21: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/21.jpg)
5th latent image estimation 5th kernel estimation
Motion deblurring: Common framework
21
![Page 22: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/22.jpg)
7th latent image estimation 5th kernel estimation
Motion deblurring: Common framework
22
![Page 23: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/23.jpg)
7th latent image estimation 7th kernel estimation
Motion deblurring: Common framework
Both estimation steps are slow We analyzed and accelerated both estimation steps
23
![Page 24: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/24.jpg)
Latent image estimation: Analysis of prev. methods
Deblurred result Blurred image
• Two important properties
– Restoration of strong edges • Inspecting around strong edges,
we can find a blur kernel
– Noise suppression in smooth regions • Avoids the effect of noise
on kernel estimation
Blurry input Latent image estimation of
[Shan et al. 2008]
Latent image estimation
Kernel estimation
24
![Page 25: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/25.jpg)
Blurry input Latent image estimation of
[Shan et al. 2008]
Latent image estimation: Analysis of prev. methods
• Two important properties
– Restoration of strong edges • Inspecting around strong edges,
we can find a blur kernel
– Noise suppression in smooth regions • Avoids the effect of noise
on kernel estimation
Deblurred result Blurred image Latent image estimation
Kernel estimation
25
![Page 26: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/26.jpg)
Latent image estimation: Analysis of prev. methods
• Two important properties
– Restoration of strong edges • Inspecting around strong edges,
we can find a blur kernel
– Noise suppression in smooth regions • Avoids the effect of noise
on kernel estimation
Blurry input Latent image estimation of
[Shan et al. 2008]
Deblurred result Blurred image Latent image estimation
Kernel estimation
26
![Page 27: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/27.jpg)
Latent image estimation: Analysis of prev. methods
• Two important properties
– Restoration of strong edges • Inspecting around strong edges,
we can find a blur kernel
– Noise suppression in smooth regions • Avoids the effect of noise
on kernel estimation
• Computationally expensive priors for q(L)
2arg min * )' (
L
B L KL Lq
Blurry input Latent image estimation of
[Shan et al. 2008]
Deblurred result Blurred image Latent image estimation
Kernel estimation
27
![Page 28: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/28.jpg)
Latent image estimation: Basic idea for acceleration
Simple Deconvolution Removes blur quickly
Low-quality results
Prediction Restores strong edges
Removes noise Simple image processing tools
We divide… Latent image estimation
Deblurred result Blurred image Latent image estimation
Kernel estimation
28
![Page 29: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/29.jpg)
Latent image estimation: Basic idea for acceleration
Simple Deconvolution Prediction
Updated kernel Current kernel
Deblurred result Blurred image Latent image estimation
Kernel estimation
29
![Page 30: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/30.jpg)
Kernel estimation: Analysis of prev. methods
• Previous methods estimate a blur kernel K by optimizing:
Latent image estimation
Kernel estimation
• B : a blurred image
• K : a blur kernel
• L : a latent sharp image
• r(K) : a regularization term for K
2arg min * )' (
K
B LK K Kr
Deblurred result Blurred image
30
![Page 31: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/31.jpg)
Kernel estimation: Analysis of prev. methods
• Previous methods estimate a blur kernel K by optimizing:
• The simplest case: r(K) ≡ α|K|2 (α: a scalar value)
• Then, K can be found by solving:
• which can be solved by a conjugate gradient (CG) method
• LTLk is computed for each CG iteration
bLkLkLTT
L: a matrix rep. of L k: a vector rep. of K b: a vector rep. of B
Latent image estimation
Kernel estimation
Deblurred result Blurred image
2arg min * )' (
K
B LK K Kr
31
![Page 32: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/32.jpg)
Kernel estimation: Analysis of prev. methods
• Computing LTLk
– Convolutions & correlations
– Lk L*K
– LTLk L *correl (L*K)
– Conv. & corr. can be accelerated using FFTs
– 4 FFTs per CG iter. for computing LTLk
• A CG method needs to iterate…
• 4 FFTs x 30 CG iters = 120 FFTs…
Latent image estimation
Kernel estimation
Deblurred result Blurred image
32
![Page 33: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/33.jpg)
Kernel estimation: Basic idea for acceleration
• Energy function using derivative images:
• With deriv. images, we can avoid boundary problem of FFTs
• ∂LT∂Lk can be computed using 2 FFTs
• CG iterations converge faster with derivative images
2 2arg min *'
K
B a KLK K
∂: partial differential operator
Latent image estimation
Kernel estimation
Deblurred result Blurred image
33
![Page 34: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/34.jpg)
Deblurred result
Blurred image
Deblurring process
Final deconvolution
Kernel estimation Prediction Deconvolution
* Deconvolution + prediction = latent image estimation
34
![Page 35: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/35.jpg)
Results
• Implementation – CPU version
• C++, OpenCV, FFTW
– GPU accelerated version
• BSGP [Hou et al. 2008] – Easy GPGPU language
• CUDA FFT library
• Testing environment – PC running MS Windows XP 32 bit ver.
– Intel Core2 Quad CPU 2.66 GHz
– 3.25GB RAM
– NVIDIA GeForce GTX 280 Graphics card
35
![Page 36: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/36.jpg)
Results
Image size Blur kernel size Processing time
(CPU) Processing time
(GPU)
1024 x 768 49 x 47 18.656 sec. 2.125 sec.
Blurry input Our result Blur kernel
36
![Page 37: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/37.jpg)
Results
Image size Blur kernel size Processing time
(CPU) Processing time
(GPU)
972 x 966 65 x 93 18.813 sec. 5.766 sec.
Blurry input Our result Blur kernel
37
![Page 38: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/38.jpg)
Results
Image size Blur kernel size Processing time
(CPU) Processing time
(GPU)
858 x 558 61 x 43 8.969 sec. 0.703 sec.
Blurry input Our result Blur kernel
38
![Page 39: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/39.jpg)
Comparison (quality)
Blurry input
[Yuan et al. 2007] * This method uses two input images.
[Shan et al. 2008] our method
39
![Page 40: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/40.jpg)
Comparison (processing time)
• [Shan et al. 2008] vs. Our method
Image
Size Processing time (sec.)
Image PSF Shan et al.
(CPU) Our method
(CPU) Our method
(GPU)
Picasso 800 x 532 27 x 19 360 7.485 0.609
Statue 903 x 910 25 x 25 762 15.891 0.984
Night 836 x 804 27 x 21 762 13.813 0.937
Red tree 454 x 588 27 x 27 309 4.703 0.438
* Processing times of our CPU code are updated from our paper
40
![Page 42: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/42.jpg)
Conclusion and future work
• Deblurring method fast enough for practical use
– Efficient latent image estimation
– Efficient kernel estimation
• Limitation and future work
– Sharp edge assumption
– Noise and saturation
– Non-uniform blur
42
![Page 43: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/43.jpg)
Non-uniform Motion
Deblurring for Camera Shakes
using Image Registration
Sunghyun Cho1 Hojin Cho1 Wu-Ying Tai2 Seungyong
Lee1
1POSTECH 2KAIST
![Page 44: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/44.jpg)
Camera Shakes
44
![Page 45: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/45.jpg)
Previous Methods Fail
• Result of [Shan et al. 2008]
45
![Page 46: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/46.jpg)
Uniform Blur
46
![Page 47: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/47.jpg)
Non-uniform Blur
47
![Page 48: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/48.jpg)
Our Method
[Shan et al. 2008] Our method
48
![Page 49: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/49.jpg)
Related Work
• Uniform motion blur
– Fergus et al., SIGGRAPH 2006
– Jia, CVPR 2007
– Shan et al., SIGGRAPH 2008
– Cho and Lee, SIGGRAPH ASIA
2009
– Etc…
49
![Page 50: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/50.jpg)
Related Work
• Non-uniform motion blur
– Whyte et al. CVPR 2010
• x, y, z rotations
50
![Page 51: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/51.jpg)
Related Work
• Non-uniform motion blur
– Whyte et al. CVPR 2010
• x, y, z rotations
– Gupta et al. ECCV 2010
• x, y translations
+ in-plane rotation
51
![Page 52: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/52.jpg)
Related Work
• Non-uniform motion blur
– Whyte et al. CVPR 2010
• x, y, z rotations
– Gupta et al. ECCV 2010
• x, y translations
+ in-plane rotation
Our method
• x, y, z translations
+ x, y, z rotations 52
![Page 53: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/53.jpg)
• Non-uniform motion deblurring
• Non-uniform blur estimation
Contributions
Blurred image
Blurred image
Deblurred result
Blurred image
Latent image
53
![Page 54: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/54.jpg)
Blind Deblurring
Input blurry
images
Blur Estimation
Latent
Image
Restoration
Deblurred
Result
54
![Page 55: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/55.jpg)
• Widely used in previous works
Uniform Blur Model
Motion blur kernel Blurred image Latent image
Convolution
operator
55
![Page 56: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/56.jpg)
• Widely used in previous works
Uniform Blur Model
translation weight Blurred image Latent image
56
![Page 57: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/57.jpg)
• Tai et al., PAMI, to appear
Non-uniform Blur Model
Homography
Blurred image Latent image
57
![Page 58: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/58.jpg)
• Tai et al., PAMI, to appear
Non-uniform Blur Model
How to estimate these?
58
![Page 59: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/59.jpg)
Blur Estimation
• Homography Estimation
Residual
image
Weighted
latent
image
59
![Page 60: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/60.jpg)
Blur Estimation
•
Residual
image
Weighted
latent
image
60
![Page 61: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/61.jpg)
• Homography Estimation
Blur Estimation
Residual
image
Weighted
latent
image
61
![Page 62: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/62.jpg)
Blur Estimation
• Weight Estimation: simple linear system
62
![Page 63: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/63.jpg)
Blur Estimation
Input Weight estimation
Solve a linear
system
Blurred
image
Latent
image
63
![Page 64: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/64.jpg)
Estimated blur Input
Blur Estimation
64
![Page 65: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/65.jpg)
Results
Blurred input 1 Blurred input 2
65
![Page 66: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/66.jpg)
Results
Estimated blur 1 Estimated blur 2
66
![Page 67: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/67.jpg)
Results
Our method Uniform deblurring
[Cho and Lee 2009]
67
![Page 68: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/68.jpg)
Results
Blurred input 1 Blurred input 2
68
![Page 69: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/69.jpg)
Results
Estimated blur 1 Estimated blur 2
69
![Page 70: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/70.jpg)
Results
Blurred input 1
Blurred input 2
Output
70
![Page 71: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/71.jpg)
Results
71
![Page 72: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/72.jpg)
Results
72
![Page 73: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/73.jpg)
Results
73
![Page 74: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/74.jpg)
Results
74
![Page 75: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/75.jpg)
Results
75
![Page 76: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/76.jpg)
Results
76
![Page 77: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/77.jpg)
Conclusion
• We have developed
– Non-uniform blur estimation method
– Blind deblurring system for non-uniform motion
blur
77
![Page 78: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/78.jpg)
• Still ongoing…
• Analysis and comparison
– Recent non-uniform deblurring methods
• More application
– Video deblurring
• More severe blur
• Zooming blur
Future Work
78
![Page 79: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/79.jpg)
Text Deblurring
79
![Page 80: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/80.jpg)
Original image
Synthetic blurred image
Fast motion deblurring result
Our result
80
![Page 81: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/81.jpg)
Results on Real Photographs (1/6)
Blurred image
Our result
81
![Page 82: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/82.jpg)
Results on Real Photographs (2/6)
Blurred image Our result
82
![Page 83: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/83.jpg)
Results on Real Photographs (3/6)
Blurred image Our result
83
![Page 84: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/84.jpg)
Results on Real Photographs (4/6)
Blurred image
Our result (very large blur: 105x105)
84
![Page 85: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/85.jpg)
Results on Real Photographs (5/6)
Blurred image Our result
85
![Page 86: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/86.jpg)
Results on Real Photographs (6/6)
Blurred image Our result
86
![Page 87: Image Deblurring - Department of Computing, …doc.gold.ac.uk/uk-korea-geomod2011/Slides/05-Seungyong...Image formation model • Convolution –Motion blur kernel •Trace of a sensor](https://reader031.fdocuments.us/reader031/viewer/2022022004/5aa65ba87f8b9ac8748e4cf2/html5/thumbnails/87.jpg)
Thank you!
http://cg.postech.ac.kr
87