Some Blind Deconvolution Techniques in Image Processing
Tony Chan
Math Dept., UCLA
Astronomical Data Analysis Software & SystemsConference Series 2004
Pasadena, CA, October 24-27, 2004
Joint work with Frederick Park and
Andy M. Yip
2
OutlinePart I:
Total Variation Blind Deconvolution
Part II:Simultaneous TV Image Inpainting and
Blind Deconvolution
Part III:Automatic Parameter Selection for TV
Blind Deconvolution
Caution: Our work not developed specifically for Astronomical images
3
Blind Deconvolution Problem
= +
Observed image
Unknown true image
Unknown point spread function
Unknown noise
Goal: Given uobs, recover both uorig and k
obsu origu k
4
Typical PSFsPSFs w/ sharp edges:
PSFs w/ smooth transitions
5
Total Variation Regularization
dxxuuTV )()(
• Deconvolution ill-posed: need regularization
• Total variation Regularization:
dxxkkTV )()(
• The characteristic function of D with height h (jump):
• TV = Length(∂D)h• TV doesn’t penalize jumps• Co-area Formula:
D
h drdsfdxufnR
ru
)(||}{
6
TV Blind Deconvolution Model
TVTVobsku
kuukukuF 21
2
,),(min
),(),(,1),(,0, yxkyxkdxdyyxkku Subject to:
Objective:
(C. and Wong (IEEE TIP, 1998))
1 determined by signal-to-noise ratio
2 parameterizes a family of solutions, corresponds to different spread of the reconstructed PSF
• Alternating Minimization Algorithm:
• Globally convergent with H1 regularization.
),(min),(
),(min ),(
111
1
kuFkuF
kuFkuF
n
k
nn
n
u
nn
7
Blind v.s. non-Blind Deconvolution
Observed Image noise-free
• An out-of-focus blur is recovered automatically
• Recovered blind deconvolution images almost as good as non-blind
• Edges well-recovered in image and PSF
non-Blind
Recovered Image PSF
Blind
1 = 2106, 2 = 1.5105
Clean image
True PSF
8
Blind v.s. non-Blind Deconvolution: High Noise
Observed Image SNR=5 dB
non-Blind
Clean image
True PSF
Blind
• An out-of-focus blur is recovered automatically
• Even in the presence of high noise level, recovered images from blind deconvolution are almost as good as those recovered with the exact PSF
1 = 2105, 2 = 1.5105
9
Controlling Focal-Length Recovered Images are 1-parameter family w.r.t. 2
Recovered Blurring Functions(1 = 2106)
0 1107 1105 11042:
The parameter 2 controls the focal-length
10
Generalizations to Multi-Channel Images
• Inter-Channel Blur Model– Color image (Katsaggelos et al, SPIE 1994):
Bobs
Gobs
Robs
B
G
R
kkk
kkk
kkk
u
u
u
u
u
u
HHH
HHH
HHH
noise
122
212
221
75
71
71
71
75
71
71
71
75
obsk unoiseuH
k1: within channel blur
k2: between channel blur
m-channel TV-norm (Color-TV)
(C. & Blomgren, IEEE TIP ‘98)
2
2
1
2)(
m
iTVim
kkTV
m
iTVim uuTV
1
2)(
11
Original image
Out-of-focus blurred blind non-blind
Gaussian blurred blind non-blind
Examples of Multi-Channel Blind Deconvolution(C. and Wong (SPIE, 1997))
• Blind is as good as non-blind
• Gaussian blur is harder to recover (zero-crossings in frequency domain)
12
TV Blind Deconvolution Patented!
13
Outline
Part I:Total Variation Blind Deconvolution
Part II:Simultaneous TV Image Inpainting and
Blind Deconvolution
Part III:Automatic Parameter Selection for TV
Blind Deconvolution
14
TV Inpainting Model(C. & Shen SIAP 2001)
EDE
dxdyuudxdyuuJ ,||2
||][ 20
,0)(||
0
uuu
ue
.0; ,,DzEz
e
Graffiti Removal
Scratch Removal
15
Images Degraded by Blurring and Missing Regions
• Blur– Calibration errors of devices– Atmospheric turbulence– Motion of objects/camera
• Missing regions– Scratches– Occlusion– Defects in films/sensors
+
16
Problems with Inpaint then Deblur
• Inpaint first reduce plausible solutions
• Should pick the solution using more information
Original Signal Blurring func.
Original Signal Blurring func.
Blurred Signal
Blurred Signal
=
=
Blurred + Occluded
Blurred + Occluded
=
17
Problems with Deblur then Inpaint
• Different BC’s correspond to different image intensities in inpaint region.
• Most local BC’s do not respect global geometric structures
Original Occluded Support of PSF
Dirichlet Neumann Inpainting
18
The Joint Model
• Do --- the region where the image is observed
• Di --- the region to be inpainted
• A natural combination of TV deblur + TV inpaint
• No BC’s needed for inpaint regions
• 2 parameters (can incorporate automatic parameter selection techniques)
Inpainting take place
Coupling of inpainting & deblur
19
Simulation Results (1)Degraded Restored
Zoom-in
• The vertical strip is completed
• Not completed
• Use higher order inpainting methods
• E.g. Euler’s elastica, curvature driven diffusion
20
Simulation Results (2)
Observed Restored
Deblur then inpaint
(many artifacts)
Inpaint then deblur
(many ringings)
Original
21
Boundary Conditions for Regular Deblurring
Original image domain and
artificial boundary outside the scene
Dirichlet B.C.
Periodic B.C.
Neumann B.C.
Inpainting B.C.
22
23
Outline
Part I:Total Variation Blind Deconvolution
Part II:Simultaneous TV Image Inpainting and
Blind Deconvolution
Part III:Automatic Parameter Selection for TV
Blind Deconvolution (Ongoing Research)
24
Automatic Blind Deblurring (ongoing research)
• Recovered images: 1-parameter family wrt 2
• Consider external info like sharpness to choose optimal 2
Problem: Find 2 automatically to recover best u & k
SNR = 15 dB20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Clean image
observed image
25
Motivation for Sharpness & Support
• Sharpest image has large gradients• Preference for gradients with small support
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
u
|| u
|| u
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Support of
26
Proposed Sharpness Evaluator
• F(u) small => sharp image with small support• F(u)=0 for piecewise constant images• F(u) penalizes smeared edges
||ofsupport of Area)( uuF
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
u
|| u20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Support of
27
Planets Example
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Rel. errors in u (blue) and k (red) v.s. 2
Proposed Objective v.s. 2
Optimal Restored Image Auto-focused Image
The minimum of the sharpness function agrees with that of the
rel. errors of u and k
(minimizer of sharpness func.)
(minimizer of rel. error in u)
1=0.02 (optimal)
105
106
107
0
0.5
1
1.5
105
106
107
1500
2000
2500
3000
3500
4000
4500
5000
28
Satellite ExampleRel. errors in u (blue) and
k (red) v.s. 2
Proposed Objective v.s. 2
Optimal Restored Image Auto-focused Image
The minimum of the sharpness function agrees with that of the
rel. errors of u and k
(minimizer of sharpness func.)
(minimizer of rel. error in u)
1=0.3 (optimal)
105
106
107
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
20 40 60 80 100 120
20
40
60
80
100
120
105
106
107
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
20 40 60 80 100 120
20
40
60
80
100
120
29
Potential Applications to Astronomical Imaging
• TV Blind Deconvolution– TV/Sharp edges useful?– Auto-focus: appropriate objective function?– How to incorporate a priori domain knowledge?
• TV Blind Deconvolution + Inpainting– Other noise models: e.g. salt-and-pepper noise
30
References
1. C. and C. K. Wong, Total Variation Blind Deconvolution, IEEE Transactions on Image Processing, 7(3):370-375, 1998.
2. C. and C. K. Wong, Multichannel Image Deconvolution by Total Variation Regularization, Proc. to the SPIE Symposium on Advanced Signal Processing: Algorithms, Architectures, and Implementations, vol. 3162, San Diego, CA, July 1997, Ed.: F. Luk.
3. C. and C. K. Wong, Convergence of the Alternating Minimization Algorithm for Blind Deconvolution, UCLA Mathematics Department CAM Report 99-19.
4. R. H. Chan, C. and C. K. Wong, Cosine Transform Based Preconditioners for Total Variation Deblurring, IEEE Trans. Image Proc., 8 (1999), pp. 1472-1478
5. C., A. Yip and F. Park, Simultaneous Total Variation Image Inpainting and Blind Deconvolution, UCLA Mathematics Department CAM Report 04-45.
Top Related