1 ECE 472/572 - Digital Image Processing Lecture 1 - Introduction 08/18/11.
ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE...
Transcript of ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE...
![Page 1: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/1.jpg)
1
ECE 472/572 - Digital Image Processing
Lecture 5 - Image Enhancement - Frequency Domain Filters 09/13/11
2
Roadmap
¬ Introduction – Image format (vector vs. bitmap) – IP vs. CV vs. CG – HLIP vs. LLIP – Image acquisition
¬ Perception – Structure of human eye
• rods vs. conss (Scotopic vision vs. photopic vision)
• Fovea and blind spot • Flexible lens (near-sighted vs. far-
sighted) – Brightness adaptation and
Discrimination • Weber ratio • Dynamic range
– Image resolution • Sampling vs. quantization
¬ Image enhancement – Enhancement vs. restoration – Spatial domain methods
• Point-based methods – Log trans. vs. Power-law
• Gamma correction • Dynamic range compression
– Contrast stretching vs. HE • What is HE? • Derivation of tran. func.
– Gray-level vs. Bit plane slicing – Image averaging (principle)
• Mask-based (neighborhood-based) methods - spatial filter
– Smoothing vs. Sharpening filter – Linear vs. Non-linear filter – Smoothing
• Average vs. weighted average • Average vs. Median
– Sharpening • UM vs. High boosting • 1st vs. 2nd derivatives
– Frequency domain methods
3
Questions
¬ In-depth understanding – Why do we need to conduct image processing in the frequency
domain? – What does Fourier series do? – What does the Fourier spectrum of an image tell you? – How to calculate the fundamental frequency? – Why is padding necessary?
¬ Properties – Is FT a linear or nonlinear process? – What would the FT of a rotated image look like? – When implementing FFT, what kind of properties are used? – What does the autocorrelation of an image tell you? – What is F(0,0)? Or Why is the center of the FT extremely bright?
![Page 2: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/2.jpg)
2
4
Why FT? – 1
5
Why FT? – 2
( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ){ }vuFvuHFyxg
vuFvuHvuGyxfyxhyxg
,,,,,,,,,
1−=
=⇔∗=
6
Another example
![Page 3: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/3.jpg)
3
7
Fourier series
¬ Fourier series (SF) can represent any function over a finite interval TF
¬ Outside TF, SF repeats periodically with period TF.
€
complex form: s t( ) = cnej2nπfF t
n=−∞
∞
∑
cn =1TF
s t( )e− j 2nπfF tdt−TF / 2
TF / 2∫
8
Fourier series (cont’)
¬ TF is the interval of signal s(t) over which the Fourier series represents
¬ fF = 1/TF is the fundamental frequency of the Fourier series representation
¬ n is called the “harmonic number” – E.g., 2fF is the second harmonic of the fundamental frequency fF.
¬ The Fourier series representation is always periodic and is linear combinations of sinusoids at fF and its harmonics.
9
Fourier transform
¬ Describe the frequency distribution
nfF
cn
u
v
(0,0)
![Page 4: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/4.jpg)
4
10
11
1-D Fourier transform
¬ Fourier transform:
¬ Inverse FT:
¬ Complex form
¬ Fourier spectrum ¬ Power spectrum
(spectral density) ¬ Phase angle
€
F u( ) = f t( )−∞
∞
∫ exp − j2πut( )dt
f t( ) = F u( )−∞
∞
∫ exp j2πut( )du
F u( ) = R u( ) + jI u( )F u( ) = F u( )e jφ u( )
F u( ) = R2 u( ) + I2 u( )
P u( ) = F u( )2
φ u( ) = tan−1I u( )R u( )
'
( )
*
+ ,
€
F u( ) =1N
f x( )x= 0
N−1
∑ exp − j2πux /N( )
f x( ) = F u( )u= 0
N−1
∑ exp j2πux /N( )
DFT
12
2-D Fourier transform
¬ CFT
¬ DFT
( ) ( ) ( )[ ]
( ) ( ) ( )[ ]∫ ∫
∫ ∫∞
∞−
∞
∞−
∞
∞−
∞
∞−
+=
+−=
dudvvyuxjvuFyxf
dxdyvyuxjyxfvuF
π
π
2exp,,
2exp,,
( ) ( ) ( )[ ]
( ) ( ) ( )[ ]∑∑
∑∑−
=
−
=
−
=
−
=
+=
+−=
1
0
1
0
1
0
1
0
//2exp,,
//2exp,1
,
M
u
N
v
M
x
N
y
NvyMuxjvuFyxf
NvyMuxjyxfMN
vuF
π
π
![Page 5: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/5.jpg)
5
13
Understanding and implementing Fourier transform
( ) ( ) ( )[ ]
( ) ( ) ( )[ ]∑∑
∑∑−
=
−
=
−
=
−
=
+=
+−=
1
0
1
0
1
0
1
0
//2exp,,
//2exp,1
,
M
u
N
v
M
x
N
y
NvyMuxjvuFyxf
NvyMuxjyxfMN
vuF
π
π
(0,0)
f(x,y)
x
y (0,0)
|F(u,v)| u
v yN
vxM
uΔ
=ΔΔ
=Δ1,1
14
0)*)1,1(*)0,1(
*)1,0(*)0,0((2*2
1)1,1(
0)*)1,1(*)0,1(
*)1,0(*)0,0((2*2
1)0,1(
5.127)*)1,1(*)0,1(
*)1,0(*)0,0((2*2
1)1,0(
5.127))1,1()0,1()1,0()0,0((2*2
1)0,0(
)2/1*12/1*1(2)2/0*12/1*1(2
)2/1*12/0*1(2)2/0*12/0*1(2
)2/1*02/1*1(2)2/0*02/1*1(2
)2/1*02/0*1(2)2/0*02/0*1(2
)2/1*12/1*0(2)2/0*12/1*0(2
)2/1*12/0*0(2)2/0*12/0*0(2
=++
+=
=++
+=
−=++
+=
=+++=
+−+−
+−+−
+−+−
+−+−
+−+−
+−+−
ππ
ππ
ππ
ππ
ππ
ππ
jj
jj
jj
jj
jj
jj
efef
efefF
efef
efefF
efef
efefF
ffffF
15
Understanding and implementing Fourier transform
¬ According to “translation”
( )( ) ( )2/,2/1,
period, complete oneFor
NvMuFyxf yx −−⇔− +
(0,0)
f(x,y) x
y
-255
255
(0,0)
f(x,y)(-1)x+y
x
y
-0
0
(0,0)
|F(u-M/2,v-N/2)| u
v
![Page 6: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/6.jpg)
6
16
Understanding and implementing Fourier transform
17
Block diagram of FTIFT
f(x,y)(-1)x+y g(x,y)(-1)x+y
18
Impulse transforms
![Page 7: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/7.jpg)
7
19
Typical transforms
¬ Gaussian hump ßà Gaussian hump
¬ Rectangular (square aperture) ßàsinc
¬ Pillbox (circular aperture) ßàjinc
20
Typical transforms
¬ Gaussian ridge
¬ Line impulse
( )2exp xπ−
( )0when
/exp 221
→
−−
τ
τπτ x
21
Reference
¬ All figures scanned from R. N. Bracewell’s “Two-Dimensional Imaging,” Prentice Hall, 1995.
![Page 8: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/8.jpg)
8
22
2D FT pairs
( )
( )( )
( )( )( )( )[ ]22
00
exp
,,,,,
,
yx
yxcombyxtriyxrectyyxx
yx
yxf
+−
±±
π
δ
δ
( )
( ) ( )( )( )( )( )[ ]22
2
00
exp
,,sin
,sin2exp2exp
1
,
vu
vucombvuc
vucvyjuxj
vuF
+−
±±
π
ππ
23
Important properties of FT
¬ Linearity (distributivity && scaling) ¬ Separability ¬ Translation ¬ Periodicity ¬ Conjugate symmetry ¬ Rotation ¬ Convolution ¬ Correlation ¬ Sampling
24
Linearity
¬ FT is a linear image processing method
Linear System
x1(t) y1(t) x2(t) y2(t)
a*x1(t) + b*x2(t) a*y1(t) + b*y2(t)
![Page 9: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/9.jpg)
9
25
Separability
( ) ( ) ( )[ ]∑∑−
=
−
=
+−=1
0
1
0
//2exp,1
,M
x
N
y
NvyMuxjyxfMN
vuF π
f(x, y) F(x, v) Row
transform F(u, v) Column
transform
26
Translation
( ) ( )[ ] ( )( ) ( ) ( )[ ]NvyuxjvuFyyxxf
vvuuFNyvxujyxf/2exp,,
,/2exp,
0000
0000
+−⇔−−
−−⇔+
π
π
€
To show one complete period,
f x,y( ) −1( )x+y⇔ F u − N /2,v − N /2( )
27
Periodicity and Conjugate symmetry
¬ Periodicity
¬ Conjugate symmetry ( ) ( ) ( ) ( )NvNuFNvuFvNuFvuF ++=+=+= ,,,,
( ) ( )( ) ( )vuFvuF
vuFvuF−−=
−−= ∗
,,,,
![Page 10: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/10.jpg)
10
28
Example
29
Rotation
( ) ( )00 ,, θφωθθ +=+ Frf
30
Averaging
( ) ( )0,0, Fyxf =
( ) ( ) ( )[ ]
( ) ( ) ( )[ ]∑∑
∑∑−
=
−
=
−
=
−
=
+=
+−=
1
0
1
0
1
0
1
0
//2exp,,
//2exp,1
,
M
u
N
v
M
x
N
y
NvyMuxjvuFyxf
NvyMuxjyxfMN
vuF
π
π
![Page 11: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/11.jpg)
11
31
Convolution
¬ Continuous and discrete convolution
¬ The convolution theorem
¬ Practically, computing the discrete convolution in the frequency domain often is more efficient than doing it in the spatial domain directly
( ) ( ) ( ) ( )( ) ( ) ( ) ( )vuGvuFyxgyxf
vuGvuFyxgyxf,,,,,,,,
∗⇔
⇔∗
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )∑∑
∫ ∫−
=
−
=
∞
∞−
−−=∗
−−=∗
1
0
1
0,,1,,
,,,,
M
m
N
neeee nymxgnmf
MNyxgyxf
ddyxgfyxgyxf βαβαβα
32
Correlation
¬ Continuous and discrete correlation
¬ The correlation theorem
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )∑∑
∫ ∫−
=
−
=
∞
∞−
++=
++=
1
0
1
0
*
,,1,,
,,,,
M
m
N
neeee nymxgnmf
MNyxgyxf
ddyxgfyxgyxf
βαβαβα
( ) ( ) ( ) ( )( ) ( ) ( ) ( )vuGvuFyxgyxf
vuGvuFyxgyxf
,,,,,,,,
*
*
⇔
⇔
33
Correlation (cont’)
¬ Autocorrelation vs. cross correlation ¬ Autocorrelation theorem
¬ Application: template or prototype matching
( ) ( ){ } ( )2,,, vuFyxfyxfF =
![Page 12: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/12.jpg)
12
34
Practical issues – Implement convolution in frequency domain
¬ In spatial domain
¬ In frequency domain – f*g çè F(f)G(g) – Phase? Mag? – How to pad?
35
36
![Page 13: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/13.jpg)
13
37
38
39
Difference image from convolution in the spatial domain
Convolution in the frequency domain
Conv. spatially
No padding With padding
![Page 14: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/14.jpg)
14
40
Different enhancement approaches
¬ Lowpass filter ¬ Highpass filter ¬ Homomorphic filter
41
Lowpass filtering
¬ Ideal filter – D(u, v): distance from point (u, v) to the origin – cutoff frequency (D0) – nonphysical – radially symmetric about the origin
¬ Butterworth filter
¬ Gaussian lowpass filter
( )( )( )!
"#
>
≤=
0
0
, if 0, if 1
,DvuDDvuD
vuH
( )( )[ ] nDvuD
vuH 20/,1
1,+
=
( ) ( ) 20
2 2/,, DvuDevuH −=
42
![Page 15: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/15.jpg)
15
43
Power ratio 99.96à99.65à99.04à97.84
( ) ( )!"
#$%
&= ∑∑
u vT vuPvuP ,/,100β
44
Highpass filter
¬ Ideal filter
¬ Butterworth filter
¬ Gaussian highpass filter
( )( )( )!
"#
>
≤=
0
0
, if 1, if 0
,DvuDDvuD
vuH
( )( )[ ] nvuDD
vuH 20 ,/11,
+=
( ) ( ) 20
2 2/,1, DvuDevuH −−=
45
![Page 16: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/16.jpg)
16
46
47
Example
48
The Laplacian in the frequency domain
€
∇2 f x,y( ) =∂2 f∂x 2
+∂2 f∂y 2
€
H(u,v) = −4π 2 u2 + v 2( )
Pay attention to the scaling factor
![Page 17: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/17.jpg)
17
49
UM in the frequency domain
€
g(x,y) = f (x,y) + k ∗ gmask (x,y) = f (x,y) + k ∗ ( f (x,y) − fLP (x,y))g(x,y) =ℑ−1{[1+ k ∗HHP (u,v)]F(u,v)}
50
Homomorphic filtering
¬ A simple image model – f(x,y): the intensity is called the gray level for
monochrome image – f(x, y) = i(x, y).r(x, y) – 0 < i(x, y) < inf, the illumination – 0< r(x, y) < 1, the reflectance
51
Homomorphic filter (cont’) ( ) ( ) ( )( ) ( ) ( ) ( )( ){ } ( ){ } ( ){ }
( ) ( ) ( ) ( ) ( )
( ) ( ) ( )
( ) ( )[ ] ( )[ ] ( )[ ]yxryxiyxsyxg
yxryxiyxs
vuFvuHvuFvuHvuS
vuFvuFvuZyxrFyxiFyxzFyxryxiyxfyxz
yxryxiyxf
ri
ri
,exp,exp,exp,
,,,
,,,,,
),(),(),(,ln,ln,,ln,ln,ln,
,,,
!!==
!+!=
+=
+=
+=
+==
⋅=
![Page 18: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/18.jpg)
18
52
Homomorphic filter (cont’)
¬ The illumination component – Slow spatial variations – Low frequency
¬ The reflectance component – Vary abruptly, particularly at the junctions of
dissimilar objects – High frequency
¬ Homomorphic filters – Affect low and high frequencies differently – Compress the low frequency dynamic range – Enhance the contrast in high frequency
53
Homomorphic Filter (cont’)
11
<
>
L
H
γ
γL
DvuDcLH evuH γγγ +−−= − ]1)[(),( )/),(( 2
02
54
![Page 19: ECE 472/572 - Digital Image Processingweb.eecs.utk.edu/~hqi/...enhancement_frequency.pdf · ECE 472/572 - Digital Image Processing Lecture 5 - Image Enhancement - Frequency Domain](https://reader030.fdocuments.us/reader030/viewer/2022041102/5edd05abad6a402d6667f3f2/html5/thumbnails/19.jpg)
19
55
Homomorphic filter - example
56
¬ Point processing
¬ Simple gray level transformations – Image negatives – Log transformations – Power-law
transformations – Contrast stretching – Gray-level slicing – Bit-plane slicing
¬ Histogram processing – Histogram
equalization – *Histogram matching
(specification) ¬ Arithmetic/logic
operations – Image averaging
¬ Mask processing (spatial filters)
¬ Smoothing filters (blur details) – Average, weighted
average – Order statistics (e.g.
median)
¬ Sharpening filters (highlight details) – Unsharp masking – High-boost filters – Derivative filters
• The Laplacian • The Gradient
• Frequency domain filters
• Smoothing filters (blur details)
• Ideal lowpass filter • Butterworth lowpass • Gaussian lowpass
• Sharpening filters (highlight details)
– Unsharp masking – High-boost filters – Derivative filters - The
Laplacian – Ideal highpass filter – Butterworth highpass filter – Gaussian highpass filter
• Homomorphic filtering
57
FFT and IFFT