ELE 488 Fall 2006 Image Processing and Transmission - …liu/488 f06/488 F06 p… · PPT file ·...
Transcript of ELE 488 Fall 2006 Image Processing and Transmission - …liu/488 f06/488 F06 p… · PPT file ·...
ELE 488 F06
ELE 488 Fall 2006Image Processing and Transmission
09-28-06
Edge Map
Laplacian
Median Filter
Filtering Images in Frequency Domain
Image Restoration
9/28/06
ELE 488 F06
Gradient, 1st-order Derivatives, and Edges
• Edge: where luminance changes abruptly• For binary image
– Take black pixels with immediate white neighbors as edge pixel
• For continuous-tone image– Find luminance gradient
• partial derivatives along two orthogonal directions• gives the direction with highest rate of changes• If gradient is larger than a threshold => edge
– To represent edge• Edge map ~ edge intensity & directions
UM
CP
EN
EE
408G
Slid
es (c
reat
ed b
y M
.Wu
& R
.Liu
© 2
002)
ELE 488 F06
The Gradient of Function f(x,y)
The gradient of a smooth function f(x,y) at (x,y) is
derivative of f in the x-direction, partial derivative of f w.r.t. x
derivative of f in the y-direction, partial derivative of f w.r.t. y
2-vector pointing in the direction of greatest increase of f fr
.rate of change of f in the direction h f
ELE 488 F06
The Discrete Gradient
The gradient of a real-valued discrete function f(m,n) at (j.k) is
The symmetric rate of change of f in the m-direction at (j,k)
The symmetric rate of change of f in the n-direction at (j,k)
(Note: for simplicity, drop the scale factor of ½ - it can always be reinserted again if needed.)
ELE 488 F06
Averaging GVF
• gradient is computed using high pass filters (discrete differentiation) will amplify noise.
• So average neighboring gradient vectors using a spatial filtering on each of two gradient components.
• The averaging filters and gradient filters are LSI and hence can be applied in any order, i.e.,
• Can compute gradient and average, or• Average pixels and then compute gradient.
Should we average uniformly in all directions or along a selected direction? e.g. along the perpendicular to the direction of the derivative? Perpendicular to the gradient?
ELE 488 F06
Averaging the GVF
Spatial averaging filter perpendicular to direction of discrete derivative
discrete derivative filter in horizontal direction
Combination of the two masks gives single “averaged gradient mask” in horizontal direction.
ELE 488 F06
Common GVF Masks
– Apply both masks to image and combine results to determine magnitude and angle (if desired).
– Note: Prewitt and Sobel operators spatially average the horizontal and vertical differences of 3 local pixels to reduce the effect of noise
0 1
-1 0
-1 0
-1 0
1
1
-1 0 1
-1 0
-2 0
1
2
-1 0 1
1 0
0 -1
-1 -1
0 0
-1
0
1 1 1
-1 -2
0 0
-1
0
1 2 1
Roberts Prewitt Sobel
ELE 488 F06
Averaged GVF and Edges
Scaled & averaged gradient of apple image plotted at every 3rd pixel.
Large gradient magnitude
Edge:Normal to direction of gradient
Small gradient magnitude
No edge
Subjective Issue: How to select the threshold on the magnitude of the gradient to “detect” an edge?
ELE 488 F06
Example: Edge Detection from GVF
Threshold magnitude of the GVF:Above threshold: edge pixelBelow threshold: no edge
ELE 488 F06
Sobel GVF
Sobel
Prewitt
ELE 488 F06
Examples of Edge Detectors
– Quantize edge intensity to 0/1:• set a threshold• white pixel denotes
strong edge
Roberts Prewitt Sobel
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
ELE 488 F06
Edges and Second Derivatives in 1-D
“edges” are at points of local maximum and minimum slope of f(x). Here at x=1 and x=3.
“edges” are at local maxima and minima of df/dx. Here at x=1 and x=3. Threshold the magnitude of the derivative…gives local region of edge.
“edges” are at points where the second derivative passes through zero. Here at x=1 and x=3.
From 1D to 2D
ELE 488 F06
2D: Derivatives of f(x,y)
ELE 488 F06
Example: Gaussian
ELE 488 F06
Derivatives
Red lines indicate zero level sets
ELE 488 F06
Difference Operators (along rows, columns similar)
Many choices:
More generally:
ELE 488 F06
Averaging Across Rows
ELE 488 F06
Common Difference MasksRoberts Prewitt Sobel
Prewitt and Sobel masks are separable. They spatially average across 3 rows (or columns).
ELE 488 F06
ExampleComparison of Sobel operation with actual derivative of Gaussian.Note scale difference of approx. 5.3The error (after adjusting for scale & the edge effect) is shown below.
ELE 488 F06
Second Differences
Examples:
In this case, change the 00 pixel of the second dm mask to make the result symmetric
ELE 488 F06
Example: Second Difference Sobel
Do the same for the Sobel column difference operator.
ELE 488 F06
Example: 2nd Diff SobelComparison of Sobel operation with actual second derivative of Gaussian.Note scale difference of approx. 28.24The error (after adjusting for scale and edge effect) is shown below.
ELE 488 F06
A 3x3 Discrete Laplacians
First difference. This operator is rotationally symmetric for 90 deg rotations.
Roberts. This operator is rotationally symmetric for 90 deg rotations.
Composite. This operator is rotationally symmetric for 90 deg rotations.
ELE 488 F06
The 5x5 Sobel Laplacian
Sobel. This operator is also rotationally symmetric for 90 deg rotations. As expected, sum of all entries is 0.
ELE 488 F06
Zero Contour of 3x3 Roberts Laplacian
ELE 488 F06
Zero Contour of Sobel Laplacian
ELE 488 F06
Zero Contour of a 17x17 Laplacian
ELE 488 F06
Spatial Masks
Spatial Mask Finite, 2-D, real valued, array.
One element is designated as (0,0).
Used to compute the weighted sum of an input image, where weights are specified by the mask.
Usually has small support 2x2, 3x3, 5x5, 7x7
Examples:
These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc.
1/4 1/4
1/4 1/4
0 1
0
1
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
ELE 488 F06
Use Averaging to Suppressing Noise
• Image with additive noise x(m,n) + N(m,n)• Averaging over window of Nw pixels:
v(m,n) = (1/Nw) x(m-k, n-j) + (1/Nw) N(m-k, n-j)
• Noise variance reduced by a factor of Nw SNR improved by a factor of Nw if x(m,n) is constant in local window
• Limit window size to avoid blurring
UM
CP
EN
EE
408G
Slid
es (c
reat
ed b
y M
.Wu
& R
.Liu
© 2
002)
1/4 1/4
1/4 1/4
0 1
0
1
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
ELE 488 F06
Coping with Salt-and-Pepper NoiseMatlab Image Toolbox Guide Fig.10-12, 10-13
UM
CP
EN
EE
408G
Slid
es (c
reat
ed b
y M
.Wu
& R
.Liu
© 2
002)
ELE 488 F06
Median Filtering
• Median of pixel values in a window of size Nw
– Median is middle value of all pixel values if Nw is odd, and is the average of two middle values if Nw is even.
– Need to order data
• “Salt-and-Pepper” noise– Isolated white or black pixels spread randomly over the image– Spatial averaging filter can cause blur
• Median filtering– Take median value over a small window as output
Note: Median { x(m) + y(m) } Median {x(m)} + Median {y(m)} nonlinear
– Commonly used window sizes • 3x3, 5x5, 7x7• 5x5 “+”–shaped window
UM
CP
EN
EE
408G
Slid
es (c
reat
ed b
y M
.Wu
& R
.Liu
© 2
002)
ELE 488 F06
Other Ways of Image Sharpening
• Combine sharpening with histogram equalization
UM
CP
EN
EE
408G
Slid
es (c
reat
ed b
y M
.Wu
© 2
002)
ELE 488 F06
Image Processing
• Pointwise processing– Each output pixel depends only on one input pixel– Histogram modification
• Spatial filtering– Each output pixel may depend on more than one pixels– Masking, linear filtering, edge detection,
• Linear filtering can be carried out in frequency domain• Extending 1D processing in frequency domain to 2D
ELE 488 F06
Signal Processing in Time and Frequency Domain
output signal y[n]
input signal x[n]
Processor
LTI system
time domain: y[n] = Σ h[k] x[n – k] , h[n] impulse response
Fourier transform: X(ω) = Σx[n] e–jnω. = FT{x[n]}
Inverse Fourier transform: x[k] = (1/2π)∫ X(ω) ejnω dω = IFT {X(ω)}
Frequency domain: Y(ω) = X(ω) H(ω),
where Y(ω) = FT{ y[n] } , H(ω) = FT{ h[n] }
ELE 488 F06
Discrete Fourier Transform
FT: X(ω) = Σx[n] e–jnω. IFT: x[k] = (1/2π)∫ X(ω) ejnω dω .
DFT: X[k] = Σ0≤n≤M–1 x[n] e–jn2π/M.
IDFT: x[n] = (1/M)Σ0≤k≤M–1 X[k] e jk2π/M.
Fast Fourier Transform (FFT)
ELE 488 F06
2D Signal Processing in Time and Frequency Domain
output signal y[m,n]
input signal x[m,n]
Processor
LTI system
time domain: y[m,n] = Σ h[k,l] x[m – k ,n – l] , h[m,n] impulse response
Fourier transform: X(ω,λ) = Σx[m,n] e–j(mω+nλ) = FT{x[m,n]}
Inverse Fourier transform: x[k,l] = (1/2π)∫ X(ω) ej(mω+nλ) dωdλ = IFT {X(ω)}
Frequency domain: Y(ω ,λ) = X(ω ,λ) H(ω ,λ),
where Y(ω ,λ) = FT{ y[m,n] } , H(ω ,λ) = FT{ h[m,n] }
ELE 488 F06
2D Discrete Fourier Transform
FT: X(ω ,λ) = Σx[n] e–j(nω +nλ) .
IFT: x[k,l] = (1/2π)2∫ ∫ X(ω ,λ) e –j(nω +nλ) dω dλ .
DFT: X[k,l] = Σ0≤m≤M–1 Σ0≤n≤N–1 x[m,n] e–j(m2π/M+n2π/N) .
IDFT: x[m,n] = (1/MN) Σ0≤k≤M–1Σ0≤n≤N–1 X[k,l] ej(k2π/M+l2π/M) .
Fast Fourier Transform (FFT)
Separable
ELE 488 F06
Review of Random Signals
A 1D random signal u[n] is stationary if its average behavior does not depend on n. It can be characterized by probability density function fu( . ) and power spectral density Φu ( . ).
Prob [ a ≤u[n] < b ] = b
a
fu(x)dx
E{ u[n] } = < u[n] > = average value (expectation) of u[n] =
x fu(x) dx (first moment)
E{ u[n]2 } = < u[n]2 > = mean square value or power of u[n] =
x2 fu(x) dx
The expectation E{ u[n] u[n+k] }, called the autocorrelation function.
Depends on k, not on n, because u is stationary. It is often denoted by ruu[k].
ELE 488 F06
Random Signal (cont)
Power of u[n] in the frequency band (ω1 , ω2 ) is (1/2π)
2
1||
Φu(ω) dω
Φu(ω) is called the power spectral density of signal u. It is the Fourier transform of ruu[k].
Total power = (1/2π)
Φu(ω) dω = E{ u[n]2 }
White noise: Φu(ω) = constant , E{ u[n]2 } = Φu(ω) Given a linear system with frequency response H(ω), the power spectral density
of the output v[n] is related to that of the input u[n] via: Φv (ω) = Φu (ω) |H(ω) |2
The output power is therefore E{ v[n]2 } = (1/2π)
Φu (ω) |H(ω) |2 dω
If input {u[n]} is white, E{ v[n]2 } = E{ u[n]2 } (1/2π)
|H(ω) |2 dω
ELE 488 F06
2D Random Signal
A 2D random signal u[m, n] is stationary if its average behavior does not depend on m and n. It can be characterized by probability density function fu( . ) and power spectral density Φu ( . ).
Prob [ a ≤u[m, n] < b ] = b
a
fu(x)dx
E{ u[m, n] } = average value (expectation) of u[m, n] =
x fu(x) dx
E{ u[m, n]2 } = mean square value or power of u[m, n] =
x2 fu(x) dx
The expectation E{ u[m, n] u[m+j, n+k] }, called the autocorrelation function.
depends only on k, not on n, because u is stationary. It is often denoted by ruu[j, k].
ELE 488 F06
2D Random Signal (cont)Power of u[m, n] in the frequency band ω1 < |ω| < ω2 and λ1 < |λ| < λ2 is
(1/2π)2
2
1||
2
1||
Φu(ω, λ) dω dλ
Φu(ω,λ) is the power spectral density of signal u. It is the Fourier transform of ruu[j, k].
Total power = (1/2π)
Φu(ω, λ) dω dλ = E{ u[m, n]2 }
White noise: Φu(ω, λ) = constant , E{ u[m, n]2 } = Φu(ω, λ) Given a linear system with frequency response H(ω, λ), the power spectral density of the output
v[m, n] is related to that of the input u[m, n] via: Φv (ω, λ) = Φu (ω, λ) |H(ω, λ) |2
Total output power is therefore E{ v[m,n]2 } = (1/2π)
Φu (ω, λ) |H(ω, λ) |2 dω
If input {u[n]} is white, E{ v[m,n]2 } = E{ u[m,n]2 } (1/2π)
|H(ω, λ) |2 dω
ELE 488 F06From Matlab ImageToolbox Documentation pp12-4
UM
CP
EN
EE
631
Slid
es (c
reat
ed b
y M
.Wu
© 2
001)
Image Restoration
ELE 488 F06
Imperfection in Image Capturing
• Blurring ~ linear spatial-invariant filter model w/ additive noise
• Impulse response h(n1, n2) & H(1, 2)– Point Spread Function (PSF) ~ positive I/O
– If h(n1, n2) = (n1, n2), no blur
– Linear translational motion blur caused by local average along motion direction
– Uniform out-of-focus blur caused by local average in a circular neighborhood
– Atomspheric turbulence blur, etc.
Hu(n1, n2 ) v(n1, n2 )
N(n1, n2 )
otherwise 0
tan and
x if L1
),;,( yx
222
Ly
Lyxh
otherwise 0
x if R1
);,(222
2 RyRyxh
UM
CP
EN
EE
631
Slid
es (c
reat
ed b
y M
.Wu
© 2
001)
ELE 488 F06
Fourier Transform of PSF for Common Distortions
From Bovik’s Handbook Sec.3.5 Fig.2&3
UMCP ENEE631 Slides (created by M.Wu © 2001)