ELEG404/604: Digital Imaging &...
Transcript of ELEG404/604: Digital Imaging &...
ELEG404/604: Digital Imaging &Photography
Gonzalo R. ArceDepartment of Electrical and Computer Engineering
University of Delaware
Chapter II
1/90
ELEG404/604
Image Enhancement in the Spatial DomainI Algorithms for improving the visual appearance of imagesI Gamma correctionI Contrast improvementsI Histogram equalizationI Noise reductionI Image sharpening
I Optimality is in the eye of the observer: Ad hoc.
2/90
ELEG404/604
What is a histogramEach individual histogram entry is defined as
h(i) = Number of pixels in l with the intensity value ifor all 0≤ i < K, that is
h(i) = card{(u,v) |I (u,v) = i}h(0) is the number of pixels with the value 0, h(1) the number of pixels withthe value 1, and so forth.
3/90
ELEG404/604
HistogramsI Used to conclude if an image is properly exposed.I Digital cameras often provide a real-time histogram overlay on the
viewfinder.I Used to improve the visual appearance of an image.I Used as a ’‘forensic‘’ tool for determining what type of processing has
been applied to an image.
4/90
ELEG404/604
Histograms
Three very different images with identical histograms.
5/90
ELEG404/604
Image Acquisition- ExposureI Histograms make classic exposure problems apparent.I Large span of the intensity range at one end is largely unused other end is
crowded with high-value peaks represents poorly exposed image.
(a) Underexposed, (b) properly exposed, (c) overexposed
6/90
ELEG404/604
Image Acquisition- ContrastI Range of intensity values effectively used.I Difference between the image’s maximum and minimum pixel values.I A full−contrast image makes effective use of the entire range of available
intensity values from a= amin · · ·amax = 0 · · ·K−1 (black to white).
(a) Low contrast, (b) Normal contrast, (c) High contrast
7/90
ELEG404/604
Image Acquisition- Dynamic RangeI Number of distinct pixel values in an image.I Ideal case: Image uses all K usable pixel values.
(a) High dynamic range, (b) low dynamic range with 64 intensity values, (c)extremely low dynamic range, 6 intensity values
8/90
ELEG404/604
CompressionEffect of JPEG compression. The original image (a) with only two differentgray values, (b) histogram. JPEG compression, results in additional grayvalues histogram (d).
9/90
ELEG404/604
Color Image HistogramsHistograms of an RGB color image: (a) original image, (b) luminancehistogram hLum,(c−e) RGB color components, and associated componenthistograms hR,hG,hB (f−h). All three color channels have saturationproblems. The spike in the distribution resulting from this is found in themiddle of the luminance histogram (b).
10/90
ELEG404/604
Spatial Domain Processing
I Utilize neighborhood operations
g(x,y) = T [f(x,y)]
I Simple case: point operations
s= T (r)
I Contrast stretchingI Thresholding
11/90
ELEG404/604
Gray Level Transformation Curves
I More general transformationsI LogI Inverse LogI nth powerI nth root
I Used to map narrow dark (log/root) orbright (inverse log/power) range to agreater dynamic range
12/90
ELEG404/604
Fourier Spectrum ExampleThe spectrum has a large dynamic rangeI Example: 0 to 106.I Compress range to view visually (dB’s)
Left: Fourier spectrum. Right: Result of applying the log transformation.
13/90
ELEG404/604
Power-Law (Gamma) Transformations
I Power-law transformation:
s= crγ
I Many devices require gamma correctionI CRTs have power function intensity-to-voltage
responsesI Monitor specificI Applied in color planes
14/90
ELEG404/604
Gamma Correction Example Monitor
Top Left: linear-wedge gray-scale image. Top right: response of monitor tolinear wedge. Bottom left: Gamma-corrected wedge. Bottom right: Output ofmonitor
15/90
ELEG404/604
Gamma Correction Example - Aerial
γ=3
γ=8γ=4
Top Left: aerial image. Top right, Bottom left and bottom right: results of applying Gamma correction with c = 1 and γ = 3,4,5 respectively.
16/90
ELEG404/604
Histogram Processing
I Light, dark, and low contrast images haveconcentrated histograms
I Images with uniform histogramsI Contain the full range of gray valuesI Have high contrastI Better general visual appearance
17/90
ELEG404/604
Histogram EqualizationI We focus on the (normalized) scalar mapping
s= T (r) 0≤ r ≤ 1
where the following are satisfied:I T (r) is single-valued and monotonically increasing in [0,1]I 0≤ T (r)≤ 1 for 0≤ r ≤ 1
I The single-valued condition allows the inverse transformation to be defined
r = T−1(s) 0≤ s≤ 1
18/90
ELEG404/604
Probability Density Function
I Let the PDF of r be the pr(r)I The CDF is Pr(r) = ∫ r
0 pr(w)dwI Note CDFs are monotonically increasing and have range [0,1]
I Defined the RV s= T (r)I The PDF of a RV function is
ps(s) = pr(r)∣∣∣∣drds
∣∣∣∣
19/90
ELEG404/604
CDF Distribution
I Set T (r) = Pr(r)
ds
dr= dT (r)
drd[∫ r0 pr(w)dw]
dr= pr(r)
I Thus the PDF of s is
ps(s) = pr(r)∣∣∣∣drds
∣∣∣∣pr(r)
∣∣∣∣ 1pr(r)
∣∣∣∣ = 1
I The CDF is uniformly distributed.
20/90
ELEG404/604
Histogram Equalization
21/90
ELEG404/604
Histogram EqualizationA 3-bit image (L= 8) of size 64×64(MN = 4096) has the intensitydistribution in table.
Intensity levels are integers in [0,L−1] = [0,7].
22/90
ELEG404/604
Histogram Equalization
Values of the histogram equalization are obtained using:
s0 = T (r0) = 70∑j=0
Pr(rj) = 7pr(r0) = 1.33
Similarly,
s1 = T (r1) = 71∑j=0
Pr(rj) = 7pr(r0) + 7pr(r1) = 3.08
and s2 = 4.55, s3 = 5.67, s4 = 6.23, s5 = 6.65, s6 = 6.86, s7 = 7.00
23/90
ELEG404/604
The s values still have fractions because they were generated by summingprobability values, so we round them to the nearest integer:
s0 = 1.33→ 1s1 = 3.08→ 3s2 = 4.55→ 5s3 = 5.67→ 6
s4 = 6.23→ 6s5 = 6.65→ 7s6 = 6.86→ 7s7 = 7.00→ 7
24/90
ELEG404/604
Histogram Equalization
I CDF mapping of gray valuesI Yields uniformed histogramI Simple, parameter-free
I Discrete caseI Results not strictly uniformI Implementation issues
25/90
ELEG404/604
Histogram EQ Mappings
26/90
ELEG404/604
Histogram Equalization Result
Image
Histograms
Cumulative
Histograms
Original image and corresponding histograms. Notice that new peaks arecreated in the resulting histogram h (d) by merging original histogram cells.
27/90
ELEG404/604
Filters
What is a filter?Main difference between filters and point operations is that filters generallyuse more than one pixel to compute each new pixel input.
No point operation can blur or sharpen an image.
28/90
ELEG404/604
Filters
Each new pixel value I ′(x,y) is computed as a function of the pixels in acorresponding region in the original image I.
29/90
ELEG404/604
Spatial Filtering
I Spatial filtering is based on amoving window or mask
I In the linear filtering case:
g(x,y) = Σas=−aΣb
t=−bw(s, t)f(x+s,y+t)
I More compactly
R = Σmi=1nwizi
Must take into account bordereffects
30/90
ELEG404/604
FilteringTo sharpen or filter an image that was taken out-of-focus and is blurred, eachpixel is replaced with a linear combination of its neighbors.
31/90
ELEG404/604
Filtering
32/90
ELEG404/604
Simple Smoothing Masks
I Simplest linear filter: spatial average: reduces noise, but introducesblurring
I Distance weight samplesI Centrally located samples are more importantI Reduces blurring (somewhat)I Example above: simple integer arithmetic
I Alternative approach: utilize spectral characteristics
33/90
ELEG404/604
34/90
ELEG404/604
Border geometry
Filter can be applied only at locations (x,y) where the filter matrix H of size(2K+ 1)× (2L+ 1) is fully contained in the image.
35/90
ELEG404/604
Linear FiltersExamples of linear filters. The “box” filter (a) and the Gauss filter (b) areboth smoothing filters with all-positive coefficients. The “Laplace” or“Mexican hat” filter (c) is a difference filter.
36/90
ELEG404/604
Spatial Filtering
37/90
ELEG404/604
Smoothing Example
I Square averaging filter resultsI Window sizes: 3, 5, 9, 15, and 35
38/90
ELEG404/604
NonLinear Filters
Any image structure is blurred by a linear filter. Image structures such as edges(top) or thin lines (bottom) are widened, and the local contrast is reduced.
39/90
ELEG404/604
Order-Statistic Filters
I Linear (weighted sum) filtersI Blur edges and detailsI Are susceptible to outliers
I Order-statistic filters (nonlinear)I Preserve edges and detailsI Are less susceptible to outliers
I Spatially ordered samples: z1, z2, · · · , zNI Rank ordered samples: z(1), z(2), · · · , z(N)
z(1) ≤ z(2) ≤ ·· · ≤ z(N)MED[z1, z2, · · · , zN ] = z((N+1)/2)
Selection-type filter
40/90
ELEG404/604
Filtering
Sometimes it is very useful to apply a non-linear filter such as the median.
41/90
ELEG404/604
Effects of a minimum filter on various local signal structuresOriginal signal (top) and result after filtering (bottom), where color barsindicate the extent of the filter. Step edge (a) and linear ramp (c) are shiftedto the right. Narrow pulse (b) is removed.
42/90
ELEG404/604
Minimum Filter
43/90
ELEG404/604
Maximum Filter
44/90
ELEG404/604
Minimum and Maximum filters applied to a grayscale image
The original image is corrupted with ’‘salt and pepper‘’ noise (a). The 3×3pixel minimum filter eliminates the bright dots and widens all dark imagestructures (b). The maximum filter shows the exact opposite effects (c).
45/90
ELEG404/604
Median Filter
Computation of a 3×3 median filter. The nine pixel values extracted from thewindow are sorted and the center value is the median
46/90
ELEG404/604
Effects of Median FilterEffects of a 3×3 median filter on two-dimensional structures. Isolated dotsare eliminated (a), as are thin lines (b). Step edge remains unchanged (c),corner is rounded off (d).
47/90
ELEG404/604
Median Filter
48/90
ELEG404/604
Linear smoothing filter vs median filter
(a) Original image corrupted with ’‘salt-and-pepper‘’ noise. (b) Linear 3×3pixel box filter (b) reduces noise but the entire image is blurred. (c) The
median filter eliminates the noise dots and keeps the structures largely intact.
49/90
ELEG404/604
Salt and Pepper Noise Example
Left: X-ray Image of circuit board corrupted by salt and pepper noise. Middle: Noisereduction with a 3x3 averaging mask. Right: Noise reduction with a 3x3 median filter.I Bad sensors and bit errors yield salt-and-pepper noiseI Heavy tailed noise distribution. Other examples: Laplacian, Cauchy, a-StableI Outliers (generally) located in the extremes of the ordered set. Do not affect median
50/90
ELEG404/604
Weighted Median Filters (WMF)
The median filter can be generalized to weighted order statistic filters.
Given x(n) = [x1, . . . ,xN ], and a set of weights w1, . . . ,wN , the WMF outputis given by
y(n) = MEDIAN [x1♦w1,x2♦w2, . . . ,xN♦wN ] (1)where xi♦wi denotes replication:
xi♦wi =wi times︷ ︸︸ ︷
xi,xi, . . . ,xi (2)and wi ∈ Z+
51/90
ELEG404/604
Weighted median example
Each pixel value is replicated multiple times, as specified by the weight matrixW. For example, the value 0 from the center pixel is inserted three times. Thepixel vector is sorted and the center value (2) is the weighted median.
52/90
ELEG404/604
Center Weighted Median Filter (CWMF)A special case is the center weighted median filter (CWMF)
y(n) = MEDIAN[x1,x2, . . . ,xc−1,xc♦wc,xc+1, . . . ,xN ] (3)where c= (N + 1)/2 =N1 + 1 is the index of the center sample.
For wc = 1, the CWMF reduces to a median filter.
For wc ≥N the CWMF is an identity filter.
By varying the parameter
more smoothing less1←−−−−−−−−→
wcN
53/90
ELEG404/604
Property:Let y(n) be the output of a CWMF, then
y(n) = MEDIAN [x1, . . . ,xc−1,xc♦wc,xc+1, . . . ,xN ]= MEDIAN [x(k),xc,x(N−k+1)]
where k = (N + 2−wc)/2 for 1≤ wc ≤N , and k = 1 for wc >N .Hence,
y(n) =
xc if x(k) ≤ xc ≤ x(N+1−k)x(k) if xc ≤ x(k)x(N+1+k) if xc ≥ x(N+1−k)
54/90
ELEG404/604
Original
55/90
ELEG404/604
Salt and pepper noise
56/90
ELEG404/604
5× 5 window - wc = 1
57/90
ELEG404/604
5× 5 window - wc = 3
58/90
ELEG404/604
5× 5 window - wc = 5
59/90
ELEG404/604
Methods for extending the image to facilitate filtering alongthe borders
Nonexisting pixels outside the original image are either set to some constant value (a), take on the value of the closest
border pixel (b), are mirrored at boundaries (c), or repeat periodically along the coordinate axes (d).
60/90
ELEG404/604
Sharpening Filters
I Objective: Highlight and enhance fine detailI Details may have been blurred in acquisition processI Method: utilize first- and second-order derivativeI Derivatives identify signal changes (details/features)I First-derivative requirements:I Zero in flat regionsI Nonzero along rampsI Second-derivative requirements:I Zero in flat regionsI Zero along ramps of constant slope
61/90
ELEG404/604
First derivative in one dimensionOriginal image (a), horizontal intensity profile f(x) along the center imageline (b), and first derivative f ′(x) (c).
df
dx(x)≈ f(x+ 1)−f(x−1)
2 = 0.5 · (f(x+ 1)−f(x−1)).
62/90
ELEG404/604
Second DerivativeI Utilize difference equations: First derivative:
∂f
∂x= f(x+ 1)−f(x)
Second derivative∂2f
∂x2 = f(x+ 1) +f(x−1)−2f(x)
63/90
ELEG404/604
Derivative Observations
I First-order derivatives generate thick edgesI Second-order derivativesI Have stronger response to detailsI Produce a double response at step changesI Order of response strength. Point, line, step.
I Second-order derivative is therefore preferred for enhancementI Use isotropic (rotation invariant) formulation
64/90
ELEG404/604
Partial Derivatives and the GradientDerivative of a multidimensional function taken along one of its coordinatespartial derivative;
∂I∂x
(x,y) and ∂I∂y
(x,y)
The gradient function is
∇I(x,y) =[∂I∂x(x,y)∂I∂y (x,y)
]
The magnitude of the gradient,
|∇I|(x,y) =√√√√(∂I
∂x(x,y))2 + (∂I
∂y(x,y))2
65/90
ELEG404/604
Derivative Filters
The gradient approximation of the horizontal derivatives is the coefficientmatrix:
HDx = [−0.5 0 0.5] = 0.5 · [−1 0 1]
where the coefficients −0.5 and +0.5 correspond to the image elementsI(x−1,y) and I(x+ 1,y). Vertical component of the gradient
HDy =
−0.50
0.5
= .5 ·
−101
66/90
ELEG404/604
Partial Derivatives of a Two Dimensional Function
(a) Synthetic image,(b) first derivatives in the horizontal direction ∂I/∂x. (c) vertical direction ∂I/∂y; (d)magnitude of the gradient |∇I| (d).
67/90
ELEG404/604
Edge Operators-Various ApproachesDiffer in the type of filter used for estimating the gradient components andhow they are combined.I Roberts Operator: Simplest and oldest edge finder. Employs 2 small filters
2×2 for estimating the directional gradient along the image diagonals:
HRx =
(0 1−1 0
)HRy =
(−1 0
0 1
)
68/90
ELEG404/604
Edge Operators-Various Approaches
I Prewitt and Sobel Operators: Classic methods that differ in the filtersthey use:I Prewitt: Computes an average gradient across 3 neighboring lines or columns
respectively
HPx =
−1 0 1−1 0 1−1 0 1
HPy =
−1 −1 −10 0 01 1 1
I Sobel: The smoothing part assigns higher weight to the current center line and
column, respectively:
HSx =
−1 0 1−2 0 2−1 0 1
HSy =
−1 −2 −10 0 01 2 1
69/90
ELEG404/604
Estimates for Local Gradient Components
I Prewitt:∇I(x,y)≈ 1
6 ·(
(I ∗HPx )(x,y)
(I ∗HPy )(x,y)
)I Sobel:
∇I(x,y)≈ 18 ·(
(I ∗HSx )(x,y)
(I ∗HSy )(x,y)
)
70/90
ELEG404/604
Edge Strength and OrientationDenote the scaled filter results as:
Dx(x,y) =Hx ∗ I and Dy(x,y) =Hy ∗ I
In both cases, the local edge strength E(x,y) is defined as the gradientmagnitude √
(Dx(x,y))2 + (Dy(x,y))2
an the local edge orientation angle φ(x,y) is
φ(x,y) = arctan(Dy(x,y)Dx(x,y))
An improved version of the Sobel filter:
HS′x = 1
32
−3 0 3−10 0 10−3 0 3
HS′y = 1
32
−3 −10 −30 0 03 10 3
71/90
ELEG404/604
Edge Operators
Process of gradient-based edge extraction. Linear filters Hx and Hy producetwo gradient images, Dx and Dy. They are used to compute the edgestrength E and orientation φ for each image position (x,y).
72/90
ELEG404/604
Laplacian Edge OperatorProblem with edge operators based on first derivatives; edges are as wide asthe underlying intensity transition. Alternative makes use of the secondderivatives.Laplacian is the simplest isotropic derivative:
∇2f = ∂2f
∂x2 + ∂2f
∂y2
I
∇2f ≈ [f(x+ 1,y) +f(x−1,y) +f(x,y+ 1) +f(x,y−1)]−4f(x,y)
I Isotropic to 90◦rotationsI Add diagonal derivatives to make it 45◦ isotropic
73/90
ELEG404/604
Mask Implementations and Enhancement
I Similar definition produces a signchange
I Enhancement adds (subtracts)derivative and observed image
g(x,y) ={f(x,y)−∇2f(x,y) if the center is negativef(x,y) +∇2f(x,y) if the center is positive
74/90
ELEG404/604
Example
Top Left: image of the north pole of the moon. Top Right: Laplacian-filtered image.Bottom Left: Laplacian image scaled for display purposes. Bottom Right: Image enhancend
75/90
ELEG404/604
Example
Top Left: Composite Laplacian mask. Top Right:scanning electron microscope imageBottom Left: filtered image with first Laplacian mask. Bottom Right: filtered image withsecond Laplacian mask
76/90
ELEG404/604
77/90
ELEG404/604
Unsharp MaskingUnsharp masking process:
fi(x,y) = f(x,y)−f(x,y)
I Originally a darkroom process combining a blurred negative and positivefilm
Generalization: high-boost filteringfhb(x,y) = Af(x,y)−f(x,y)fhb(x,y) = (A−1)f(x,y)−fs(x,y)
I Using the Laplacian to obtain the sharp image fs(x,y)
fhb ={Af(x,y)−∇2f(x,y) if the center is negativeAf(x,y) +∇2f(x,y) if the center is positive
78/90
ELEG404/604
79/90
ELEG404/604
Edge Sharpening with second derivativeOriginal intensity function f(x), first derivative f ′(x), second derivativef ′′(x), and sharpened intensity function f̆(x) = f(x)−w ·f ′′(x)
80/90
ELEG404/604
Results of Laplace filter HL
(a) Image I(a), (b) second partial derivative ∂2I/∂2u in the horizontaldirection, (c) second partial derivative ∂2I/∂2v in the vertical direction, and(d) Laplace filter ∇2I(u,v).
81/90
ELEG404/604
Edge Sharpening with the Laplace filterOriginal image and marked line (a, b), result of Laplace filter HL (c, d), andsharpened image (e, f).
82/90
ELEG404/604
Unsharp Mask
(a,b) Original image and detail,(c) intensity profile of line; USM filtering withGaussian smoothing σ = 2.5 (d - f) and 10.0 (g - i). Sharpening factor is 1.0(100%).
83/90
ELEG404/604
Mask Implementation
Standard Laplacian sharpening: A= 1.Increasing A, reduces the level of sharpeningI Scales (brightens) original image
84/90
ELEG404/604
High-Boost Example
Top Left: original image. Top Right: Laplacian of the original image withmask(left) in previous image (A= 0). Bottom Left: Laplacian of the originalimage with mask(right) in previous image (A= 1). Bottom Right: Laplacianof the original image with mask(right) in previous image (A= 1.7).
85/90
ELEG404/604
Edges and Contours
I Edges and contours, are of high importance for the visual perception andinterpretation of images.
I Edges: local intensity changes distinctly along a particular orientation.The amount of change with respect to spatial distance is the firstderivative of a function.
86/90
ELEG404/604
Edge Detection
87/90
ELEG404/604
Edge Detection
The derivative of a digital image canbe found by applying the linear filter.
88/90
ELEG404/604
Edge Detection
The edges are then thinned by applying a threshold.
89/90
ELEG404/604
Sobel Example
Left: Optical image of contact lens. Right: Sobel gradient. Commonapplication: edge detectionI Threshold sobel outputI Binary edge mask
90/90
ELEG404/604
Comparison of Edge Operators