Midterm Review

Image Processing

CSE 166

Lecture 10

Announcements

• Midterm exam is on May 4

CSE 166, Spring 2020 2

Topics covered

• Image acquisition, geometric transformations, and image interpolation

• Intensity transformations

• Spatial filtering

• Fourier transform and filtering in the frequency domain

• Image restoration

• Color image processing

CSE 166, Spring 2020 3

Image acquisition

CSE 166, Spring 2020 4

Geometric transformations

CSE 166, Spring 2020 5

Geometric transformations

CSE 166, Spring 2020 6

Euclideantransformation

Similaritytransformation

Affinetransformation

Composition and inversion of transformations

CSE 166, Spring 2020 7

Composition of transformations Inverse of transformation

Firsttransformation

Secondtransformation

Thirdtransformation

Intensity transformations

CSE 166, Spring 2020 8

Contrast stretchingfunction

Thresholdingfunction

Intensity transformations

CSE 166, Spring 2020 9

Some basic transformation functions

Gamma transformation

CSE 166, Spring 2020 10

Gamma transformation

CSE 166, Spring 2020

γ < 1

Darkimage

11

Gamma transformation

CSE 166, Spring 2020

γ > 1

Lightimage

12

Piecewise-linear transformations

• Contrast stretching

• Intensity-level slicing

• Bit-plane slicing

CSE 166, Spring 2020 13

Contrast stretching

CSE 166, Spring 2020 14

Intensity-level slicing

CSE 166, Spring 2020 15

Bit-plane slicing

CSE 166, Spring 2020 16

Histogram

CSE 166, Spring 2020

Similar to probability density function (pdf)17

Histogram equalization

CSE 166, Spring 2020 18

Histogram equalization

CSE 166, Spring 2020 19

Histogram equalization

CSE 166, Spring 2020 20

Spatial filtering (2D)

CSE 166, Spring 2020 21

2D correlation

2D convolution

Correlation and convolution (2D)

CSE 166, Spring 2020 22

Smoothing kernels

CSE 166, Spring 2020 23

Average(box kernel)

Weighted average(Gaussian kernel)

Smoothing with box kernel

CSE 166, Spring 2020 24

3x3

11x11 21x21

Inputimage

Smoothing with Gaussian kernel

CSE 166, Spring 2020 25

Standard deviation σ Percent of total volume under surface

1 39.35

2 86.47

3 98.89

Volume under surface greater than 3σ is negligible

Smoothing with Gaussian kernel

CSE 166, Spring 2020 26

σ = 3.521x21

Input image σ = 743x43

Derivatives

CSE 166, Spring 2020 27

Sharpening filters

CSE 166, Spring 2020 28

Overview: Image processing in the frequency domain

CSE 166, Spring 2020 29

Image in spatial domain

f(x,y)

Image in spatial domain

g(x,y)

Fouriertransform

Image in frequency domain

F(u,v)

Inverse Fourier

transform

Image in frequency domain

G(u,v)

Frequency domain processing

Jean-Baptiste Joseph Fourier1768-1830

Review

• Complex numbers

• Euler’s formula

• Complex functions

CSE 166, Spring 2020 30

1D Fourier series

CSE 166, Spring 2020

Sinesand

cosines

31

Periodicfunction

Weighted by magnitude

Shifted byphase

Period T

Sampling

CSE 166, Spring 2020 32

Sampling

CSE 166, Spring 2020

Over-sampled

Critically-sampled

Under-sampled

1/ΔT

Fourier transform of function

Fourier transforms of sampled function

33

Fourier transform of sampled functionand extracting one period

CSE 166, Spring 2020 34

1D

2D

Over-sampled Under-sampled

Recovered Imperfect recoverydue to

interference

Aliasing

CSE 166, Spring 2020 35

Continuous Sampled

IdenticalDifferent

Sampled at same rate

Over-sampled

Under-sampledAlias: a

false identity

Aliasing

CSE 166, Spring 2020

1D

2D

Aliasing

Original

36

Fourier transform of sampled functionand extracting one period

CSE 166, Spring 2020 37

1D

2D

Over-sampled Under-sampled

Recovered Imperfect recoverydue to

interference

2D discrete Fourier transform (DFT)

• (Forward) Fourier transform

• Inverse Fourier transform

CSE 166, Spring 2020 38

Centering the DFT

CSE 166, Spring 2020

1D

2D

In MATLAB, use fftshift and ifftshift

39

Centering the DFT

CSE 166, Spring 2020

Original

DFT(look at corners)

Shifted DFTLog of

shifted DFT

40

Contributions of magnitude and phaseto image formation

CSE 166, Spring 2020

Phase

IDFT: Phase only

(zero magnitude)

IDFT: Magnitude

only (zero phase)

IDFT: Boy magnitude

and rectangle phase

IDFT: Rectangle magnitude

and boy phase 41

2D convolution theorem

• 2D discrete (circular) convolution

• 2D convolution theorem

CSE 166, Spring 2020 42

Filtering using convolution theorem

CSE 166, Spring 2020

Filtering in spatial domain

using convolution

expectedresult

Filtering in frequencydomain

using productwithout

zero-padding

wraparounderror

43

Filtering using convolution theorem

CSE 166, Spring 2020

Filtering in frequencydomain

using product

withzero-padding

no wraparounderror

Gaussian lowpass filter in

frequency domain

44

Fourier transform

Product

Inverse Fourier transform

Zero padding

Filtering using convolution theorem

CSE 166, Spring 2020

Filtering in spatial

domain using

convolution

Filtering in frequencydomain

usingproduct

Identical results

DFT

45

Filtering in the frequency domain

• Ideal lowpass filter (LPF)

– Frequency domain

CSE 166, Spring 2020 46

Filtering in the frequency domain

• Ideal lowpass filter (LPF)

– Spatial domain

CSE 166, Spring 2020 47

H(u,v) h(x,y)

Filtering in the frequency domain

• Gaussian lowpass filter (LPF)

CSE 166, Spring 2020 48

Filtering in the frequency domain

• Butterworth lowpass filter (LPF)

CSE 166, Spring 2020 49

Gaussian LPF

Filtering in the frequency domain

CSE 166, Spring 2020

Ideal LPF Butterworth LPF

50

Highpass filter (HPF)Frequency domain

CSE 166, Spring 2020

Ideal HPF

Gaussian HPF

Butterworth HPF

51

Butterworth HPF

Highpass filter (HPF)Spatial domain

CSE 166, Spring 2020

Ideal HPF Gaussian HPF

52

Filtering in the frequency domain

CSE 166, Spring 2020

Ideal HPF Gaussian HPF Butterworth HPF

53

H (u)

u u

x x

H (u)

h (x)

1

16–– 3

h (x)

2 12 12 1

2 1 8 2 1

2 12 12 1

0 2 1 0

2 1 4 2 1

0 2 1 0

1 2 1

2

1

9–– 3

4 2

1 2 1

1 1 1

1 1 1

1 1 1

Filtering in the frequency domain

CSE 166, Spring 2020

1D

Lowpass filter Sharpening filter54

Frequencydomain

Spatialdomain

Filtering in the frequency domain

CSE 166, Spring 2020

2D

55

Lowpass filter Highpass filter Offset highpass filter

Bandreject filters

CSE 166, Spring 2020 56

ButterworthIdeal Gaussian

Model of image degradation

• Spatial domain

• Frequency domain

CSE 166, Spring 2020 57

Noiseimage

Originalimage

Degradedimage

Degradationfunction

Noiseimage

Originalimage

Degradedimage

Degradationfunction

Model of image degradation, then restoration

CSE 166, Spring 2020 58

Estimate of original imageOriginal image

Histograms of sample patches

CSE 166, Spring 2020

Sample “flat” patches from images with noise

Identify closest probability density function (pdf) match:

59

Gaussian Rayleigh Uniform

Mean filters

CSE 166, Spring 2020

Additive Gaussian

noise

Geometric mean

filtered

Arithmetic mean

filtered

60

X-ray image

Order-statistic filters

CSE 166, Spring 2020

Additivesalt and peppernoise

1x median filtered

3x median filtered

2x median filtered

61

Comparing filters

CSE 166, Spring 2020

Alpha-trimmed mean filtered

Median filtered

Arithmetricmean

filtered

Geometric mean filtered

62

Additive uniform +

salt and peppernoise

Adaptive filters

CSE 166, Spring 2020

AdditiveGaussian

noise

Arithmetricmean filtered

Geometric mean

filtered

Adaptive noise reduction

filtered

63

Periodic noise

CSE 166, Spring 2020

Conjugate impulses

64

Additive sinusoidal noise

DFT magnitude

Notch reject filters

CSE 166, Spring 2020 65

Notch reject filter

CSE 166, Spring 2020 66

Filter in frequency

domain

Estimate of original

image

Degraded image

DFT magnitudeConjugate

impulses

Conjugate impulses

Estimating the degradation function

• Methods

– Observation

– Experimentation

– Mathematical modeling

CSE 166, Spring 2020 67

Estimation of degradation function by experimentation

CSE 166, Spring 2020 68

Imaged (degraded) impulseImpulse of light

Estimation of degradation function by mathematical modeling

CSE 166, Spring 2020

Atmospheric turbulence

model

69

Image restoration

CSE 166, Spring 2020

Inversefiltering

Wienerfiltering

Degraded image

70

Motion blur and

additive noise

Constrained least squares filtering

Less noise

Much less noise

RGB color model

CSE 166, Spring 2020

RGB coordinates

71

RGB color cube

HSI color model:Relationship to RGB color model

CSE 166, Spring 2020

All colors with cyan

hue

RGB color cube rotated such that line joining black and white

(intensity axis) is vertical

72

HSI color model

CSE 166, Spring 2020

RGB color cube rotated such that

observer is on intensity axis, beyond white looking towards

black

Shape does not matter, only angle from red73

RGB color cube

HSI intensity axis

CSE 166, Spring 2020

HSI

RGB

CMYK

74

Color models

CMY

Pseudocolor image processing

• Intensity slicing

• Intensity to color transformations

CSE 166, Spring 2020 75

Intensity slicing

CSE 166, Spring 2020 76

Grayscale to 2 colors

Intensity slicing

CSE 166, Spring 2020

Grayscale to 2 colors

77

Intensity slicing

CSE 166, Spring 2020

Grayscale to 256 colors

Colorbar

78

Intensity to color transformations

CSE 166, Spring 2020

Grayscale input image

RGBoutput image

79

Intensity to color transformations

CSE 166, Spring 2020

X-ray grayscale input image

Misses explosive

RGB output images

Without explosive

With explosive

80

Intensity to color transformations

CSE 166, Spring 2020

Multiple grayscale

input images

SingleRGB

output image

81

Intensity to color transformations

Near infrared (NIR)

Red (R)

NIR,G,B as RGB R,NIR,B as RGB82

Green (G) Blue (B)

Multiple satellite

grayscale input

images

Output RGB imagesCSE 166, Spring 2020

Full-color image processing

• Two approaches

– Process each channel independently

– Process color vector space

CSE 166, Spring 2020 83

Full-color image processing

CSE 166, Spring 2020

Spatial filtering: process each channel independently

84

Full-color image processing

CSE 166, Spring 2020

All RGB channels

HSI intensity channel only

Spatial filtering: image sharpening

85

Difference

Full-color image processing

CSE 166, Spring 2020

Histogram equalization: do not process each channel independently

1. RGB to HSI2. Histogram

equalize HSI intensity channel

3. HSI to RGB86

4. HSI saturation channel adjustment

Next Lecture

• Basis vectors and matrix-based transforms

– After midterm exam

• Reading

– Sections 6.1 and 6.2: Wavelet and Other Image Transforms

CSE 166, Spring 2020 87