Lecture 3: Linear Filters - Artificial...
Transcript of Lecture 3: Linear Filters - Artificial...
![Page 1: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/1.jpg)
Lecture 3 -Fei-Fei Li
Lecture 3: Linear Filters
Professor Fei‐Fei LiStanford Vision Lab
3‐Oct‐121
![Page 2: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/2.jpg)
Lecture 3 -Fei-Fei Li
What we will learn today?
• Images as functions• Linear systems (filters)• Convolution and correlation• Discrete Fourier Transform (DFT)• Sampling and aliasing
3‐Oct‐122
Some background reading:Forsyth and Ponce, Computer Vision, Chapter 7 & 8Jae S. Lim, Two‐dimensional signal and image processing, Chapter 1, 4, 5
![Page 3: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/3.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐123
Images as functions• An Image as a function f from R2 to RM:
• f( x, y ) gives the intensity at position ( x, y ) • Defined over a rectangle, with a finite range:
f: [a,b] x [c,d ] [0,255]Domainsupport
range
x
yf(x,y)
x
yf(x,y)
![Page 4: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/4.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐124
Images as functions• An Image as a function f from R2 to RM:
• f( x, y ) gives the intensity at position ( x, y ) • Defined over a rectangle, with a finite range:
f: [a,b] x [c,d ] [0,255]
( , )( , ) ( , )
( , )
r x yf x y g x y
b x y
• A color image:
Domainsupport
range
![Page 5: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/5.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐125
• Images are usually digital (discrete):– Sample the 2D space on a regular grid
• Represented as a matrix of integer valuespixel
Images as discrete functions
![Page 6: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/6.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐126
Cartesian coordinates
Images as discrete functions
Notation for discrete functions
![Page 7: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/7.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐127
Matlab notation
Array coordinates
Images as discrete functions
![Page 8: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/8.jpg)
Lecture 3 -Fei-Fei Li
What we will learn today?
• Images as functions• Linear systems (filters)• Convolution and correlation• Discrete Fourier Transform (DFT)• Sampling and aliasing
3‐Oct‐128
Some background reading:Forsyth and Ponce, Computer Vision, Chapter 7 & 8Jae S. Lim, Two‐dimensional signal and image processing, Chapter 1, 4, 5
![Page 9: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/9.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐129
Systems and Filters• Filtering:
– Form a new image whose pixels are a combination original pixel values
Goals: ‐Extract useful information from the images
• Features (edges, corners, blobs…)
‐Modify or enhance image properties: • super‐resolution; in‐painting; de‐noising
![Page 10: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/10.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1210
Super‐resolutionDe‐noising
In‐painting
Bertamioet al
Salt and pepper noise
![Page 11: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/11.jpg)
Lecture 3 -Fei-Fei Li
2D discrete‐space systems (filters)
3‐Oct‐1211
![Page 12: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/12.jpg)
Lecture 3 -Fei-Fei Li
Filters: Examples• 2D DS moving average over a 3 × 3 window of neighborhood
3‐Oct‐1212
111
111
111
],[],[91],)[(
,lnkmhlkfnmhf
lk
h
![Page 13: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/13.jpg)
Lecture 3 -Fei-Fei Li
Moving average
3‐Oct‐1213
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
This image cannot currently be displayed.
Courtesy of S. Seitz
![Page 14: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/14.jpg)
Lecture 3 -Fei-Fei Li
Moving average
3‐Oct‐1214
14
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
]ln,km[g]l,k[f]n,m)[gf(l,k
![Page 15: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/15.jpg)
Lecture 3 -Fei-Fei Li
Moving average
3‐Oct‐1215
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
]ln,km[g]l,k[f]n,m)[gf(l,k
![Page 16: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/16.jpg)
Lecture 3 -Fei-Fei Li
Moving average
3‐Oct‐1216
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
]ln,km[g]l,k[f]n,m)[gf(l,k
![Page 17: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/17.jpg)
Lecture 3 -Fei-Fei Li
Moving average
3‐Oct‐1217
0 10 20 30 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
]ln,km[g]l,k[f]n,m)[gf(l,k
![Page 18: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/18.jpg)
Lecture 3 -Fei-Fei Li
Moving average
3‐Oct‐1218
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0
]ln,km[g]l,k[f]n,m)[gf(l,k
Source: S. Seitz
![Page 19: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/19.jpg)
Lecture 3 -Fei-Fei Li
Moving average
3‐Oct‐1219
In summary:• Replaces each pixel with an average of its neighborhood.
• Achieve smoothing effect (remove sharp features)
111
111
111
],[g
![Page 20: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/20.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1220
Moving average
![Page 21: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/21.jpg)
Lecture 3 -Fei-Fei Li
Filters: Examples
3‐Oct‐1221
• Image segmentation based on a simple threshold:
![Page 22: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/22.jpg)
Lecture 3 -Fei-Fei Li
Classification of systems
3‐Oct‐1222
• Amplitude properties• Linearity• Stability• Invertibility
• Spatial properties• Causality• Separability• Memory• Shift invariance• Rotation invariance
![Page 23: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/23.jpg)
Lecture 3 -Fei-Fei Li
Shift‐invariance
3‐Oct‐1223
• If then
for every input image f[n,m] and shifts n0,m0
• Is the moving average shift invariant a system ?
![Page 24: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/24.jpg)
Lecture 3 -Fei-Fei Li
Is the moving average system is shift invariant?
3‐Oct‐1224
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0
![Page 25: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/25.jpg)
Lecture 3 -Fei-Fei Li
Is the moving average system is shift invariant?
3‐Oct‐1225
Yes!
![Page 26: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/26.jpg)
Lecture 3 -Fei-Fei Li
Linear Systems (filters)
3‐Oct‐1226
• Linear filtering:– Form a new image whose pixels are a weighted sum of original pixel values
– Use the same set of weights at each point
• S is a linear system (function) iff it S satisfies
superposition property
![Page 27: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/27.jpg)
Lecture 3 -Fei-Fei Li
• Is the moving average a linear system?
• Is thresholding a linear system?– f1[n,m] + f2[n,m] > T– f1[n,m] < T– f2[n,m]<T
3‐Oct‐1227
Linear Systems (filters)
No!
![Page 28: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/28.jpg)
Lecture 3 -Fei-Fei Li
LSI (linear shift invariant) systems
3‐Oct‐1228
Impulse response
![Page 29: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/29.jpg)
Lecture 3 -Fei-Fei Li
LSI (linear shift invariant) systems
3‐Oct‐1229
111
111
111
h
Example: impulse response of the 3 by 3 moving average filter:
![Page 30: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/30.jpg)
Lecture 3 -Fei-Fei Li
LSI (linear shift invariant) systems
3‐Oct‐1230
An LSI system is completely specified by its impulse response.
superposition
Discrete convolution
sifting property of the delta function
![Page 31: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/31.jpg)
Lecture 3 -Fei-Fei Li
What we will learn today?
• Images as functions• Linear systems (filters)• Convolution and correlation• Discrete Fourier Transform (DFT)• Sampling and aliasing
3‐Oct‐1231
Some background reading:Forsyth and Ponce, Computer Vision, Chapter 7 & 8Jae S. Lim, Two‐dimensional signal and image processing, Chapter 1, 4, 5
![Page 32: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/32.jpg)
Lecture 3 -Fei-Fei Li
Discrete convolution
3‐Oct‐1232
• Fold h[n,m] about origin to form h[−k,−l]• Shift the folded results by n,m to form h[n − k,m − l]• Mul ply h[n − k,m − l] by f[k, l]• Sum over all k,l• Repeat for every n,m
h[k,l]
h[‐k,‐l] h[n‐k,m‐l]
n
![Page 33: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/33.jpg)
Lecture 3 -Fei-Fei Li
Discrete convolution
3‐Oct‐1233
• Fold h[n,m] about origin to form h[−k,−l]• Shift the folded results by n,m to form h[n − k,m − l]• Mul ply h[n − k,m − l] by f[k, l]• Sum over all k,l• Repeat for every n,m
h[k,l]
h[‐k,‐l] h[n‐k,m‐l]
n
![Page 34: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/34.jpg)
Lecture 3 -Fei-Fei Li
Discrete convolution
3‐Oct‐1234
• Fold h[n,m] about origin to form h[−k,−l]• Shift the folded results by n,m to form h[n − k,m − l]• Mul ply h[n − k,m − l] by f[k, l]• Sum over all k,l• Repeat for every n,m
h[n‐k,m‐l]
f[k,l]
f[k,l] x h[n‐k,m‐l]
Sum (f[k,l] x h[n‐k,m‐l])
n
![Page 35: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/35.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D ‐ examples
3‐Oct‐1235
•0•0•0
•0•1•0
•0•0•0
Original
?=
Courtesy of D
Low
e
![Page 36: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/36.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D ‐ examples
3‐Oct‐1236
•0•0•0
•0•1•0
•0•0•0
Original Filtered (no change)
=
Courtesy of D
Low
e
![Page 37: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/37.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D ‐ examples
3‐Oct‐1237
•0•0•0
•1•0•0
•0•0•0
Original
?=
Courtesy of D
Low
e
![Page 38: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/38.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D ‐ examples
3‐Oct‐1238
•0•0•0
•1•0•0
•0•0•0
Original Shifted rightBy 1 pixel
=
Courtesy of D
Low
e
![Page 39: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/39.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D ‐ examples
3‐Oct‐1239
Original
?•1•1•1
•1•1•1
•1•1•1
=
Courtesy of D
Low
e
![Page 40: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/40.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D ‐ examples
3‐Oct‐1240
Original
•1•1•1
•1•1•1
•1•1•1
Blur (with abox filter)
=
Courtesy of D
Low
e
![Page 41: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/41.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D ‐ examples
3‐Oct‐1241
Original
•1•1•1•1•1•1•1•1•1
•0•0•0•0•2•0•0•0•0 - = ?
(Note that filter sums to 1)
•1•1•1•1•1•1•1•1•1
•0•0•0•0•1•0•0•0•0 -
•0•0•0•0•1•0•0•0•0
+
“details of the image”
Courtesy of D
Low
e
![Page 42: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/42.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1242
• What does blurring take away?
original smoothed (5x5)
–
detail
=
sharpened
=
• Let’s add it back:
original detail
+ a
![Page 43: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/43.jpg)
Lecture 3 -Fei-Fei Li
Convolution in 2D –Sharpening filter
3‐Oct‐1243
Original
•1•1•1•1•1•1•1•1•1
•0•0•0•0•2•0•0•0•0 -
Sharpening filter: Accentuates differences with local average
=
Courtesy of D
Low
e
![Page 44: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/44.jpg)
Lecture 3 -Fei-Fei Li
Convolution properties
3‐Oct‐1244
• Commutative property:
• Associative property:
• Distributive property:
The order doesn’t matter!
![Page 45: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/45.jpg)
Lecture 3 -Fei-Fei Li
Convolution properties
3‐Oct‐1245
• Shift property:
• Shift‐invariance:
![Page 46: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/46.jpg)
Lecture 3 -Fei-Fei Li
Image support and edge effect
3‐Oct‐1246
•A computer will only convolve finite support signals.
• That is: images that are zero for n,m outside some rectangular region
• MATLAB’s conv2 performs 2D DS convolution of finite‐support signals.
N1 ×M1
N2 ×M2 (N1 + N2 − 1) × (M1 +M2 − 1)* =
![Page 47: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/47.jpg)
Lecture 3 -Fei-Fei Li
Image support and edge effect
3‐Oct‐1247
•A computer will only convolve finite support signals. • What happens at the edge?
fh
• zero “padding”• edge value replication• mirror extension• more (beyond the scope of this class)
‐> Matlab conv2 uses zero‐padding
![Page 48: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/48.jpg)
Lecture 3 -Fei-Fei Li
Cross correlation
3‐Oct‐1248
Cross correlation of two 2D signals f[n,m] and g[n,m]
• Equivalent to a convolution without the flip
(k, l) is called the lag
![Page 49: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/49.jpg)
Lecture 3 -Fei-Fei Li
Cross correlation – example
3‐Oct‐1249
MATLAB’s xcorr2
Courtesy of J. Fessler
![Page 50: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/50.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1250
Cross correlation – exampleLeft Right
scanline
Norm. corr
![Page 51: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/51.jpg)
Lecture 3 -Fei-Fei Li
Convolution vs. Correlation
• A convolution is an integral that expresses the amount of overlap of one function as it is shifted over another function. – convolution is a filtering operation
• Correlation compares the similarity of two sets of data. Correlation computes a measure of similarity of two input signals as they are shifted by one another. The correlation result reaches a maximum at the time when the two signals match best .– correlation is a measure of relatedness of two signals
2010.12.1851
![Page 52: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/52.jpg)
Lecture 3 -Fei-Fei Li
What we will learn today?
• Images as functions• Linear systems (filters)• Convolution and correlation• Discrete Fourier Transform (DFT)• Sampling and aliasing
3‐Oct‐1252
Some background reading:Forsyth and Ponce, Computer Vision, Chapter 7 & 8Jae S. Lim, Two‐dimensional signal and image processing, Chapter 1, 4, 5
![Page 53: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/53.jpg)
Lecture 3 -Fei-Fei Li
2D Discrete‐Signal Fourier Transform (DSFT)
3‐Oct‐1253
2D discrete‐signal Fourier transform (DSFT) of a 2D discrete‐space signal g[n,m]:
Inverse 2D DSFT:
For memory refresher: Forsyth and Ponce, Ch 8
![Page 54: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/54.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1254
![Page 55: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/55.jpg)
Lecture 3 -Fei-Fei Li
DSFT – properties
3‐Oct‐1255
Shift:
Convolution:
Delta function:
![Page 56: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/56.jpg)
Lecture 3 -Fei-Fei Li
Example: DSFT of moving average filter
3‐Oct‐1256
],[],[91],)[(
,lnkmhlkfnmhf
lk
111
111
111
h
![Page 57: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/57.jpg)
Lecture 3 -Fei-Fei Li
![Page 58: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/58.jpg)
Lecture 3 -Fei-Fei Li
Why is DFT important?
3‐Oct‐1258
• Perform efficient linear convolution as product of DFTs • Each DFT can be implemented using the FFT (Fast Fourier Transform ) [see appendix for details]
![Page 59: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/59.jpg)
Lecture 3 -Fei-Fei Li
What we will learn today?
• Images as functions• Linear systems (filters)• Convolution and correlation• Discrete Fourier Transform (DFT)• Sampling and aliasing
3‐Oct‐1259
Some background reading:Forsyth and Ponce, Computer Vision, Chapter 7 & 8Jae S. Lim, Two‐dimensional signal and image processing, Chapter 1, 4, 5
![Page 60: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/60.jpg)
Lecture 3 -Fei-Fei Li
Sampling
3‐Oct‐1260
Throw away every other row and column to create a 1/2 size image
![Page 61: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/61.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1261
• Down‐sampling operation:(trivial form of image compression)
Sampling
![Page 62: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/62.jpg)
Lecture 3 -Fei-Fei Li
Why is a multi‐scale representation useful?
3‐Oct‐1262
• Find template matches at all scales– e.g., when finding hands or faces, we don’t know what size they will be in a particular image
– Template size is constant, but image size changes
• Efficient search for correspondence– look at coarse scales, then refine with finer scales
• Examining all levels of detail– Find edges with different amounts of blur– Find textures with different spatial frequencies (levels of detail)
Slide credit: David Lowe (UBC)
![Page 63: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/63.jpg)
Lecture 3 -Fei-Fei Li
Things can go wrong! ‐‐ Aliasing
3‐Oct‐1263
![Page 64: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/64.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1264
Things can go wrong! ‐‐ Aliasing
![Page 65: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/65.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1265
![Page 66: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/66.jpg)
Lecture 3 -Fei-Fei Li
Sampling Theorem (Nyquist)
3‐Oct‐1266
• When sampling a signal at discrete intervals, the sampling frequency must be 2 fmax
• fmax = max frequency of the input signal.
good
bad
v v v
![Page 67: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/67.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1267
x
y
• N = sampling period• T = periodicity of the replicas of the DFT (g)
T
T 1/N
gsampled [n,m]g [n,m]Small sampling period N
![Page 68: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/68.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1268
gsampled [n,m]
x
y
…
…
…
……
…
g [n,m]Large sampling period N
• N = sampling period• T = periodicity of the replicas of the DFT (g)
T 1/N
![Page 69: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/69.jpg)
Lecture 3 -Fei-Fei Li
Anti‐aliasing
3‐Oct‐1269
Solutions:• Sample more often
• Get rid of all frequencies that are greater than half the new sampling frequency– Will lose information ‐ but it’s better than aliasing– Apply a smoothing filter to remove high
frequencies
![Page 70: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/70.jpg)
Lecture 3 -Fei-Fei Li 3‐Oct‐1270
Anti‐aliasing
x
)(G
Low pass filter (smoothing)
Apply a smoothing filter to remove high frequencies:
![Page 71: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/71.jpg)
Lecture 3 -Fei-Fei Li
Sampling algorithm
3‐Oct‐1271
![Page 72: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/72.jpg)
Lecture 3 -Fei-Fei Li
Resampling with Prior Smoothing
• Note: We cannot recover the high frequencies, but we can avoid artifacts by smoothing before resampling.
Image Source: Forsyth & Ponce
3‐Oct‐12
![Page 73: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/73.jpg)
Lecture 3 -Fei-Fei Li
The Gaussian Pyramid
High resolution
Low resolution
2)*( 23 gaussianGG
1G
Image0G
2)*( 01 gaussianGG
2)*( 12 gaussianGG
2)*( 34 gaussianGG
blur
blur
blur
blur
3‐Oct‐1273
![Page 74: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/74.jpg)
Lecture 3 -Fei-Fei Li
Gaussian Pyramid – Stored Information
Source: Irani & Basri
All the extra levels add very little overhead for memory or computation!
3‐Oct‐1274
![Page 75: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/75.jpg)
Lecture 3 -Fei-Fei Li
Summary: Gaussian Pyramid
• Construction: create each level from previous one– Smooth and sample
• Smooth with Gaussians, in part because– a Gaussian*Gaussian = another Gaussian – G(1) * G(2) = G(sqrt(1
2 + 2 2))
• Gaussians are low‐pass filters, so the representation is redundant once smoothing has been performed. There is no need to store smoothed images at the
full original resolution.
Slide credit: David Lowe
3‐Oct‐1275
![Page 76: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/76.jpg)
Lecture 3 -Fei-Fei Li
Figure from “Computer Vision for Interactive Computer Graphics,” W.Freeman et al, IEEE Computer Graphics and Applications, 1998 copyright 1998, IEEE
Application: Vision system for TV remote control
- uses template matching
3‐Oct‐1276
![Page 77: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/77.jpg)
Lecture 3 -Fei-Fei Li
What we have learned today?
• Images as functions• Linear systems (filters)• Convolution and correlation• Discrete Fourier Transform (DFT)• Sampling and aliasing
3‐Oct‐1277
![Page 78: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/78.jpg)
Lecture 3 -Fei-Fei Li
Appendix
3‐Oct‐1278
![Page 79: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/79.jpg)
Lecture 3 -Fei-Fei Li
Convergence
If g absolutely summable:
then
If g is square summable(energy signal):
3‐Oct‐1279
![Page 80: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/80.jpg)
Lecture 3 -3‐Oct‐12 80
![Page 81: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/81.jpg)
Lecture 3 -Fei-Fei Li
Fast Fourier transforms (FFT)
3‐Oct‐1281
• Brute‐force evaluation of the 2D DFT would require O((NM)2) flops
• DFT is a separable operation we can reduce greatly the computation
![Page 82: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/82.jpg)
Lecture 3 -Fei-Fei Li
Fast Fourier transforms (FFT)
3‐Oct‐1282
• DFT is a separable operation:
•Apply the 1D DFT to each column of the image, and then apply the 1D DFT to each row of the result.
• Use the fast Fourier transform (1‐D FFT) for these 1D DFTs!
![Page 83: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/83.jpg)
Lecture 3 -Fei-Fei Li
Fast Fourier transforms (FFT)
3‐Oct‐1283
• FFT computational efficiency:
• inner set of 1D FFTs : N O(M logM)• outer set of 1D FFTs: M O(N logN)
•Total: O(MN logMN) flops
• A critical property of FFT is that N = 2k with k =integer
If x = 512x512 image saving is a factor of 15000 relative to the brute‐force 2D DFT!
Matlab: fft2 = fft(fft(x).’).’
![Page 84: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/84.jpg)
Lecture 3 -Fei-Fei Li
FFT & Efficiency
3‐Oct‐1284
• General goal: perform efficient linear convolution •Perform convolution as product of DFTs • Pros: DFT can be implemented using the FFT (fast fourier transform)
•FFT is very efficient (fast!)
• Cons: DFT perform circular convolution•Compensate the wrap‐around effect
•Cons: Online‐memory storage•Use the overlap‐add method or overlap‐save method
![Page 85: Lecture 3: Linear Filters - Artificial Intelligencevision.stanford.edu/teaching/cs231a_autumn1213/lecture/lecture3_linear_filters_cs231a...Fei-Fei Li Lecture 3 - 3 3‐Oct‐12 Images](https://reader031.fdocuments.us/reader031/viewer/2022041101/5ed905b96714ca7f47690057/html5/thumbnails/85.jpg)
Lecture 3 -Fei-Fei Li
FFT & Efficiency
3‐Oct‐1285
•Suppose we wish to convolve a 256 × 256 image with a 17 × 17 filter. •The result will be 272×272.•The smallest prime factors of 272 is 2. •So one could pad to a 512 × 512 image•Note: only 28% of the final image would be the part we care about ‐ the rest would be zero in exact arithmetic.
•Handling to a 512 × 512 image requires much memory
Use overlap‐add method