Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1....
Transcript of Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1....
![Page 1: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/1.jpg)
Templates, Image Pyramids, and Filter Banks
Computer Vision
James Hays, Brown
Slides: Hoiem and others
![Page 2: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/2.jpg)
Reminder
• Project 1 due Friday
![Page 3: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/3.jpg)
Fourier Bases
This change of basis is the Fourier Transform
Teases away fast vs. slow changes in the image.
![Page 4: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/4.jpg)
Fourier Bases
in Matlab, check out: imagesc(log(abs(fftshift(fft2(im)))));
![Page 5: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/5.jpg)
Man-made Scene
![Page 6: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/6.jpg)
Can change spectrum, then reconstruct
![Page 7: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/7.jpg)
Low and High Pass filtering
![Page 8: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/8.jpg)
• What is the spatial representation of the hard cutoff in the frequency domain?
Sinc Filter
Frequency Domain Spatial Domain
![Page 9: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/9.jpg)
Review
1. Match the spatial domain image to the Fourier magnitude image
1 5 4
A
3 2
C
B
D
E
![Page 10: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/10.jpg)
Today’s class
• Template matching
• Image Pyramids
• Filter banks and texture
![Page 11: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/11.jpg)
Template matching
• Goal: find in image
• Main challenge: What is a good similarity or distance measure between two patches? – Correlation
– Zero-mean correlation
– Sum Square Difference
– Normalized Cross Correlation
![Page 12: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/12.jpg)
Matching with filters
• Goal: find in image
• Method 0: filter the image with eye patch
Input Filtered Image
],[],[],[,
lnkmflkgnmhlk
What went wrong?
f = image
g = filter
![Page 13: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/13.jpg)
Matching with filters
• Goal: find in image
• Method 1: filter the image with zero-mean eye
Input Filtered Image (scaled) Thresholded Image
)],[()],[(],[,
lnkmgflkfnmhlk
True detections
False
detections
mean of f
![Page 14: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/14.jpg)
Matching with filters
• Goal: find in image
• Method 2: SSD
Input 1- sqrt(SSD) Thresholded Image
2
,
)],[],[(],[ lnkmflkgnmhlk
True detections
![Page 15: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/15.jpg)
Matching with filters
• Goal: find in image
• Method 2: SSD
Input 1- sqrt(SSD)
2
,
)],[],[(],[ lnkmflkgnmhlk
What’s the potential
downside of SSD?
![Page 16: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/16.jpg)
Matching with filters
• Goal: find in image
• Method 3: Normalized cross-correlation
5.0
,
2
,
,
2
,
,
)],[()],[(
)],[)(],[(
],[
lk
nm
lk
nm
lk
flnkmfglkg
flnkmfglkg
nmh
Matlab: normxcorr2(template, im)
mean image patch mean template
![Page 17: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/17.jpg)
Matching with filters
• Goal: find in image
• Method 3: Normalized cross-correlation
Input Normalized X-Correlation Thresholded Image
True detections
![Page 18: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/18.jpg)
Matching with filters
• Goal: find in image
• Method 3: Normalized cross-correlation
Input Normalized X-Correlation Thresholded Image
True detections
![Page 19: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/19.jpg)
Q: What is the best method to use?
A: Depends
• SSD: faster, sensitive to overall intensity
• Normalized cross-correlation: slower, invariant to local average intensity and contrast
• But really, neither of these baselines are representative of modern recognition.
![Page 20: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/20.jpg)
Q: What if we want to find larger or smaller eyes?
A: Image Pyramid
![Page 21: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/21.jpg)
Review of Sampling
Low-Pass Filtered Image
Image
Gaussian
Filter Sample Low-Res Image
![Page 22: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/22.jpg)
Gaussian pyramid
Source: Forsyth
![Page 23: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/23.jpg)
Template Matching with Image Pyramids
Input: Image, Template
1. Match template at current scale
2. Downsample image
3. Repeat 1-2 until image is very small
4. Take responses above some threshold, perhaps with non-maxima suppression
![Page 24: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/24.jpg)
Coarse-to-fine Image Registration
1. Compute Gaussian pyramid
2. Align with coarse pyramid
3. Successively align with finer pyramids – Search smaller range
Why is this faster?
Are we guaranteed to get the same result?
![Page 25: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/25.jpg)
2D edge detection filters
is the Laplacian operator:
Laplacian of Gaussian
Gaussian derivative of Gaussian
![Page 26: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/26.jpg)
Laplacian filter
Gaussian unit impulse
Laplacian of Gaussian
Source: Lazebnik
![Page 27: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/27.jpg)
Computing Gaussian/Laplacian Pyramid
http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html
Can we reconstruct the original
from the laplacian pyramid?
![Page 28: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/28.jpg)
Laplacian pyramid
Source: Forsyth
![Page 29: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/29.jpg)
Hybrid Image
![Page 30: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/30.jpg)
Hybrid Image in Laplacian Pyramid
High frequency Low frequency
![Page 31: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/31.jpg)
Image representation
• Pixels: great for spatial resolution, poor access to frequency
• Fourier transform: great for frequency, not for spatial info
• Pyramids/filter banks: balance between spatial and frequency information
![Page 32: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/32.jpg)
Major uses of image pyramids
• Compression
• Object detection
– Scale search – Features
• Detecting stable interest points
• Registration – Course-to-fine
![Page 33: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/33.jpg)
Application: Representing Texture
Source: Forsyth
![Page 34: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/34.jpg)
Texture and Material
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
![Page 35: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/35.jpg)
Texture and Orientation
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
![Page 36: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/36.jpg)
Texture and Scale
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
![Page 37: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/37.jpg)
What is texture?
Regular or stochastic patterns caused by bumps, grooves, and/or markings
![Page 38: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/38.jpg)
How can we represent texture?
• Compute responses of blobs and edges at various orientations and scales
![Page 39: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/39.jpg)
Overcomplete representation: filter banks
LM Filter Bank
Code for filter banks: www.robots.ox.ac.uk/~vgg/research/texclass/filters.html
![Page 40: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/40.jpg)
Filter banks
• Process image with each filter and keep responses (or squared/abs responses)
![Page 41: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/41.jpg)
How can we represent texture?
• Measure responses of blobs and edges at various orientations and scales
• Idea 1: Record simple statistics (e.g., mean, std.) of absolute filter responses
![Page 42: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/42.jpg)
Can you match the texture to the response?
Mean abs responses
Filters A
B
C
1
2
3
![Page 43: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/43.jpg)
Representing texture by mean abs response
Mean abs responses
Filters
![Page 44: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/44.jpg)
Representing texture
• Idea 2: take vectors of filter responses at each pixel and cluster them, then take histograms (more on in later weeks)
![Page 45: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/45.jpg)
Review of last three days
![Page 46: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/46.jpg)
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Credit: S. Seitz
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Review: Image filtering
1 1 1
1 1 1
1 1 1
],[g
![Page 47: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/47.jpg)
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Image filtering
1 1 1
1 1 1
1 1 1
],[g
Credit: S. Seitz
![Page 48: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/48.jpg)
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Image filtering
1 1 1
1 1 1
1 1 1
],[g
Credit: S. Seitz
![Page 49: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/49.jpg)
Filtering in spatial domain -1 0 1
-2 0 2
-1 0 1
* =
![Page 50: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/50.jpg)
Filtering in frequency domain
FFT
FFT
Inverse FFT
=
![Page 51: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/51.jpg)
Review of Last 3 Days
• Filtering in frequency domain
– Can be faster than filtering in spatial domain (for large filters)
– Can help understand effect of filter
– Algorithm:
1. Convert image and filter to fft (fft2 in matlab)
2. Pointwise-multiply ffts
3. Convert result to spatial domain with ifft2
![Page 52: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/52.jpg)
Review of Last 3 Days
• Linear filters for basic processing
– Edge filter (high-pass)
–Gaussian filter (low-pass)
FFT of Gaussian
[-1 1]
FFT of Gradient Filter
Gaussian
![Page 53: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/53.jpg)
Review of Last 3 Days
• Derivative of Gaussian
![Page 54: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/54.jpg)
Review of Last 3 Days
• Applications of filters – Template matching (SSD or Normxcorr2)
• SSD can be done with linear filters, is sensitive to overall intensity
– Gaussian pyramid • Coarse-to-fine search, multi-scale detection
– Laplacian pyramid • Teases apart different frequency bands while keeping
spatial information
• Can be used for compositing in graphics
– Downsampling • Need to sufficiently low-pass before downsampling
![Page 55: Templates, Image Pyramids, and Filter Bankscs.brown.edu/courses/csci1430/2013/lectures/06.pdf · 1. Match template at current scale 2. Downsample image 3. Repeat 1-2 until image is](https://reader034.fdocuments.us/reader034/viewer/2022043012/5fab70d85417397c4c4b05d5/html5/thumbnails/55.jpg)
Next Lectures
• Image representation (e.g. SIFT) and matching across multiple views (e.g. Stereo, Structure from Motion).