Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing...
Transcript of Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing...
![Page 1: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/1.jpg)
1 © 2011 The MathWorks, Inc.
Computer Vision with MATLAB
MATLAB Expo 2012
Steve Kuznicki
![Page 2: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/2.jpg)
2
Today’s Topics
Introduction
Computer Vision
– Feature-based registration
– Automatic image registration
– Object recognition/Rotation correction with SURF and
RANSAC
– Face detection with Viola-Jones
– Histogram-based tracking with CAMShift
– Stereo image rectification
Video processing
Summary
![Page 3: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/3.jpg)
3
Computer Vision
Using images and video to detect, classify, and track
objects or events in order to “understand” a real-world
scene
Computer
Vision Interpretation
Detect
Identify
Classify
Recognize
Track
…
Pedestrian
Bicyclist
Truck
Car
Traffic violation
Accident
…
Image
Processing
Remove noise
Adjust contrast
Measure
…
![Page 4: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/4.jpg)
4
Examples of Computer Vision with MATLAB
![Page 5: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/5.jpg)
5
Typical Computer Vision Challenges
Variable lighting conditions
Unknown scene depth or perspective
Background clutter
Partially hidden objects (occlusion)
Differences in scale, location, and orientation
![Page 6: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/6.jpg)
6
Technical Computing with MATLAB
Reporting and
Documentation
Outputs for Design
Deployment
Share
Explore and Discover
Data Analysis
and Modeling
Algorithm
Development
Application
Development
Files
Software
Hardware
Access
Code and Applications
![Page 7: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/7.jpg)
7
Key Products for Computer Vision
Computer Vision System Toolbox - NEW
Image Processing Toolbox
MATLAB
Statistics Toolbox
![Page 8: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/8.jpg)
8
Computer Vision System Toolbox
Design and simulate computer vision
and video processing systems
Feature detection
Feature extraction and matching
Feature-based registration
Stereo vision
Video processing
Motion estimation and tracking
Video file I/O, display, and graphics
![Page 9: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/9.jpg)
9
Features are Critical to Computer Vision
Edge
Corner
Template
SURF
MSER
![Page 10: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/10.jpg)
10
Feature-Based Registration Demo
Workflow
– Feature detection
– Feature extraction
– Feature matching
– Geometric transformation
estimation with RANSAC
![Page 11: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/11.jpg)
11
RANSAC
Random Sample Consensus
– Iterative estimation of parameters to a mathematical model
from a set of observed data that contains outliers
Our uses of RANSAC
– Estimate Geometric Transformation
– Estimate Fundamental Matrix (useful for stereo vision)
![Page 12: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/12.jpg)
12
Rotation Correction with SURF Demo
Workflow
– SURF Feature detection
– SURF Descriptor extraction
– Feature matching
– Geometric transformation
estimation with RANSAC
![Page 13: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/13.jpg)
13
Object Recognition with Features Demo
Workflow
– Use SURF features to represent object
– Detect features in video
– Match features from two sources
– Use RANSAC to estimate object location
![Page 14: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/14.jpg)
14
Viola-Jones Face Detection Demo
Algorithm details
– Haar wavelet features using
integral image
– Adaboost classifier for
feature selection
– Cascading of classifiers to
quickly weed out negative
candidates
– Use several modes together
to overcome false positives
![Page 15: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/15.jpg)
15
Histogram-based Tracking Demo
Workflow
– Detect face and nose – get ROI
– Use Hue channel from HSV space
– Initialize histogram tracker
– Reacquire face if track lost
![Page 16: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/16.jpg)
16
Stereo Image Rectification
![Page 17: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/17.jpg)
17
Recovering Scene Depth with Stereo Cameras
![Page 18: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/18.jpg)
18
Epipolar Geometry
![Page 19: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/19.jpg)
19
Fundamental Matrix Demo
XLT
FXR = 0
![Page 20: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/20.jpg)
20
![Page 21: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/21.jpg)
![Page 22: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/22.jpg)
22
![Page 23: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/23.jpg)
23
Video Processing
Video file I/O and display
Video pre-processing
Motion estimation and analysis
![Page 24: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/24.jpg)
24
Motion Estimation and Analysis
Techniques
– Block matching
– Optical flow
– Template matching
– Background estimation
using Gaussian mixture models
Applications
Object tracking
Interpolation
Compression
![Page 25: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/25.jpg)
25
Typical Parts of a Computer Vision Algorithm
1. Image/video acquisition
2. Image/video pre-processing
3. Feature detection
4. Feature extraction
5. Feature matching
6. Using features
– Stabilization, mosaicking
– Stereo image rectification
7. Feature classification
Image Acquisition Toolbox
Statistics Toolbox
Image Processing Toolbox
Computer Vision
System Toolbox
![Page 26: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/26.jpg)
26
Review: Key Products for Computer Vision
with MATLAB
Computer Vision System Toolbox
Image Processing Toolbox
Image Acquisition Toolbox
Statistics Toolbox
![Page 27: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/27.jpg)
27
Statistics Toolbox
Perform statistical analysis, modeling,
and algorithm development
Clustering
– Principle components analysis
– K-means
– Gaussian mixture models
Classification
– Naïve Bayes
– K-nearest neighbor search
– Boosted decision trees
AdaBoost, GentleBoost, LogitBoost,…
![Page 28: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/28.jpg)
28
Why Use MATLAB for Computer Vision?
Comprehensive environment
– Analysis, algorithm development, visualization, etc.
Broad library of algorithms
– Computer vision
– Image processing
– Classification and clustering
Documentation, examples, and technical support
Increased productivity over C/C++ programming
![Page 29: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/29.jpg)
29
For More Information
mathworks.com/products/computer-vision
Relevant demos:
– Barcode Recognition
– Image Rectification
– Traffic Warning Sign Recognition
– People Tracking
– Video Mosaicking
Documentation
Contact your sales representative
![Page 30: Computer Vision with MATLAB Master Class · Image/video acquisition 2. Image/video pre-processing 3. Feature detection 4. Feature extraction 5. Feature matching 6. Using features](https://reader035.fdocuments.us/reader035/viewer/2022062602/5e83a6c4c6061051587eab3a/html5/thumbnails/30.jpg)
30
Questions and Answers