Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of...
Transcript of Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of...
![Page 1: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/1.jpg)
1© 2014 The MathWorks, Inc.
Simplifying Computer Vision Application
Development
Tabrez Khan
![Page 2: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/2.jpg)
2
Agenda
Tracking Objects of Interest
Stereo Vision
Using OpenCV with MATLAB
Generating C-code from MATLAB
Summary, Next Steps
![Page 3: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/3.jpg)
3
Agenda
Tracking Objects of Interest
Stereo Vision
Using OpenCV with MATLAB
Generating C-code from MATLAB
Summary, Next Steps
![Page 4: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/4.jpg)
4
Computer Vision – What is it?
Using images and video to detect, classify, and track
objects, activities, or events in order to “understand” a
real-world scene
![Page 5: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/5.jpg)
5
Computer Vision Workflow
Image Acquisition Toolbox
Statistics & Machine
Learning Toolbox
Image Processing Toolbox
Computer Vision
System Toolbox
1. Image/video acquisition
2. Image pre-processing
3. Feature detection
4. Feature extraction
5. Feature matching
6. Using features
– Stabilization, mosaicking
– Targeting, tracking
7. Feature classification
![Page 6: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/6.jpg)
6
Detecting and Tracking a Single Face
![Page 7: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/7.jpg)
7
Workflow: Detecting and Tracking a Single Face
First detect
person/object
Use point tracking
algorithm to track
persons movements
from frame-frame
![Page 8: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/8.jpg)
8
Face Detection ,Tracking and Feature Extraction
Many algorithms available
– vision.CascadeObjectDetector (Viola-Jones)
– vision.HistogramBasedTracker (CAMShift)
– Vision.ForegroundDetector (GMM)
– Fast Retina Keypoint (FREAK) algorithm for feature extraction
– Cascade object detector training using Haar, Histogram of
Oriented Gradients (HOG), and Local Binary Pattern (LBP)
features
![Page 9: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/9.jpg)
9
Agenda
Tracking Objects of Interest
Stereo Vision
Using OpenCV with MATLAB
Generating C-code from MATLAB
Summary, Next Steps
![Page 10: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/10.jpg)
10
Workflow: Depth Estimation from Stereo Camera
Given pair of stereo images. Measure the distances from
the camera to the three faces in the scene.
![Page 11: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/11.jpg)
11
Stereo Vision Workflow
1. Stereo Calibration
Lens distortion
correction
Rectification
2. Depth estimation
3. 3D Scene
reconstruction
![Page 12: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/12.jpg)
12
Result
![Page 13: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/13.jpg)
13
Agenda
Tracking Objects of Interest
Stereo Vision
Using OpenCV with MATLAB
Generating C-code from MATLAB
Summary, Next Steps
![Page 14: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/14.jpg)
14
OpenCV Interface: What's in the package ?
Based on MEX Interface
Provides “custom MEX” command to bring OpenCV
based code into MATLAB
Provides extensive set of data type conversions
between OpenCV and MATLAB
Examples and documentation to help users get started
![Page 15: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/15.jpg)
15
Why use this OpenCV Interface ?
Extensive support for data type conversions
– Differences in data storage paradigms accounted for
Row-major (OpenCV) and column-major (MATLAB)
0-based indexing ( OpenCV) and 1-based indexing (MATLAB)
Ability to bring full OpenCV-based projects not just
single functions
– More efficient
Pre-built OpenCV binaries provided with interfaces
![Page 16: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/16.jpg)
16
Why use OpenCV with MATLAB?
MATLAB advantages for OpenCV users
– MATLAB environment for algorithm development and
exploration
– Visualizations and plots
– Simpler interfaces
OpenCV advantages for MATLAB users
– Explore new algorithms submitted by researchers
– Explore additional methods for common algorithms (e.g. optical
flow and background subtraction)
Collaborate with colleagues using OpenCV or MATLAB
![Page 17: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/17.jpg)
17
Agenda
Tracking Objects of Interest
Stereo Vision
Using OpenCV with MATLAB
Generating C-code from MATLAB
Summary, Next Steps
![Page 18: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/18.jpg)
18
Typical Development Workflow
Algorithm Development Implementation
• Is my idea new? What is required?
• Is it robust to all kinds of conditions?
(lighting noise, etc.)
• Consideration of HW platform
• FPGA? CPU? DSP? GPU?
• Speed and resource requirement
• Resolution, Frame-rate constraint
• Memory constraint
Many iterations Many iterations
Development of the algorithm and implementation
are often done by different groups
Concept Development
Algorithm Development
Prototyping Architecture PrototypingDetailed Design
![Page 19: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/19.jpg)
19
Why Engineers translate MATLAB to C today?
Deploy MATLAB algorithms on Windows/Linux
desktop PC
Integrate MATLAB algorithms w/ existing C
environment
Hand-off code to software engineers for embedded
processor implementation
Accelerate MATLAB algorithms
.exe
.lib
.dll
.c
.cpp
MEX
![Page 20: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/20.jpg)
20
Improved MATLAB Coder app with
Integrated Editor and Simplified Workflow
New user interface simplifies
code generation workflow
![Page 21: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/21.jpg)
21
Vision HDL ToolboxDesign and prototype video image processing systems
Modeling hardware behavior of
the algorithms
– Pixel-based functions and blocks
– Conversion between frames and
pixels
– Standard and custom frame sizes
Prototyping algorithms on
hardware
– (With HDL Coder) Efficient and
readable HDL code
– (With HDL Verifier) FPGA-in-the-loop
testing and acceleration
![Page 22: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/22.jpg)
22
Summary: Why Use MATLAB for Computer
Vision
Save time
– Simpler interfaces
– Pre-trained models for detection and recognition
– Apps to simplify time consuming workflows
Extensive algorithm coverage
– Object detection , recognition and tracking , 3D vision , point
cloud processing
– Feature detection, extraction and matching
Use documentation to learn computer vision
Use OpenCV with MATLAB
![Page 23: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/23.jpg)
23
For More Information
Check videos, webinars, user stories, and demos online
Contact us
– Talk to a sales representative
for a trial copy
– Discuss your projects
with MathWorks applications
engineers
![Page 24: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/24.jpg)
24
Training ServicesExploit the full potential of MathWorks products
Flexible delivery options:
Public training available in several cities
Onsite training with standard or
customized courses
Web-based training with live, interactive
instructor-led courses
More than 30 course offerings:
Introductory and intermediate training on MATLAB, Simulink,
Stateflow, code generation, and Polyspace products
Specialized courses in control design, signal processing, parallel computing,
code generation, communications, financial analysis and other areas.
www.mathworks.in/training
![Page 25: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/25.jpg)
25
Public Trainings in the next Few Months
Course Dates Location
MATLAB Fundamentals 8th – 10th June Chennai
Simulink for System and Algorithm Modeling 11th – 12th June Chennai
Signal Processing with MATLAB 16th – 17th June Bangalore
Image Processing with MATLAB 18th – 19th June Bangalore
MATLAB Fundamentals 29th Jun – 01st July Bangalore
Simulink for System and Algorithm Modeling 2nd – 3rd July Bangalore
Statistical Methods in MATLAB 20th – 21st July Bangalore
MATLAB based Optimization Techniques 22nd Jul Bangalore
Building Interactive Applications with MATLAB 23rd July Bangalore
MATLAB to C with MATLAB Coder 3rd – 4th Aug Bangalore
Embedded Coder for Production Code Generation 5th – 7th Aug Bangalore
Generating HDL Code from Simulink 7th – 8th Sep Bangalore
Programming Xilinx Zynq SoCs with MATLAB and Simulink 10th – 11th Sep Bangalore
Email: [email protected] URL: http://www.mathworks.in/services/training Phone: 080-6632-6000
Guaranteed to run
![Page 26: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/26.jpg)
26
Training Certification
Recommended CoursesMATLAB Fundamentals (MLBE)
Accelerate professional growth
Validate proficiency with MATLAB
Increase productivity and
project success
MathWorks Certified MATLAB Associate Examination
Bangalore Pune29th July & 25th Nov 3rd June
Email: [email protected] URL: http://in.mathworks.com/#training Phone: 080-6632-6000
![Page 27: Simplifying Computer Vision Application Development · (lighting noise, etc.) • Consideration of HW platform • FPGA? CPU? DSP? GPU? • Speed and resource requirement • Resolution,](https://reader030.fdocuments.us/reader030/viewer/2022041221/5e0b78b2c7921458ee2aa791/html5/thumbnails/27.jpg)
27
MathWorks India Contact Details
URL: http://www.mathworks.in
E-mail: [email protected]
Technical Support: www.mathworks.in/myservicerequests
Tel: +91-80-6632 6000
Fax: +91-80-6632 6010
• MathWorks India Private Limited9th Floor, ‘B’ Wing, Etamin BlockPrestige Technology Park IIMarathahalli – Sarjapur Ring RoadBangalore – 560103, KarnatakaIndia
Thank You for Attending
Talk to Us – We are Happy to Support You