BDTI ESCSV 2012 Intro Computer Vision

8/2/2019 BDTI ESCSV 2012 Intro Computer Vision

1/34

Berkeley Design Technology, Inc.Oakland, California USA

+1 (510) 451-1800

[email protected]://www.BDTI.com

The most trusted source of analysis, advice, and engineeringfor embedded processing technology and applications

1Copyright 2012 Berkeley Design Technology, Inc.

Introduction to Computer Visionusing OpenCV

Eric Gregori

Presented at the2012 Embedded Systems Conference in San Jose


2/34

ANALYSIS ADVICE ENGINEERINGFOR EMBEDDED PROCESSING TECHNOLOGY 2012 BDTI

What is OpenCV?

2

An open source

library of over500 functions

Over 2 dozenexamples

An easy tool for

experimentingwith computervision

C/C++/PythonAPI

Windows/Linux/Android/iPhoneplatforms

Over 3,000,000

downloads


3/34


What Can OpenCV Do?

3
http://www.pedestrian-detection.com/http://opencv.willowgarage.com/wiki/GSOC_OpenCV2011http://www.google.com/imgres?q=opencv+road+detection&hl=en&rlz=1C1ASUT_enUS444US444&nord=1&biw=1248&bih=680&tbm=isch&tbnid=0-P03W71_m-2pM:&imgrefurl=http://www.vustv.com/A1K4FuZ1wvmef&docid=q0a36MDoQZyc2M&itg=1&w=480&h=360&ei=ikxuTujcJ5HIsQKcs7mvBA&zoom=1&iact=hc&vpx=918&vpy=260&dur=3959&hovh=193&hovw=258&tx=125&ty=91&page=7&tbnh=134&tbnw=189&start=101&ndsp=16&ved=1t:429,r:15,s:101


4/34


OpenCV in The Embedded Space

OpenCV has always been available to theembedded space under Linux.

The library has been ported to: PowerPC,MIPS, Blackfin, Xscale and ARM.

If it can run Linux, it can run OpenCV.

4

http://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.html

On YouTube, you can find demonstrations of OpenCV

running on the TI Beagleboard, Freescale i.MX53Quick Start Board, and various NVIDIA based tabletsand phones.

In the mobile market you can find examples ofOpenCV running on the iPhone and of-course Androiddevices.
http://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://leanxcam.origo.ethz.ch/wiki/screenshotshttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://whatnicklife.blogspot.com/2010/05/beagle-has-2-eyes-opencv-stereo-on.htmlhttp://www.bitartist.org/2010/05/stereo-vision-on-beagleboard.html


5/34


OpenCV Licensing

OpenCVis released under a BSD license, it is free for both academicand commercial use.

The BSD License allows proprietary use, and for the software releasedunder the license to be incorporated into proprietary products. Worksbased on the material may be released under a proprietary license oras closed source software. This is the reason for widespread use of theBSD code in proprietary products, ranging from JuniperNetworks routers to Mac OS X.

5


6/34


Try OpenCV for Yourself

BDTI has created some interactive examples to demonstrate just asmall part of what you can do with OpenCV.

The build tools, OpenCV libraries, and examples are shipped as aVMware image.

The BDTI OpenCV VMware image provides an easy to use pre-built

environment to get you up and running on OpenCV in minutes.

Simply download the free VMware player and BDTI OpenCV VMwareimage and get started in developing with OpenCV now.

You can download the BDTI OpenCV VMware image from the

Embedded Vision Alliance website at:

http://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImage

6
http://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImage


7/34


GETTING STARTED WITH OPENCV THE EASY WAY

7


8/34


All The Installation and Configuration Has Been

Done for You

8

The Ubuntu OSand GCC compilerruns in a VMwareimage.

OpenCV is pre-installed andconfigured with allsource.

Example

applications usethe Eclipsegraphicaldebuggingenvironment.


9/34


The BasicsAn Image and AFrame

Both images and frames aremade up of individual pixelsorganized in a 2 dimensionalarray.

For a color image, each pixel

can be anything from 8 to 32 bitswide.

Most monochrome images use 8bits per pixel.

A frame is a single image in a

video sequence.

9

X

Y

pixel


10/34


FeatureA Fundamental Concept in Computer

Vision

feature (fchr)n.

A prominent or distinctive aspect, quality, or characteristic: a feature ofonespersonality; a feature of the landscape.http://www.thefreedictionary.com/feature

10

The concept of, a feature of an object is very important for most computervision algorithms.

In a image or frame, a feature is a group of pixels with some uniqueattribute.

corner points edge contrast motion
http://www.thefreedictionary.com/featurehttp://en.wikipedia.org/wiki/File:Optical_flow_example_v2.pnghttp://www.thefreedictionary.com/featurehttp://www.thefreedictionary.com/feature


11/34


Some Basic Building Block Algorithms Used in

Computer Vision

Detection

Motion DetectionFinds groups of pixels (features) that are inmotion (change in position from one frame to the next).

Line DetectionFinds groups of pixels (features) that are

organized in straight lines, along edges. Face DetectionFinds groups of pixels organized in a group that

fits the template of a face.

Tracking

Optical Flow based trackingA combination of algorithms usedto track moving objects in a video using features.

11


12/34


MOTION DETECTION

12


13/34


Motion Detection

Motion detection in this context is doneusing frame subtraction, commonlyreferred to as background subtraction.

The video is converted to monochromeand each pixel in the previous frame is

subtracted from the current frame.

If nothing changed between frames, theresult of all the pixel subtractions will be 0.

13

previous framecurrent frame =


14/34


Demo 1Motion Detection

LearnRateRegulates theupdate speed (how fast theaccumulator forgets aboutearlier images).

ThresholdThe minimum value

for a pixel difference to beconsidered moving.

14


15/34


LINE DETECTION

15


16/34


Line Detection Using Edges

In a monochrome image, a line is defined as a group of pixelsorganized along a straight edge.

An edge in a monochrome image is defined as a dark pixel next to alighter pixel.

16

original image detected edge


17/34


Line Detection Using Edges

After the edge detector finds the edges,another algorithm called the Houghtransform finds edge pixels that line-up instraight lines.

Straight lines are a valuable feature in aimage.

Straight lines define the boundaries ofobjects in an image and can be used fortracking purposes in a video stream.

17


18/34


Demo 2Line Detection Using Edges

ThresholdSet the minimum differencebetween adjoining groups of pixels to beclassified as an edge.

MinLengthThe minimum number ofcontinuous edge pixels required to beclassified as a straight line.

MaxGapThe maximum number ofmissing edge pixels within a straight line,while still being considered continuous.

18


19/34


FACE DETECTION

19


20/34


Face Detection

20

As the name implies, face detection is used to find faces in an image. Theunderlying algorithm can actually be used to detect any object. This algorithmis trained to look for specific features, in a specific order.

Training is done offline, and is

accomplished by showing thelearning algorithm both positiveand negative images (imageswith a face and without a face).

The result of the training is a file

that describes the object todetect, using very specificfeatures.


21/34


Face Detection

Face detection, as demonstrated in this context, uses a set of fourdistinct templates to define unique features. Templates are usedbecause they can be processed faster then other techniques. Thetemplate is laid over a portion of the image, and a weight is calculatedbased on the pixels under the template.

21
http://en.wikipedia.org/wiki/File:Prm_VJ_fig1_featureTypesWithAlpha.png


22/34


Face Detection

How does training work?

A face of 24

24 pixels can have 45,396 possible combinations/scalesof the templates from the pervious slide.

The purpose of training is to reduce the 45,396 possible combinations

down to a minimum number and an ideal order.

22

All Sub-Windows Further Processing

Reject Sub-Window

1 2 3T T T

F FF


23/34


Demo 3Face Detection

MinSizeThe smallest face to detect.As a face gets further from the camera, itappears smaller. This parameter alsodefines the farthest a face can be fromthe camera and still get detected.

MinNThe Minimum Neighbor

parameter groups faces that aredetected multiple times into onedetection.

ScaleFScale Factor determines thenumber of times the face detector is runat each pixel location. The Haar

Cascade (xml file) that determines whatthe detector will detect, is designed foran object of only one size. In order todetect objects of various sizes (facesclose to the camera as well as far fromthe camera) the detector must be scaled.

23


24/34


OPTICAL FLOW BASED TRACKING

24


25/34


Optical Flow Based Tracking

Optical flow is the change in position of a group of pixels (feature) fromone image to the next.

25


26/34


Optical Flow

The first step in optical flow is determining which features to use.

The algorithm used in this example uses features like corners, edges,and points of brightness.

These good features are found in each frame of a video stream.

With this data, a tracking algorithm can then be applied to predictwhere the object will appear in the next frame. This is object tracking.

26


27/34


Demo 4Optical Flow Based Tracking

MaxCountThe maximum number ofgood features to look for in a frame.

qlevelThe quality of the features toaccept. A higher quality feature is morelikely to be unique, and be correctly found

in the next frame. A low quality featuremay get lost in the next frame, or worsebe confused with another point in theimage of the next frame.

minDistThe minimum distance

between features selected.

27


28/34


Summary

Computer vision represents the software sensor of the future.

Computer vision trades unique hardware for software.

In some instances, computer vision can be considered a softwarescalable sensor. As the available CPU horsepower increases, the

capabilities of the technology increase. This class used OpenCV to demonstrate just a few algorithms

available in the OpenCV library.

OpenCV is a free computer vision library that has been downloadedover 3 million times.

This presentation covered only 4 of the over 2,000 algorithmsavailable in OpenCV.

28


29/34


To Probe Further

Visit the Embedded Vision Alliance web siteat www.Embedded-Vision.com

BDTI provides consulting services to companiesdeveloping and using vision technology:

Technology selection Product development engineering services

Competitive analysis

Visit us at www.BDTI.com

29
http://www.embedded-vision.com/http://www.bdti.com/http://www.bdti.com/http://www.embedded-vision.com/http://www.embedded-vision.com/http://www.embedded-vision.com/http://www.embedded-vision.com/


30/34


31/34


Selected Resources: The Embedded Vision Alliance

The VMware image used inthis presentation can bedownloaded at:www.embeddedvisionacademy.com/vmwareimage.

Simply install the free VMwareplayer and download the BDTIOpenCV VMware image from

www.embeddedvisionacademy.com/vmwareimage.

31
http://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImagehttp://embedded-vision.com/platinum-members/bdti/embedded-vision-training/downloads/pages/OpenCVVMwareImage


32/34


Selected Resources

OpenCV:

http://opencv.willowgarage.com/wiki/

Bradski and Kaehler, Learning OpenCV: Computer Vision withthe OpenCV Library, OReilly, 2008

Robert Laganire, OpenCV 2 Computer Vision ApplicationProgramming Cookbook, Packt, 2011

32
http://opencv.willowgarage.com/wiki/http://opencv.willowgarage.com/wiki/


33/34


Free VMware player: http://www.vmware.com/products/player/

Ubuntu 10.04 LTS: http://www.ubuntu.com/download/ubuntu/download

Eclipse CDT: http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1

33

Selected Resources
http://www.vmware.com/products/player/http://www.ubuntu.com/download/ubuntu/downloadhttp://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers-includes-incubating-components/indigosr1http://www.ubuntu.com/download/ubuntu/downloadhttp://www.ubuntu.com/download/ubuntu/downloadhttp://www.vmware.com/products/player/http://www.vmware.com/products/player/


34/34

ANALYSIS ADVICE ENGINEERING

Additional Resources

BDTIs web site, www.BDTI.com, provides a varietyof free information on processors used in visionapplications.

BDTIs free InsideDSP email newslettercovers tools, chips, and other technologies

for embedded vision and other DSPapplications. Sign up at www.BDTI.com.

The Embedded Vision Insights newslettershowcases tutorials, interviews, and other videos,along with technical articles, industry analysis reports,

news write-ups, and forum discussions that haverecently appeared on the Embedded VisionAllianceswebsite. Sign up at http://www.embedded-vision.com/user/register.
http://www.bdti.com/http://www.embedded-vision.com/user/registerhttp://www.embedded-vision.com/user/registerhttp://www.embedded-vision.com/user/registerhttp://www.embedded-vision.com/user/registerhttp://www.embedded-vision.com/user/registerhttp://www.embedded-vision.com/user/registerhttp://www.embedded-vision.com/user/registerhttp://www.bdti.com/

BDTI ESCSV 2012 Intro Computer Vision

Documents

Transcript of BDTI ESCSV 2012 Intro Computer Vision