OpenCV Training courseBy Theerayod Wiangtong

GoalsGoals Develop a universal toolbox for research

and development in the field of Computer Vision

Why use OpenCV? Fast development time, more than 500

algorithms in OpenCV libraries C/C++ based programming Both Windows and Linux supported Open and free, BSD license Loads of developers using OpenCV Loads of information and documents Etc

History of OpenCV Originally developed by Intel, currently

maintained by Willow Garage

OpenCV - Features

Table Courtesy Learning OpenCV: Computer Vision with the OpenCV Library

5

Cross-platform and extremely portable Free! for both research and commercial use Targeted for real-time applications

OpenCV – Architecture & Modules

6

CvAux Area for experimental algorithms: e.g. HMM, Stereo

vision, 3D tracking, Bg/fg segmentation, camera calibration, Shape matching, Gesture recognition, ..

OpenCV Comparisons

Examples of Using OpenCV functions Click here

OpenCV: Algorithmic Content

OpenCV FunctionalityOpenCV FunctionalityBasic structures and operations Image AnalysisStructural AnalysisObject RecognitionMotion Analysis and Object Tracking3D Reconstruction

(more than 500 (more than 500 algorithms!!)algorithms!!)

Image ThresholdingImage Thresholding Fixed threshold; Adaptive

threshold;

StatisticsStatistics min, max, mean value, standard

deviation over the image Multidimensional histograms Norms C, L1, L2

Multidimensional HistogramsMultidimensional Histograms Histogram operations : calculation,

normalization, comparison, back project

Histogram Equalization

Histograms comparisonHistograms comparison

Image PyramidsImage Pyramids

Convolution in image The source pixel and its surrounding

pixels are all mathematically merged to produce a single destination pixel. The matrix slides across the surface of the source image, producing pixels for the destination image

http://beej.us/blog/data/convolution-image-processing/

Image PyramidsImage Pyramids

Gaussian and Laplacian

Morphological OperationsMorphological OperationsTwo basic morphology operations

using structuring element: erosion dilation

Distance TransformDistance Transform Calculate the distance for all non-feature points

to the closest feature point Two-pass algorithm, 3x3 and 5x5 masks, various

metrics predefined

Flood FillingFlood Filling

•grayscale image, floating range •grayscale image, fixed range

Feature DetectionFeature Detection

Fixed filters (Sobel operator, Canny operator, Laplacian, Scharr filter)

Hough transform (find lines and circles)

http://www.stevens-tech.edu/wireless/klin/EdgeDetection/EdgeDetectionInfo.htm

Edge detection operators Simple

Cross

2 -1

-1 0

This means: pixel(i,j) = 2*pixel(i,j) - pixel(i,j+1) - pixel(i+1,j).

1 0

0 -10 1

-1 0

Template 1: Template 2: pixel(i,j) = maximum(template 1, template 2)

Edge detection operators Prewitt

Sobel

1 0 -1

1 0 -1

1 0 -1

1 1 1

0 0 0

-1 -1 -1

X-axis Template: Y-axis Template:pixel(i,j) = sqrt((x-axis template)^2 + (y-axis template)^2)

1 0 -1

2 0 -2

1 0 -1

1 2 1

0 0 0

-1 -2 -1

Canny Edge DetectorCanny Edge Detector

http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html

Hough TransformHough Transform

Contour RetrievingContour Retrieving The contour representation:

Chain code (Freeman code) Polygonal representation

Initial Point

Chain code for the curve: 34445670007654443

Contour representation

Hierarchical representation of contours

Image Boundary

(W1) (W2) (W3)

(B2) (B3) (B4)

(W5) (W6)

Contours ExamplesContours Examples

Source Picture(300x600 = 180000 pts total)

Retrieved Contours (<1800 pts total)

After Approximation(<180 pts total)

And it is rather fast: ~70 FPS for 640x480 on complex scenes

Contour ProcessingContour Processing

Approximation: RLE algorithm (chain code) Teh-Chin approximation (polygonal) Douglas-Peucker approximation (polygonal);

Contour moments (central and normalized up to order 3) Matching of contours

Contours matchingContours matchingMatching based on hierarchical

representation of contours

Object Recognition: Eigen Object Recognition: Eigen ImageImage

One person – one HMMStage 1 – Train every HMM

Stage 2 – Recognition

Pi - probability

Choose max(Pi)

…1

n

i

Object Recognition: HMMObject Recognition: HMM

Motion Analysis and Object Motion Analysis and Object TrackingTracking

Background subtraction Motion templates Optical flow Active contours Estimators

Background SubtractionBackground Subtraction Background: any static or periodically moving

parts of a scene that remain static or periodic over the period of interest. How about waving trees, light on/off..?!?

Background statistics functions

Average Standard deviation Connect component

Background Subtraction Background Subtraction ExampleExample

Motion TemplatesMotion TemplatesObject silhouetteMotion history imagesMotion history gradientsMotion segmentation algorithm

silhouette MHI

MHG

Motion Templates ExampleMotion Templates Example

•Motion templates allow to retrieve the dynamic characteristics of the moving object

ObjectObject trackingtracking Mean-shift

Choose a search window (width and location) Compute the mean of the data in the search window Center the search window at the new mean location Repeat until convergence

Cam-shift: Continuously Adaptive Mean SHIFT

Region ofinterest

Center ofmass

Mean Shiftvector

Mean shift

Slide by Y. Ukrainitz & B. Sarel

Region ofinterest

Center ofmass

Mean Shiftvector

Mean shift

Slide by Y. Ukrainitz & B. Sarel

Region ofinterest

Center ofmass

Mean Shiftvector

Mean shift

Slide by Y. Ukrainitz & B. Sarel

Region ofinterest

Center ofmass

Mean Shiftvector

Mean shift

Slide by Y. Ukrainitz & B. Sarel

Region ofinterest

Center ofmass

Mean Shiftvector

Mean shift

Slide by Y. Ukrainitz & B. Sarel

Region ofinterest

Center ofmass

Mean Shiftvector

Mean shift

Slide by Y. Ukrainitz & B. Sarel

Region ofinterest

Center ofmassMean shift

Slide by Y. Ukrainitz & B. Sarel

Particle filter

ObjectObject trackingtracking

Optical flow is the relation of the motion field. It is a 2D projection of the physical movement of points relative to the observer

Optical flow, LK

)1( tI

Optical FlowOptical Flow

}{),( iptI

1p

2p

3p

4p

1v

2v

3v

4v

}{ iv

Velocity vectorsVelocity vectors

OpenCV shape classification OpenCV shape classification capabilitiescapabilitiesContour approximationMoments (image&contour)Convexity analysisPair-wise geometrical

histogramFitting functions (line, ellipse)

Using contours and geometry Using contours and geometry to classify shapesto classify shapesGiven the contour

classify the geometrical figure shape (triangle, circle, etc)

MomentsMoments

Contour moments (faster)Not applicable for different sizes,

orientationHu invariants

Here p is the x-order and q is the y-order, whereby order means the power to which the corresponding component is taken in the sum just displayed. E.g. m00 moment is actually just thelength in pixels of the contour.

Image segmentationImage segmentation Separate image into coherent “objects”

image human segmentation

Segmentation MethodsSegmentation MethodsEdge-based approach

Color segmentation: histogramCalculate the histogram. Find the objects of the selected histogram in the image.

Apply edge detector (sobel, laplace, canny, gradient strokes).Find connected components in an inverted image

OpenCV: Getting started

Getting StartedGetting Started

56

Download OpenCV http://opencv.willowgarage.com/wiki/

There exists a short walkthrough video on YouTube at http://www.youtube.com/watch?v=9nPpa_WiArI

Learning OpenCV: Computer Vision with the OpenCV Library by Gary Bradski and Adrian Kaehler http://proquest.safaribooksonline.com/9780596516130

OpenCV 2.1 with Visual Studio OpenCV 2.1 with Visual Studio 20082008

Download the OpenCV 2.1.0 Windows installer from SourceForge - "OpenCV-2.1.0-win32-vs2008.exe".

Install it to a folder (without any spaces in it), say "C:\OpenCV2.1\". This article will refer to this path as $openCVDir

During installation, enable the option "Add OpenCV to the system PATH for all users".

Configure Visual Studio Configure Visual Studio 20082008 Open VC++ Directories configuration: Tools >

Options > Projects and Solutions > VC++ Directories

Choose "Show directories for: Include files" • Add "$openCVDir\include\opencv"

Choose "Show directories for: Library files" • Add "$openCVDir\lib"

Choose "Show directories for: Source files" • Add "$openCVDir\src\cv" • Add "$openCVDir\src\cvaux" • Add "$openCVDir\src\cxcore" • Add "$openCVDir\src\highgui"

Configure your ProjectConfigure your Project Open Project Properties: Project > %projectName

% Properties... Open Linker Input properties: Configuration

Properties > Linker > Input Open the "..." window to edit "Additional

Dependencies" and on each line put: • "cv210.lib" • "cxcore210.lib" • "highgui210.lib" • And any other lib file, e.g, cvaux.lib, necessary for

your project Your project should now build. If you get any errors

try restarting Visual Studio and then doing a clean Rebuild.

More info

http://opencv.willowgarage.com/documentation/c/index.html http://dasl.mem.drexel.edu/~noahKuntz/openCVTut1.html http://sapachan.blogspot.com/search/label/Learning%20OpenCV http://www.shervinemami.co.cc/introToOpenCV.html http://note.sonots.com/OpenCV/Install.html

Questions