Generic Object Detection using Feature Maps

Generic Object Detection using Feature Maps

Oscar Danielsson ([email protected])Stefan Carlsson ([email protected])

Outline

Detect all Instances of an Object Class

The classifier needs to be fast (on average). This is typically accomplished by:1. Using image features that can be computed quickly2. Using a cascade of increasingly complex classifiers (Viola and Jones IJCV 2004)

Outline

Famous Object Detectors (1)

Dalal and Triggs (CVPR 05) use a dense Histogram of Oriented Gradients (HOG) representation - the window is tiled into (overlapping) sub-regions and gradient

orientation histograms from all sub-regions are concatenated. A linear SVM is used for classification.

€

H(I) = wT ⋅

Mhi1Mhin

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

M

⎡

⎣

⎢ ⎢ ⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥ ⎥ ⎥

+b

Famous Object Detectors (2)Felzenszwalb et. al. (PAMI 10, CVPR 10) extend the Dalal and Triggs model to include

high resolution parts with flexible location.

€

H(I) = wT ⋅

Mhi1Mhin

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

M

⎡

⎣

⎢ ⎢ ⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥ ⎥ ⎥

+b


€

pi f i I( ) < piθ i

Viola and Jones (IJCV 2004) construct a weak classifier by thresholding the response of a Haar filter (computed using integral images). Weak classifiers are combined into a

strong classifier using AdaBoost.

€

H(I) = α ii p i f i (I )< p iθ i{ }

∑

Outline

Motivation

Corners Corners + Blobs Regions Edges

Different object classes are characterized by different features. So we want to leave the choice of features up to the user.

Therefore we construct an object detector based on feature maps. Any feature detectors in any combination can be used to generate feature maps.

Our Object Detector

€

pidi I( ) < piθ i

We use AdaBoost to build a strong classifier. We construct a weak classifier by thresholding the distance from a measurement point to the closest occurrence of a

given feature.

€

H(I) = α ii p i d i (I )< p iθ i{ }

∑

Outline

Extraction of Training Data

€

f =d1

Mdn

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

1. Feature maps are extracted by some external feature detectors2. Distance transforms are computed for each feature map3. (For each training window) Distances from each measurement point to the closest

occurrence of the corresponding feature are concatenated into a vector

Training

Cascade

Strong Learner

Weak Learner

Decision Stump Learner

1. Require positive training examples and background images

2. Randomly sample background images to extract negative training examples

3. Loop:1. Train strong classifier2. Append strong classifier to

current cascade3. Run cascade on background

images to harvest false positives4. If number of false positives

sufficiently few, stop

{ fi }, { Ij }

{ fi }, { ci }, T

{ fi }, { ci }, { di }

{ fi }, { ci }, { di }

Viola-Jones Cascade Construction

Training

Cascade

Strong Learner

Weak Learner


1. Require labeled training examples and number of rounds

2. Init. weights of training examples 3. For each round

1. Train weak classifier2. Compute weight of weak

classifier3. Update weights of training

examples

{ fi }, { ci }, T

{ fi }, { Ij }

{ fi }, { ci }, { di }

{ fi }, { ci }, { di }

AdaBoost

Training

Cascade

Strong Learner

Weak Learner


1. Require labeled and weighted training examples

2. Compute node output3. Train decision stump4. Split training examples using decision

stump5. Evaluate stopping conditions6. Train decision tree on left subset of

training examples7. Train decision tree on right subset of

training examples

{ fi }, { ci }, { di }

{ fi }, { Ij }

{ fi }, { ci }, T

{ fi }, { ci }, { di }

Decision Tree Learner

Training

Cascade

Strong Learner

Weak Learner


1. Require labeled and weighted training examples

2. For each measurement point1. Compute a threshold by

assuming exponentially distributed distances

2. Compute classification error after split

3. If error lower than previous errors, store threshold and measurement point

{ fi }, { ci }, { di }

{ fi }, { Ij }

{ fi }, { ci }, T

{ fi }, { ci }, { di }

Feature and Threshold Selection

Outline

Hierarchical DetectionEvaluate an “optimistic” classifier on regions in search space. Split positive regions

recursively.

Hierarchical Detection

x

y

s

x

y

s

Search space Image space

Each point in search space corresponds to a window in the image. In the image is a measurement point.


x

y

s

x

ySearch space Image space

A region in search space corresponds to a set of windows in the image. This translates to a set of locations for the measurement point.


x

y

s

x

ySearch space Image space

We can then compute upper and lower bounds for the distance to the closest occurrence of the corresponding feature. Based on these bounds we construct an

optimistic classifier.

€

Mdmin

M

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥≤ f ≤

Mdmax

M

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

Outline

Experiments

• Detection results obtained on the ETHZ Shape Classes dataset, which was used for testing only

• Training data downloaded from Google images : 106 applelogos, 128 bottles, 270 giraffes, 233 mugs and 165 swans

• Detections counted as correct if Aintersect / Aunion ≥ 0.2

• Features used: edges, corners, blobs, Kadir-Brady + SIFT + quantization

ResultsReal AdaBoost slightly better than Dirscrete and Gentle AdaBoost

ResultsDecision tree weak classifiers should be shallow

ResultsUsing all features are better than using only edges

ResultsUsing the asymmetric weighting scheme of Viola and Jones yields a slight

improvement

ResultsApplelogos Bottles Mugs Swans

ResultsHierarchical search yields a significant speed-up

Outline

Conclusion• Proposed object detection scheme based on feature maps• Used distances from measurement points to nearest

feature occurrence in image to construct weak classifiers for boosting

• Showed promising detection performance on the ETHZ Shape Classes dataset

• Showed that a hierarchical detection scheme can yield significant speed-ups

– Thanks for listening!


€

piwiTfi I( ) < piθ i

Laptev (IVC 09) construct a weak classifier using a linear discriminant on a histogram of oriented gradients (HOG – computed by integral histograms) from a sub-region of the window. Again, weak classifiers are combined into a strong classifier using AdaBoost.

€

H(I) = α ii p iw i

Tf i (I )< p iθ i{ }

∑

Generic Object Detection using Feature Maps

Documents

Transcript of Generic Object Detection using Feature Maps