Regionlets for Generic Object Detection -...
Transcript of Regionlets for Generic Object Detection -...
![Page 1: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/1.jpg)
Regionlets for Generic Object Detection
Xiaoyu Wang †
Miao Sun ‡
Yuanqing Lin †
Tony X. Han ‡ Shenghuo Zhu †
†
‡
A test on ImageNet
Tianbao Yang †
University of Missouri
![Page 2: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/2.jpg)
Introduction
12/14/2013 Regionlets for Generic Object Detection 2
Generic object detection is challenging
Rich deformation
Arbitrary scales
Arbitrary viewpoints
Limitations of current state of the art
Hand-crafted parameters to handle different degrees of deformation
Sub-optimal multiple scales/viewpoints handling
![Page 3: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/3.jpg)
A flexible and general object-level representation
Data-driven deformation handling
Multiple scales/viewpoints handling using a single and flexible model (Detecting an object at its original scale and aspect ratio)
Fast and easy to be extended with different features
Motivation
12/14/2013 Regionlets for Generic Object Detection 3
![Page 4: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/4.jpg)
Detection Framework3
12/14/2013 Regionlets for Generic Object Detection 4
2. K. E. A. Van de Sande, et. al. Segmentation as selective search for object recognition. ICCV 2011
1. B. Alexe , et. al. What is an object? CVPR 2010
3. X. Wang, et. al. Regionlets for Generic Object Detection. ICCV 2013
![Page 5: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/5.jpg)
Regionlet: Definition
12/14/2013 Regionlets for Generic Object Detection 5
Figure 1
Detection
bounding
box
Feature
extraction
Region
Regionlets
![Page 6: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/6.jpg)
Relative normalized position
Regionlet: Definition(cont.)
12/14/2013 Regionlets for Generic Object Detection 6
Traditional
Normalized
(50,50,180,180)
(.25, .25, .90,.90) Figure 2
![Page 7: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/7.jpg)
Regionlet: Feature extraction
12/14/2013 Regionlets for Generic Object Detection 7
Could be SIFT, HOG, LBP , Covariance features,
whatever feature your like!
Figure 3
Non-local
pooling
![Page 8: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/8.jpg)
Regionlets: Training
Constructing the regions/regionlets pool
Uniformly sample the position/configuration space of regions/regionlets
Learning realBoost1 cascades
16K region/regionlets candidates for each cascade
Learning of each cascade stops when the error rate is achieved (1% for positive, 37.5% for negative)
Last cascade stops after collecting 5000 weak classifiers
Result in 4-7 cascades
2-3 hours to finish training one category on a 8-core machine
12/14/2013 Regionlets for Generic Object Detection 8
1. C. Huang, et. al. Boosting nested cascade detector for multi-view face detection. ICPR, 2004.
![Page 9: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/9.jpg)
Two-layers deformation handling
Data-driven feature extraction region
Larger region -> more robust to deformation
Small region -> finer spatial layout
Data-driven non-local max-pooling over regionlets
Permutation invariance among regionlets
Exclusive feature representation among regionlets
Deformation Handling
12/14/2013 Regionlets for Generic Object Detection 9
![Page 10: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/10.jpg)
Arbitrary scale/viewpoints handling
Coordinates of regionlets are normalized in a model
Absolute regionlets coordinates are computed on the fly based on
The normalized coordinates
Resolution of the detection window
Scale/viewpoints Handling
12/14/2013 Regionlets for Generic Object Detection 10
Figure 4
![Page 11: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/11.jpg)
Experiments
Datasets
PASCAL VOC 2007, 2010
20 object categories
ImageNet Large Scale Object Detection Dataset
200 object categories
Investigated Features
HOG
LBP
Covariance
Deep Convolutional Neural Network (DCNN) feature
12/14/2013 Regionlets for Generic Object Detection 11
![Page 12: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/12.jpg)
Regionlets on PASCAL
12/14/2013 Regionlets for Generic Object Detection 12
Table 1. Performance on the PASCAL VOC 2007 dataset (Evaluated using Average
Precision or mean Average Precision: mAP, no DCNN feature, no outside data)
Table 2:
Performance
comparison with
state of the art
![Page 13: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/13.jpg)
Regionlets on PASCAL
Regionlets with Deep CNN feature (outside data)
12/14/2013 Regionlets for Generic Object Detection 13
Deep CNN convolutional layer feature (outside data)
CNN(ImageNet) + layer5 + SVM1 40.1%
CNN(ImageNet) + layer5 + Hand-crafted feature + Regionlets 49.3%
Deep CNN fine-tuned full connected layer feature (outside data)
CNN(fine-tuned on PASCAL) + FC7 + SVM1 48.0%
1. R Girshick, et. al. Rich feature hierarchies for accurate object detection and semantic segmentation. TR. 2013
Will Regionlets model perform at 49.3% + 7.9% = 57.2% using
fine-tuned full connected layer feature?
Table 3. Performance with Deep CNN feature
![Page 14: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/14.jpg)
Regionlets on ImageNet
ImageNet Challenge
12/14/2013 Regionlets for Generic Object Detection 14
Methods mAP
UvA-EuVision 22.6% (with DCNN feature)
Regionlets with deep features(1) 20.9% (with DCNN feature)
Regionlets without deep features 19.6% (no DCNN feature)
OverFeat-NYU 19.4% (DCNN)
Toronto A 11.2% (N/A)
SYSU_Vision 10.5% (N/A)
(1) It’s a preliminary result, we have a better performance now!
![Page 15: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/15.jpg)
Regionlets on ImageNet
Performance on the validation dataset
12/14/2013 Regionlets for Generic Object Detection 15
![Page 16: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/16.jpg)
Regionlets on ImageNet
Top 3 easiest categories: butterfly
12/14/2013 Regionlets for Generic Object Detection 16
![Page 17: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/17.jpg)
Regionlets on ImageNet
Top 3 easiest categories: Basketball
12/14/2013 Regionlets for Generic Object Detection 17
![Page 18: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/18.jpg)
Regionlets on ImageNet
Top 3 easiest categories: Dog
12/14/2013 Regionlets for Generic Object Detection 18
![Page 19: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/19.jpg)
Regionlets on ImageNet
Top 3 hardest categories: backpack
12/14/2013 Regionlets for Generic Object Detection 19
![Page 20: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/20.jpg)
Regionlets on ImageNet
Top 3 hardest categories: Spatula
12/14/2013 Regionlets for Generic Object Detection 20
![Page 21: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/21.jpg)
Regionlets on ImageNet
Top 3 hardest categories: Ladle
12/14/2013 Regionlets for Generic Object Detection 21
![Page 22: Regionlets for Generic Object Detection - ImageNetimage-net.org/challenges/LSVRC/2013/slides/ILSVRC201… · · 2014-02-09Regionlets for Generic Object Detection Xiaoyu †Wang](https://reader031.fdocuments.us/reader031/viewer/2022030600/5acdde357f8b9ab10a8e23e6/html5/thumbnails/22.jpg)
Conclusions
A new object representation for object detection
Non-local max-pooling of regionlets
Relative normalized locations of regionlets
Flexibility to incorporate various types of features
A principled data-driven detection framework, effective in handling deformation, multiple scales, multiple viewpoints
Superior performance with a fast running speed (.2 seconds per image)
12/14/2013 Regionlets for Generic Object Detection 22