Enhancing Exemplar SVMs using Part Level Transfer Regularization
description
Transcript of Enhancing Exemplar SVMs using Part Level Transfer Regularization
Enhancing Exemplar SVMs usingPart Level Transfer Regularization
1
Problem Definition:
Image Retrieval
2
Problem Definition:
Image Retrieval
query
3
Problem Definition:
Image Retrieval
query
Image Database
Retrieved Images
query Retrieved Images
Retrieving same category in a similar pose
Example: bicycle facing left
4
A Candidate Solution:
Exemplar SVM (E-SVM)
Training a SVM with a single positive and many negative samples
Linear SVMsover
HoG features
[Dalal &Triggs’05], [Felzenszwalb’08] Exemplar SVM
5
[Malisiewicz’11][Shrivastava’11]
A Candidate Solution:
Exemplar SVM (E-SVM)
Linear SVMsover
HoG features
[Dalal &Triggs’05], [Felzenszwalb’08] Exemplar SVM
Training a SVM with a single positive and many negative samples
Retrieval via sliding window search on the image database
Imag
e D
atab
ase
6
A Candidate Solution:
Exemplar SVM (E-SVM)
Retrieval via sliding window search on the image database
Imag
e D
atab
ase
7
Linear SVMsover
HoG features
[Dalal &Triggs’05], [Felzenszwalb’08] Exemplar SVM
Training a SVM with a single positive and many negative samples
Retrieved Images
Framework:
Enhanced Exemplar SVM (EE-SVM)positive sample
negative samples
Train E-SVMover
HoG features
Part-LevelTransfer
Enhanced E-SVM
Exemplar SVM Previously Trained Classifiers
8
Benefit:
Enhanced Exemplar SVM (EE-SVM)Exemplar SVM
Retrieved Subwindows
Enhanced E-SVM
SubwindowRetrieval
SubwindowRetrieval
Image Database
Retrieved Subwindows
9
Que
ry Im
age
Overview
• Transfer Learning in Computer Vision– Classification & Detection
• Enhanced Exemplar SVM
• Feature Augmentation vs Transfer
• Results & Discussion10
Transfer Learning in Computer Vision
• Image Classification– Adaptive SVMs, – Transfer from Multiple Models, – Adaptive Multiple Kernel Learning
• Object Detection– Rigid Transfer– Flexible Transfer
13
[Yang et al. ICDM’07][Tommasi et al. BMVC’09] [Tommasi et al. CVPR’10] [Luo et al. ICCV’11][Duan et al. CVPR’10]
[Stark et al. ICCV’09][Aytar and Zisserman ICCV’11][Gao et al. ECCV’12]
Learning new classes by building upon previously learned classes.
Transfer Learning for Detection
• Rigid Transfer [Aytar and Zisserman ICCV’11]– Transfer between fixed sized templates– Good performance, especially for smaller number of training samples.– Hard to find visually similar detectors with same aspect ratio and size.
• Flexible Transfer – Transfer between different sized templates.– Transferring shape features [Stark et al. ICCV’09]– Deformable Transfer [Aytar and Zisserman ICCV’11] – Transfer via Structured Priors [Gao et al. ECCV’12]
Fixed SizedTransfer
14
FlexibleTransfer
Overview
• Transfer Learning in Computer Vision– Classification & Detection
• Enhanced Exemplar SVM
• Feature Augmentation vs Transfer
• Results & Discussion15
Framework:
Enhanced Exemplar SVM (EE-SVM)
Train E-SVM
Part-LevelTransfer
Enhanced E-SVM Exemplar SVM
Prev
ious
ly T
rain
ed
Cla
ssifi
ers
16
Query
Framework:
Part-Level Transfer Regularization
17Exemplar SVMui
ui
Parameters:
Part-Level Transfer Regularization
18
close to E-SVMclose to construction from ui’s
Framework:
Matching Classifier Patches
19
Previously Learned Classifiers ui
Exemplar SVM
Why is it beneficial?
Part-Level Transfer Regularization• Part level transfer is beneficial because…
– parts can be relocated (deformation), – the possibility of finding a good match for transfer increases when we
look at smaller classifier patches.
• Advantages of transferring parts from well trained classifiers:– Better background suppression and discriminativity due to well trained
source classifiers.– Better handling of local variations since source classifiers are trained
on many positive samples.
• No additional cost on runtime
20
• Unusual Poses
• Composition of Objects [Visual Phrases - Sadeghi CVPR’11]
21
Where is it beneficial?
Part-Level Transfer Regularization
PASCAL 2007:
Results - Left Facing Horse
22
query
E-SVM
Enhanced E-SVM
PASCAL 2007:
Results - Left Facing Bicycle
23
E-SVM
Enhanced E-SVM
query
PASCAL 2007:
Visual Phrase – Riding Horse
24
query
E-SVM
Enhanced E-SVM
ImageNet:
Unusual Pose - Bicycle
25
E-SVM
Enhanced E-SVM
query
Overview
• Transfer Learning in Computer Vision– Classification & Detection
• Enhanced Exemplar SVM
• Feature Augmentation vs Transfer
• Results & Discussion27
. . . .
Implementation:
Transfer vs. Feature Augmentation
29
0.2 0.7 0.1 . . .
is equivalent to learning
Transfer Regularization
“normal” SVM with augmented features.
Implications:
Transfer vs. Feature Augmentation
• This equivalence is not specific to Exemplar SVMs.
• Transfer regularization can be implemented as feature augmentation.
• Transfer regularization can be efficiently solved using standard SVM packages.
30
Overview
• Transfer Learning in Computer Vision– Classification & Detection
• Enhanced Exemplar SVM
• Feature Augmentation vs Transfer
• Results & Discussion31
PASCAL 2007:
Quantitative Results
32
ImageNet:
Quantitative Results
• Three queries are evaluated for each of the five classes.• Precisions at top 5, 10, 50 and 100 are reported.
33
Handling Occlusions
34
Que
ryE-
SVM
EE-S
VM
Que
ryE-
SVM
EE-S
VM
Handling Truncation
35
36
Conclusions
• Boosted the performance of E-SVM which incurs no additional cost on runtime.
• Presented the equivalence between Transfer regularization and feature augmentation.
• Showed the benefit for unusual poses and visual phrases.
• Handling truncation and occlusion.