Everingham Family Tree - Overtown Miscellany (Overtown.org ...
Deep condolence to Professor Mark Everingham
Transcript of Deep condolence to Professor Mark Everingham
![Page 1: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/1.jpg)
Deep condolence to Professor Mark Everingham
![Page 2: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/2.jpg)
Towards VOC2012 Object Classification Challenge
National University of Singapore Learning & Vision Group
Panasonic Singapore Laboratories Media Processing Group
Shuicheng YAN Zhongyang HUANG
Jian DONG, Qiang CHEN, Zheng SONG, Yan PAN, Wei XIA
Yang HUA, Shengmei SHEN
Generalized Hierarchical Matching for Sub-category Aware Object Classification
![Page 3: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/3.jpg)
Visual Features
Feature Pooling SPM
Local Feature Extraction
Feature Coding VQ
Framework – NUS_PSL_2010
Classification
SVM
Regression
Post Processing
Kernel Regression
Confidence Refinement
with Exclusive prior
Detection Results
Max pooling
Kernel
Nonlinear Kernel
Linear Kernel
Chair
![Page 4: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/4.jpg)
Feature Pooling SPM
Feature Pooling SPM, GHM
Feature Coding VQ
Feature Coding VQ, LLC, FK
Chair
Detection Results Subcategory
Detection Results
Max pooling
Framework – NUS_PSL_2012
Visual Features
Classification
SVM
Regression
Post Processing
Kernel Regression
Confidence Refinement
with Exclusive prior
Local Feature Extraction
Kernel
Nonlinear Kernel
Linear Kernel
Nonlinear + Linear Kernel
Subcategory Mining
Flipping
Flipping
Flipping
![Page 5: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/5.jpg)
Feature Pooling SPM
Feature Pooling SPM, GHM
Feature Coding VQ
Feature Coding VQ, LLC, FK
Chair
Detection Results Subcategory
Detection Results
Max pooling
Framework – NUS_PSL_2012
Visual Features
Classification
SVM
Regression
Post Processing
Kernel Regression
Confidence Refinement
with Exclusive prior
Local Feature Extraction
Kernel
Nonlinear Kernel
Linear Kernel
Nonlinear + Linear Kernel
Subcategory Mining
Flipping
Flipping
Flipping
Ⅰ Generalized
Hierarchical Matching
Ⅱ Subcategory
Mining
![Page 6: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/6.jpg)
Ⅰ Generalized Hierarchical Matching
Traditional Pooling: SPM = approximate geometric constraint
Not optimal for object recognition due to misalignment
(a) Images (b) SPM partitions (c) Object Confidence Map partition
![Page 7: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/7.jpg)
Ⅰ Generalized Hierarchical Matching
Encoded local feature vs. side information
(b) Hierarchically cluster by side information. Level 1 (top),2 (mid),3 (bottom)
(a) Side information and Image
(c) Hierarchical structure representation
(d) Matching/pooling within each cluster
Utilize side information to hierarchically pool local features
![Page 8: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/8.jpg)
Ⅰ Generalized Hierarchical Matching
Fusing
Images
Object Confidence Maps
sub-window Sliding
window
Score vote back to image
Shape Model
Appearance Model
Process Score vote back to image
Side Information - Detection Confidence Map
![Page 9: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/9.jpg)
Ⅱ Sub-Category Mining
Intra-class diversity:
Foreground distribution is diverse due to appearance, occlusion variance
Aspect ratio is not enough to grasp these types of variance
![Page 10: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/10.jpg)
Ⅱ Sub-Category Mining
Intra-class diversity:
Foreground distribution is diverse due to appearance, occlusion variance
Aspect ratio is not enough to grasp these types of variance
Subcategory awareness is necessary !
![Page 11: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/11.jpg)
Ⅱ Sub-Category Mining
Inter-class ambiguity:
Chairs are ambiguous with sofas
![Page 12: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/12.jpg)
Ⅱ Sub-Category Mining
Inter-class ambiguity: Some sub-categories may be ambiguous with certain specific object
categories
Chair Sofa
Chair Diningtable
![Page 13: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/13.jpg)
Ⅱ Sub-Category Mining
Inter-class ambiguity: Some sub-categories may be ambiguous with certain specific object
categories
Chair Sofa
Chair Diningtable
Solution: Ambiguity guided subcategory mining
![Page 14: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/14.jpg)
Calculate the sample intra-class similarity
Calculate the sample inter-class ambiguity
Detect dense subgraphs by graph shift algorithm [1]
Subgraphs to subcategories.
Ⅱ Sub-Category Mining
Subcategory Mining based on both Similarity & Ambiguity
Chair
Sofa Ambiguous Categories
[1] Hairong Liu, Shuicheng Yan. Robust Graph Mode Seeking by Graph Shift. ICML 2010
Ambiguity
Similarity
![Page 15: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/15.jpg)
Sub-Category Aware Detection & Classification
Visual Feature Subcategory 1
Result Detection
Score
Chair
Context Feature
Subcategory N Result
Category Level
Result
Detection Model Classification Model Subcategory 1
Training Phase
Testing Phase
Contextualized
![Page 16: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/16.jpg)
The results 2010 2011 2012
Our Best Other's Best Our Best Other's Best Our Best Other's Best
aeroplane 93 93.3 95.5 94.5 97.3 92
bicycle 79 77 81.1 82.6 84.2 74.2
bird 71.6 69.9 79.4 79.4 80.8 73
boat 77.8 77.2 82.5 80.7 85.3 77.5
bottle 54.3 53.7 58.2 57.8 60.8 54.3
bus 85.2 85.9 87.7 87.8 89.9 85.2
car 78.6 80.4 84.1 85.5 86.8 81.9
cat 78.8 79.4 83.1 83.9 89.3 76.4
chair 64.5 62.9 68.5 66.6 75.4 65.2
cow 64 66.2 74.7 74.2 77.8 63.2
diningtable 62.9 61.1 68.5 69.4 75.1 68.5
dog 69.6 71.1 76.4 75.2 83 68.9
horse 82 76.7 83.3 83 87.5 78.2
motorbike 84.4 81.7 87.5 88.1 90.1 81
person 91.6 90.2 92.8 93.5 95 91.6
pottedplant 48.6 53.3 56.5 58.7 57.8 55.9
sheep 65.4 66.3 77.7 75.5 79.2 69.4
sofa 59.6 58 67 66.3 73.4 65.4
train 89.4 87.5 91.2 90 94.5 86.7
tvmonitor 77.2 76.2 77.5 77.2 80.7 77.4
MAP 73.8 78.7 82.2
![Page 17: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/17.jpg)
Discussions
Classification, detection and segmentation are essentially closely related problems. It is predictable that these three problems shall be explored within a unified framework in the near future!
Effectiveness seems fine now, how about efficiency?
![Page 18: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/18.jpg)
Acknowledgement
We would thank Mr. Tsutomu MURAJI, Mr. Keisuke MATSUO, Mr. Ryouichi KAWANISHI from Panasonic Corporation for their support to this collaboration project.
![Page 19: Deep condolence to Professor Mark Everingham](https://reader030.fdocuments.us/reader030/viewer/2022012617/619efbcec332a4084e095493/html5/thumbnails/19.jpg)