Category Independent Region Proposals
description
Transcript of Category Independent Region Proposals
![Page 1: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/1.jpg)
Category Independent Region Proposals
Ian Endres and Derek HoiemUniversity of Illinois at Urbana-Champaign
![Page 2: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/2.jpg)
Finding Objects
![Page 3: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/3.jpg)
Scanning Window
HorseDogCatCarTrain… 10,000+ windows
![Page 4: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/4.jpg)
Category Independent Search
~100 regions
![Page 5: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/5.jpg)
Finding Unfamiliar Objects
![Page 6: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/6.jpg)
Finding Objects
Objectives:1. Minimize number of proposed regions2. Maintain high recall of all objects3. Provide detailed spatial support (i.e. segmentation)
![Page 7: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/7.jpg)
Challenges
• Objects extremely diverse– Variety of shapes, sizes– Many different appearances
• Within object variation– Multiple materials and textures– Strong interior boundaries
• Many objects in an image
![Page 8: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/8.jpg)
Overview
1 2 3 4 ...
Generate Proposals:Maximize recall
Rank Proposals:Small diverse set of object regions
![Page 9: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/9.jpg)
Generating Proposals1. Select Seed 2. Compute affinities for seed
3. Construct binary CRF
+
Unary term:Affinities
Pairwise term:Occlusion Boundaries
4. Compute proposal
5. Change parametersRepeat
![Page 10: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/10.jpg)
Generating Seeds
• Compute occlusion boundaries (Hoiem et al. ICCV ‘07)
• Generate hierarchal segmentation– Incrementally merge regions of oversegmentation
• Use regions with sufficient size and boundary strength– Avoids redundant or uninformative seeds
![Page 11: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/11.jpg)
Region Affinity
• Learned from pairs of regions belonging to an object– Computed between the seed and each region of
the hierarchy
– Features: color and texture similarity, boundary crossings, layout agreement
![Page 12: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/12.jpg)
Color/Texture Similarity•Color, texture histograms for each region•Compute histogram intersection distance between two regions
![Page 13: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/13.jpg)
Boundary Crossing•Draw line between region centers of mass
•Compute strength of occlusion boundaries crossed
![Page 14: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/14.jpg)
Layout Agreement•Predict object extent from each region
•Compute strength of agreement between two regions
![Page 15: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/15.jpg)
CRF Segmentation
• Binary segmentation• Graph composition:– Nodes: Superpixels– Edges: Adjacent superpixels
+
![Page 16: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/16.jpg)
CRF Segmentation
• Graph Potentials– Unary Potential: affinity values for each superpixel– Edge Potential: occlusion boundary strength
• Parameters (25 combinations)– Node/Edge weight tradeoff– Node bias
+
Unary potential:Affinities
Edge potential:Occlusion Boundaries
![Page 17: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/17.jpg)
Ranking Proposals
wT X1
wT X3
Appearance scores
wT X4
1.
2.
3.
4.
wT X2Sort
scores
GeneratedRanking
![Page 18: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/18.jpg)
Lacks Diversity
• But in an image with many objects, one object may dominate 1
2
3
4
…
20
150
100
…
50…
…
![Page 19: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/19.jpg)
Encouraging Diversity
• Suppress regions with high overlap with previous proposals
…
1
2
3
10
4
…
20
50
100
…
…
![Page 20: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/20.jpg)
Ranking as Structured Prediction
• Find the max scoring ordering of proposals
• Greedily add proposals with best overall score
Appearance score
Overlap penalty
Gives higher weight to higher ranked proposals
Overall score
![Page 21: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/21.jpg)
Learning to Rank(Max-margin Structured Learning)
• Score of ground truth ordering (R(n)) should be greater than all other orderings (R):
• Loss ( ) encourages good orderings:– Higher quality proposals should have higher rank– Each object should have a highly ranked proposal
![Page 22: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/22.jpg)
Experimental Setup• Train on 200 BSDS images
• Test 1: 100 BSDS images
• Test 2: 512 Images from Pascal 2008 Seg. Val.
![Page 23: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/23.jpg)
Evaluation
• Region overlap
• Recall at 50% region overlap– Typically more strict that 50% bounding box overlap– Measures detection quality and segment quality
Ai Aj
![Page 24: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/24.jpg)
Qualitative Results
Pascal
BSDS(Rank, % overlap)
![Page 25: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/25.jpg)
Vs. Standard Segmentation
Standard: 53%3000 proposals
Ours: 53%18 proposals
Standard: 80%70,000 proposals
(merge 2 adjacent regions)
Ours: 80%180 proposals
![Page 26: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/26.jpg)
Recalling Pascal Categories
![Page 27: Category Independent Region Proposals](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815205550346895dc048db/html5/thumbnails/27.jpg)
Future work
• Object Discovery• Incorporate into detection systems– Label regions directly– Voting from proposed regions
• Refine proposals with domain knowledge– i.e. wheel or head models