Closing Remarks: What can we do with multiple diverse solutions?
description
Transcript of Closing Remarks: What can we do with multiple diverse solutions?
![Page 1: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/1.jpg)
CVPR 2013 Diversity Tutorial
Closing Remarks:What can we do with multiple
diverse solutions?
Dhruv Batra Virginia Tech
![Page 2: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/2.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 2
Example Result
Now what?
![Page 3: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/3.jpg)
CVPR 2013 Diversity Tutorial
Your Options• Nothing
– User in the loop
• (Approximate) Min Bayes Risk– Use solutions to estimate the distribution and optimize
Bayes Risk
• Re-ranking– Pick a good solution from the list
(C) Dhruv Batra 3
Increasing Side Information
![Page 4: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/4.jpg)
CVPR 2013 Diversity Tutorial
Interactive Segmentation• Setup
– Model: Color/Texture + Potts Grid CRF– Inference: Graph-cuts– Dataset: 50 train/val/test images
(C) Dhruv Batra 4
Image + Scribbles Diverse 2nd Best2nd Best MAPMAP
1-2 Nodes Flipped 100-500 Nodes Flipped
![Page 5: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/5.jpg)
CVPR 2013 Diversity Tutorial
Interactive Segmentation
(C) Dhruv Batra 5
MAP M-Best-MAP Confidence DivMBest89%
90%
91%
92%
93%
94%
95%
96%
+0.05%
+1.61%
+3.62%
(Oracle) (Oracle) (Oracle)
M=6
Seg
men
tatio
n A
ccur
acy
Better
![Page 6: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/6.jpg)
CVPR 2013 Diversity Tutorial
Your Options• Nothing
– User in the loop
• (Approximate) Min Bayes Risk– Use solutions to estimate the distribution and optimize
Bayes Risk
• Re-ranking– Pick a good solution from the list
(C) Dhruv Batra 6
![Page 7: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/7.jpg)
CVPR 2013 Diversity Tutorial
Statistics 101• Loss
– PCP, Pascal Loss, etc
• “True” Distribution
• Expected Loss:
• Min Bayes Risk
(C) Dhruv Batra 7
![Page 8: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/8.jpg)
CVPR 2013 Diversity Tutorial
Structured Output Problems• Min Bayes Risk
• Two Problems
• Approximate MBR:
(C) Dhruv Batra 8
IntractableIntractable
![Page 9: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/9.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation• Setup
– Models: • Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09]
• Second-Order Pooling [Carreira ECCV ‘12]
– Inference: • Alpha-expansion• Greedy
– Dataset: Pascal Segmentation Challenge (VOC 2012) • 20 categories + background; ~1500 train/val/test images
(C) Dhruv Batra 9
![Page 10: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/10.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 10
Large-Margin Re-ranking
![Page 11: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/11.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 11
Input MAP Best of 10-Div
![Page 12: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/12.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 12
PAC
AL
Acc
urac
y
Better
#Solutions / Image
1 2 3 4 5 6 7 8 9 1044%
47%
50%
53%
56%
59%
MAP[State-of-art circa 2012]
15%-gain possible
Same FeaturesSame Model
DivMBest (Oracle)
Rand (Re-rank)
MBR
![Page 13: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/13.jpg)
CVPR 2013 Diversity Tutorial
Your Options• Nothing
– User in the loop
• (Approximate) Min Bayes Risk– Use solutions to estimate the distribution and optimize
Bayes Risk
• Re-ranking– Pick a good solution from the list
(C) Dhruv Batra 13
![Page 14: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/14.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 14
Large-Margin Re-ranking
![Page 15: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/15.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 15
Large-Margin Re-ranking
![Page 16: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/16.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 16
Large-Margin Re-ranking
![Page 17: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/17.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 17
Large-Margin Re-ranking
Discriminative Re-ranking of Diverse Segmentation
[Yadollahpour et al., CVPR13, Wednesday Poster]
![Page 18: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/18.jpg)
CVPR 2013 Diversity Tutorial
Semantic Segmentation
(C) Dhruv Batra 18
PAC
AL
Acc
urac
y
Better
#Solutions / Image
1 2 3 4 5 6 7 8 9 1044%
47%
50%
53%
56%
59%
MAP[State-of-art circa 2012]
DivMBest (Oracle)
Rand (Re-rank)
DivMBest (Re-ranked) [Y.B.S., CVPR ‘13]
MBR
![Page 19: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/19.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 19
![Page 20: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/20.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 20
![Page 21: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/21.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Success
(C) Dhruv Batra 21
![Page 22: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/22.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 22
![Page 23: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/23.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 23
![Page 24: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/24.jpg)
CVPR 2013 Diversity Tutorial
Qualitative Results: Failures
(C) Dhruv Batra 24
![Page 25: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/25.jpg)
CVPR 2013 Diversity Tutorial
Summary• All models are wrong
• Some beliefs are useful
• Diverse Multiple Solutions– A way to get useful beliefs out.
• DivMBest + Reranking– Big impact possible on many applications!
(C) Dhruv Batra 25
![Page 26: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/26.jpg)
CVPR 2013 Diversity Tutorial
Summary
• What does my model believe?
(C) Dhruv Batra 26
Posterior Summary
![Page 27: Closing Remarks: What can we do with multiple diverse solutions?](https://reader036.fdocuments.us/reader036/viewer/2022062301/56816196550346895dd13f5a/html5/thumbnails/27.jpg)
CVPR 2013 Diversity Tutorial
Thanks!
(C) Dhruv Batra 27