Segmentation Through Optimization
description
Transcript of Segmentation Through Optimization
![Page 1: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/1.jpg)
Segmentation Through Optimization
Pyry Matikainen
![Page 2: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/2.jpg)
“He who fights with monsters should look to it that he himself does not become a monster.”
-Friedrich Nietzsche, Beyond Good and Evil
![Page 3: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/3.jpg)
Formulate Problem
Force problem into favorite
algorithm
“Refine”
Gradient ascent via parameter
tweaking
Publish
Retroacti
vely
justify decis
ions
![Page 4: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/4.jpg)
What is wrong with this?
• Difficult to use• Difficult to extend• Difficult to study
Formulate Problem
Force problem into favorite
algorithm
“Refine”
Gradient ascent via parameter
tweaking
Publish
Retroacti
vely
justify decis
ions
![Page 5: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/5.jpg)
Z. Tu and S. C. Zhu (2002)to the rescue!
and also Ren and Malik (2003)…
![Page 6: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/6.jpg)
Z. Tu and S. C. Zhu. Image Segmentation by Data-Driven Markov Chain Monte Carlo. PAMI, vol.24, no.5, pp. 657-673, May, 2002:
The DDMCMC paradigm combines and generalizes these [all other] segmentation methods in a principled way.
![Page 7: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/7.jpg)
Evaluator
Optimizer
Segmenter
![Page 8: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/8.jpg)
![Page 9: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/9.jpg)
*),(maxarg WIWf
)|( IWP
![Page 10: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/10.jpg)
Evaluator
Optimizer
![Page 11: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/11.jpg)
“What is a good segment?”Ren and Malik (2003)
![Page 12: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/12.jpg)
![Page 13: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/13.jpg)
How do we model a segment?
Raw pixel values
ContoursTexture
![Page 14: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/14.jpg)
)|()|()|( 21 WRpWRpWRp K
![Page 15: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/15.jpg)
G(x)
h(x)
h(f(x))
G(b(x) - x)
Rx
x2
![Page 16: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/16.jpg)
(gaussian) (histogram) (gabor) (Bezier)
![Page 17: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/17.jpg)
![Page 18: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/18.jpg)
Number of regionsRegion perimeter length (smoothness)
Region areaRegion appearance model complexity
Notably absent: the data
![Page 19: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/19.jpg)
![Page 20: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/20.jpg)
Superpixels(normalized cuts)
Oriented energy
Brightness
Texture(textons)
![Page 21: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/21.jpg)
Classifier
*
G(W|I)
![Page 22: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/22.jpg)
![Page 23: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/23.jpg)
![Page 24: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/24.jpg)
![Page 25: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/25.jpg)
Evaluator
Optimizer
![Page 26: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/26.jpg)
![Page 27: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/27.jpg)
MCMC is a technique for sampling from distributions.
![Page 28: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/28.jpg)
![Page 29: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/29.jpg)
Number of regions
Region
Region?
? ? ?
![Page 30: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/30.jpg)
Merge Split
Boundary competition
Switching image models
Model adaptation
Ren and Malik The ‘data driven’part revealed!
![Page 31: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/31.jpg)
![Page 32: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/32.jpg)
![Page 33: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/33.jpg)
![Page 34: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/34.jpg)
Data driven = do some clustering to make the MCMC faster.
![Page 35: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/35.jpg)
Evaluator
Optimizer
![Page 36: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/36.jpg)
Tu & Zhu
![Page 37: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/37.jpg)
![Page 38: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/38.jpg)
Ren & Malik
![Page 39: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/39.jpg)
![Page 40: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/40.jpg)
![Page 41: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/41.jpg)
Tu & Zhu Ren & Malik
New paradigm?
Combines and generalizes other techniques?
Principled?
Good results?
1/2 1/2
1/20
0 0
1 1/3
![Page 42: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/42.jpg)
Evaluator
Optimizer
OptimizerEvaluator
![Page 43: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/43.jpg)
(gaussian)
(mixture of gaussians)
(3x Bezier spline)
![Page 44: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/44.jpg)
(gaussian)
(g2)
(g3)
(g4)
(g1)
(histogram)
(gabor filter)
(Bezier spline)
![Page 45: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/45.jpg)
Number of regions
Pixels in region
Region appearance model
Region appearance model parameters
![Page 46: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/46.jpg)
MCMC
![Page 47: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/47.jpg)
Xiaofeng Ren and Jitendra Malik. Learning a Classification Model for Segmentation. ICCV 2003.
![Page 48: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/48.jpg)
Boundary between i and j
![Page 49: Segmentation Through Optimization](https://reader035.fdocuments.us/reader035/viewer/2022062518/56813f63550346895daa3599/html5/thumbnails/49.jpg)
Tu and Zhu 2002Sampling P(W|I)
Generative modelsPixels
Ren and Malik 2003Maximizing G(W|I)Discriminative modelsSuperpixels
Classification certainty