CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper...
Transcript of CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper...
![Page 1: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/1.jpg)
CS381V Experiment Presentation
Chun-Chen Kuo
![Page 2: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/2.jpg)
The Paper• Indoor Segmentation and Support Inference from
RGBD Images. N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. ECCV 2012.
50 100 150 200 250 300 350 400 450 500 550
50
100
150
200
250
300
350
400
![Page 3: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/3.jpg)
Pipeline
segmentation support inference
![Page 4: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/4.jpg)
Outline
• Run the segmentation pipeline
• Experiment on the segmentation pipeline
• Run the support inference pipeline
• Address strength and weakness
![Page 5: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/5.jpg)
Outline
• Run the segmentation pipeline
• Experiment on the segmentation pipeline
• Run the support inference pipeline
• Address strength and weakness
![Page 6: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/6.jpg)
Segmentation Pipeline
Image920, RGB Depth Map
![Page 7: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/7.jpg)
Compute Surface Normal
y
x
z
![Page 8: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/8.jpg)
Align to room coordinates
![Page 9: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/9.jpg)
Aligned Surface Normal
y
x
z
![Page 10: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/10.jpg)
After Alignment
![Page 11: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/11.jpg)
Find Major Planes by RANSAC
0.2454x+0.1918y+0.9503z-4.2327
![Page 12: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/12.jpg)
Reassign Pixels to Planes
![Page 13: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/13.jpg)
Watershed Segmentation• Force the over-segmentation to be consistent with
the previous planes
1614
![Page 14: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/14.jpg)
Hierarchical Grouping • Bottom-up grouping by boundary classifier
(Logistic regression AdaBoost)
309 145 85
78
77
![Page 15: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/15.jpg)
AdaBoost Decision Tree
• Reweigh misclassified regions
• Optimize new tree with reweighed regions
• Score the tree
• Weighted sum over all trees optimized in each iteration
merge?
![Page 16: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/16.jpg)
Final Regions
77
Ground truth
![Page 17: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/17.jpg)
Outline
• Run the segmentation pipeline
• Experiment on the segmentation pipeline
• Run the support inference pipeline
• Address strength and weakness
![Page 18: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/18.jpg)
Experiment on Segmentation Pipeline
• NYU Depth Dataset V2
• Images 909~1200
• Assign pixels to major planes
• AdaBoost decision tree as boundary classifier
![Page 19: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/19.jpg)
Hypothesis• The trade-off between matching to 3D values,
normals, and gradient smoothing
• If alpha is small, neighbor pixels with similar RGB tend to be assigned to a same plane
• If alpha is large, match pixels to planes based on 3D points and normals, regardless gradient smoothing
![Page 20: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/20.jpg)
Result of Plane Labeling
alpha=0
![Page 21: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/21.jpg)
Result of Plane Labeling
alpha=2500
![Page 22: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/22.jpg)
Result of Plane Labeling
alpha=0.25
![Page 23: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/23.jpg)
Result of Plane Labeling
alpha=0.25alpha=0 alpha=2500
![Page 24: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/24.jpg)
Segmentation Scorealpha=0.25 alpha=2.5alpha=0.25e-12
![Page 25: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/25.jpg)
Hypothesis• Number of iteration of an AdaBoost decision forest
boundary classifier (underfit vs. overfit)
• At higher stage, the number of training example(boundary) decreases, causing lower accuracy and overfitting
• Accuracy at lower stage is more important because of error propagation
![Page 26: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/26.jpg)
stage 1 stage 2 stage 3
stage 4 stage 5
![Page 27: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/27.jpg)
ROC Curve at Stage 1• iteration = 30 • iteration = 5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.904903, Test AUC: 0.894981
trainingtesting
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.917977, Test AUC: 0.903215
trainingtesting
true
posi
tive
rate
false positive rate
![Page 28: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/28.jpg)
ROC Curve at Stage 2• iteration = 30 • iteration = 5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.796447, Test AUC: 0.780641
trainingtesting
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.816867, Test AUC: 0.777968
trainingtesting
![Page 29: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/29.jpg)
ROC Curve at Stage 3• iteration = 30 • iteration = 5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.762504, Test AUC: 0.746715
trainingtesting
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.802231, Test AUC: 0.737996
trainingtesting
![Page 30: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/30.jpg)
ROC Curve at Stage 4• iteration = 30 • iteration = 5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.727054, Test AUC: 0.718036
trainingtesting
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.773312, Test AUC: 0.718135
trainingtesting
![Page 31: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/31.jpg)
ROC Curve at Stage 5• iteration = 30 • iteration = 5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.727807, Test AUC: 0.720329
trainingtesting
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Train AUC: 0.774677, Test AUC: 0.713322
trainingtesting
![Page 32: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/32.jpg)
Accuracy versus Iteration at Stage 1
0 5 10 15 20 25 30iteration
0.83
0.84
0.85
0.86
0.87
0.88
0.89
0.9
0.91
0.92accuracy
trainingtesting
![Page 33: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/33.jpg)
Accuracy versus Iteration at Stage 2
0 5 10 15 20 25 30iteration
0.77
0.78
0.79
0.8
0.81
0.82
0.83
0.84
0.85accuracy
trainingtesting
![Page 34: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/34.jpg)
Accuracy versus Iteration at Stage 3
0 5 10 15 20 25 30iteration
0.73
0.74
0.75
0.76
0.77
0.78
0.79
0.8
0.81accuracy
trainingtesting
![Page 35: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/35.jpg)
Accuracy versus Iteration at Stage 4
0 5 10 15 20 25 30iteration
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78accuracy
trainingtesting
![Page 36: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/36.jpg)
Accuracy versus Iteration at Stage 5
0 5 10 15 20 25 30iteration
0.71
0.72
0.73
0.74
0.75
0.76
0.77
0.78accuracy
trainingtesting
![Page 37: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/37.jpg)
Segmentation Scoreiteration = [5 5 5 5 5]iteration = [10 10 10 10 10]iteration = [30 30 30 30 30]
Accuracy at lower stage is more important!
![Page 38: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/38.jpg)
Segmentation Scoreiteration = [1 1 1 1 1]
Accuracy at lower stage is more important!
0 5 10 15 20 25 30iteration
0.83
0.84
0.85
0.86
0.87
0.88
0.89
0.9
0.91
0.92
accuracy
trainingtesting
![Page 39: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/39.jpg)
Outline
• Run the segmentation pipeline
• Experiment on the segmentation pipeline
• Run the support inference pipeline
• Address strength and weakness
![Page 40: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/40.jpg)
Support Inference Pipeline
![Page 41: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/41.jpg)
Structure Class Classifier
![Page 42: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/42.jpg)
Structure Class Classifier
![Page 43: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/43.jpg)
Support Classifier
containment, geometry, and horz featuretake ~1 day to extract features for 292 images!
![Page 44: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/44.jpg)
Support Classifier
![Page 45: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/45.jpg)
Infer by Linear Program
6 minutes for an image!
![Page 46: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/46.jpg)
Structure and Support Inference
stripe: incorrect structure prediction
groundfurniturepropsstructure
50 100 150 200 250 300 350 400 450 500 550
50
100
150
200
250
300
350
400
![Page 47: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/47.jpg)
Structure and Support Inference
50 100 150 200 250 300 350 400 450 500 550
50
100
150
200
250
300
350
400
![Page 48: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/48.jpg)
50 100 150 200 250 300 350 400 450 500 550
50
100
150
200
250
300
350
400
Structure and Support Inference
over-segmentation(color variance in an object)
clutter, small objectsout of 4 classes
![Page 49: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/49.jpg)
Outline
• Run the segmentation pipeline
• Experiment on the segmentation pipeline
• Run the support inference pipeline
• Address strength and weakness
![Page 50: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/50.jpg)
Strength• Reason joint assignment for structure and support
• ~73% accuracy if ground truth segmentation is given
![Page 51: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/51.jpg)
Weakness• Slow in testing time
-5 minutes for feature extraction -6 minutes for inference by linear programming
• Clutters, small(thin) objects, color variance in objects
• Only 4 structure classes(no human, pet,…etc)
• ~55% accuracy if bottom up segmentation followed by support inference
![Page 52: CS381V Experiment Presentationvision.cs.utexas.edu/381V-spring2016/slides/kuo-expt.pdf · The Paper • Indoor Segmentation and Support Inference from RGBD Images. N. Silberman, D.](https://reader033.fdocuments.us/reader033/viewer/2022051812/60299321e3b4591db8055ebd/html5/thumbnails/52.jpg)
Reference
• Code:http://cs.nyu.edu/~silberman/projects/indoor_scene_seg_sup.html