Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh...
-
Upload
christopher-thompson -
Category
Documents
-
view
218 -
download
0
Transcript of Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh...
![Page 1: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/1.jpg)
Discontinuity Preserving Stereo with Small Baseline
Multi-Flash Illumination
Rogerio Feris1, Ramesh Raskar2, Longbin Chen1, Karhan Tan3 and Matthew Turk1
1University of California, Santa Barbara2Mitsubishi Electric Research Labs
3Epson Palo Alto Lab
![Page 2: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/2.jpg)
Introduction
Correspondence Problem
Stereo Near Depth Discontinuities:
- Occlusion Problem
- Perspective Distortions
- Violation of Smoothness Constraints
Passive Versus Active Methods
![Page 3: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/3.jpg)
Introduction
Our Approach:
Small Baseline Multi-Flash Illumination
- Simple, Inexpensive
- Compact, Self-Contained
- Discontinuity Preserving
![Page 4: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/4.jpg)
Depth Edges with Multi-Flash
Raskar, Tan, Feris, Yu, Turk – ACM SIGGRAPH 2004
![Page 5: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/5.jpg)
![Page 6: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/6.jpg)
![Page 7: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/7.jpg)
![Page 8: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/8.jpg)
![Page 9: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/9.jpg)
Bottom Flash Top Flash Left Flash Right Flash
Ratio images and directions of epipolar traversal
Shadow-Free
Depth Edges
Shadow-Free Depth Edges
![Page 10: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/10.jpg)
Qualitative Depth Map
![Page 11: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/11.jpg)
Qualitative Depth
Sign of Depth Edge
- Indicates which side is the foreground and which side is the background
Shadow Width
- Encodes object relative distances
![Page 12: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/12.jpg)
Sign of Depth Edge
+ -+-
(+) Foreground (-) Background
Original Ratio Left Ratio Right Signed Edges
![Page 13: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/13.jpg)
Shadow Width Bottom Flash Image Ratio Image
Plot Along Scanline
![Page 14: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/14.jpg)
Shadow Width Bottom Flash Image Ratio Image
Shadow Width Estimation:
Meanshift Segmentation algorithm applied on the ratio image
![Page 15: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/15.jpg)
Imaging Geometry
Object
Flash
Shadow
CameraB
z1
z2
f
21
12 )(
zz
zzfBd
Shadow Width
d
![Page 16: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/16.jpg)
Qualitative Depth
Working on this Equation …
)log()log()1log(
)11log()1log(
1
12
1
22
1
22
zzd
z
z
fB
dz
z
z
fB
dz
Log Depth DifferenceShadow Width
Gradient-Domain
Problem!
![Page 17: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/17.jpg)
Qualitative Depth
1) Compute Sharp Depth Gradient G = (Gh,Gv)
otherwise ),()1log(
edgedepth anot is y)(x, if 0),(
yxsdyxG
hhh
Log Depth Difference Sign of depth edge
2) Compute Q’ by integrating G (Poisson Equation)3) Qualitative depth map Q = exp(Q’)
![Page 18: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/18.jpg)
Qualitative Depth
Useful Prior Information for Stereo !
![Page 19: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/19.jpg)
Occlusion Map
![Page 20: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/20.jpg)
Partial Occlusion Problem
Object
Camera
Occlusion
A B
(Seen by A but not by B)
![Page 21: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/21.jpg)
Occlusion Bounded by Shadows
Object
CameraA B
Flash
Occlusion (Seen by A but not by B)
![Page 22: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/22.jpg)
Occlusion Bounded by Shadows
Object
CameraA B
Flash
Lower Bound Shadow
![Page 23: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/23.jpg)
Occlusion Bounded by Shadows
Object
CameraA B
Flash
Upper Bound Shadow
![Page 24: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/24.jpg)
Occlusion Bounded by Shadows
Object
Camera
Occlusion
A B
Average of Upper/Lower Shadow widths
Flash
![Page 25: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/25.jpg)
Occlusion Bounded by Shadows
Occlusion Map
Left View Right View
![Page 26: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/26.jpg)
Discontinuity Preserving Stereo
Matching
![Page 27: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/27.jpg)
Local Stereo
Problem: Shape and size of correlation window
- Small Window Ambiguities / Noise
- Large Window Problems at Depth Discontinuities
Depth Edge Preserving Local Stereo
Object Boundary (Depth Edge)
Correlation Window
![Page 28: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/28.jpg)
Local Stereo
Smooth Disparity
Delimited by depth edges + Occlusions
Correlation Window
Problem: Shape and size of correlation window
- Small Window Ambiguities / Noise
- Large Window Problems at Depth Discontinuities
Depth Edge Preserving Local Stereo
![Page 29: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/29.jpg)
Local Stereo
Left View Depth Edges + Occlusion Ground Truth
Challenging Scene:
- Ambiguous patterns, textureless regions, geometrically complex object, thin structures
![Page 30: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/30.jpg)
Local StereoConventional 9x9
Conventional 31x31
Our Approach 31x31--- Conventional
Stereo Our Approach
![Page 31: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/31.jpg)
Global Stereo
Global Optimization – Markov Random Field (MAP-MRF)
X = {xs} Disparity of each pixel (Hidden)
Y = {ys} Matching cost at each disparity (Observed)
X3 X1 X2 X7
X4 X6
X5 X8
y1 y2
![Page 32: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/32.jpg)
Global Stereo
Global Optimization – Markov Random Field (MAP-MRF)
s s sNt
tsstsss xxyxYXP)(
),(),()|(
X = {xs} Disparity of each pixel (Hidden)
Y = {ys} Matching cost at each disparity (Observed)
Data Term
Smoothness Term
Inference by Belief Propagation [Jian Sun et al, 2003]
![Page 33: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/33.jpg)
Global Stereo
Qualitative Depth Map as Evidence
- Used to set the smoothness term
- Information propagation is stopped at depth edges
- Encourage disparities for neighboring pixels according to depth difference in qualitative map
Occlusion Penalty
![Page 34: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/34.jpg)
Global Stereo
Conventional Belief Propagation Our Approach
RMS: 0.9589 RMS: 0.4590
![Page 35: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/35.jpg)
Conclusions
Contributions
- Stereo with small baseline illumination
- Useful Feature Maps (Qualitative Depth + Occlusion Map)
- Enhanced Local and Global Stereo Algorithms
Pros / Cons
- Robust, Simple, Inexpensive and Compact
- Limited to handle outdoor scenes and motion
Website (datasets, source code)
- http://www.cs.ucsb.edu/~rferis/multi-flash-stereo
![Page 36: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/36.jpg)
Thank you !
Multi-Flash Stereo Webpage
http://www.cs.ucsb.edu/~rferis/multi-flash-stereo
Four Eyes Lab, UCSB
http://ilab.cs.ucsb.edu
![Page 37: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/37.jpg)
![Page 38: Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew.](https://reader035.fdocuments.us/reader035/viewer/2022062618/551469fc550346414e8b5d3a/html5/thumbnails/38.jpg)
Occlusion Bounded by Shadows
Occlusion Detection by averaging length of shadows
Images taken with light sources surrounding the other camera