Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan...
-
Upload
maude-hensley -
Category
Documents
-
view
216 -
download
0
Transcript of Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan...
![Page 1: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/1.jpg)
Minimal Surfaces for Stereo
Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan
MIT, HarvardMicrosoft Research, MIT
![Page 2: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/2.jpg)
Motivation
• Optimization based stereo over greed based– No early commitment– Enforce interactions: each pixel sees unique item– Penalize interactions: non-smoothness
![Page 3: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/3.jpg)
Stereo by Optimization
• Early algorithms: dynamic programming– (Baker ‘81, Belumeur & Mumford ‘92…)– Don’t generalize beyond 2 camera, single scanline
![Page 4: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/4.jpg)
Stereo by Optimization
• Recent Algorithms: iterative expansion– (… Kolmogorov & Zabih ‘01)– very general– NP-Complete
• Local opt found quickly in practice
• Recent algorithms: MIN-CUT– (Roy & Cox ‘96, Ishikawa & Geiger ‘98) – Polynomial time global optimum– New interpretation to such methods
![Page 5: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/5.jpg)
Contributions
• Stereo as a discrete minimal surface problem• Algorithms: Polynomial time globally optimal
surface– Using MIN-CUT (Sullivan ‘90)– Build from shortest path
• Applications to stereo vision– Rederive previous MIN-CUT stereo approaches– New 3-camera stereo formulation (Ayache ‘88)
![Page 6: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/6.jpg)
Planar Graph Shortest Path
• Given: an embedded planar graph– faces,
edges, vertices
![Page 7: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/7.jpg)
Planar Graph Shortest Path
• A non negative cost on each edge
57
![Page 8: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/8.jpg)
Planar Graph Shortest Path
• Two boundary points on the exterior of the complex
![Page 9: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/9.jpg)
Planar Graph Shortest Path
• Find minimal curve: (collection of edges) with given boundary
![Page 10: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/10.jpg)
Planar Graph For stereo
Camera Left Camera Right
Selected MatchSelected Occlusion
![Page 11: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/11.jpg)
Algorithms
• Classic: Dijkstra’s– Works even for non-planar graphs
• Wacky: use duality– But this will generalize to higher dimension
![Page 12: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/12.jpg)
Duality
![Page 13: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/13.jpg)
Duality
• face vertex • edge cross edge
- same cost57
![Page 14: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/14.jpg)
Duality
• Split exterior
![Page 15: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/15.jpg)
Source
Sink
Source
Duality
• Add source and sink
![Page 16: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/16.jpg)
Cuts
Source
Sink
• Cuts of dual graph = partitions of dual verts• Cost = sum of dual edges spanning the partition• MIN-CUT can be found in polynomial time
![Page 17: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/17.jpg)
Source
SinkCuts
• Claim: Primalization of MIN-CUT will be shortest path
![Page 18: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/18.jpg)
Sink
SourceSource
SinkWhy this works
• Cuts of dual graph = partitions of dual verts
![Page 19: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/19.jpg)
Sink
SourceSource
SinkWhy this works
• Partition of dual verts = partition of primal faces
![Page 20: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/20.jpg)
Sink
SourceSource
Sink
Source
SinkWhy this works
• Partition of primal faces = primal path
![Page 21: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/21.jpg)
Sink
SourceSource
Sink
Source
SinkWhy this works
• Cuts in dual correspond to paths in primal
• MIN-CUT in dual corresponds to shortest path in primal
![Page 22: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/22.jpg)
Same idea works for surfaces!
![Page 23: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/23.jpg)
Increasing the dimension
Planar graph: verts, edges, faces cost on edges boundary: 2 points on exterior sol: min path
Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface
![Page 24: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/24.jpg)
Increasing the dimension
Planar graph: verts, edges, faces boundary: 2 points on exterior sol: min path
Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface
![Page 25: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/25.jpg)
Increasing the dimension
Planar graph: verts, edges, faces boundary: 2 points on exterior sol: min path
Spacial compex: verts, edges, faces, cells cost on faces boundary: loop on exterior sol: min surface
![Page 26: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/26.jpg)
Dual construction for min surf
• face vertex • edge cross edge
Sink
Source
• cell vertex • face cross edge
MIN-CUT primalizes to min surf
![Page 27: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/27.jpg)
Checkpoint
• Solve for minimal paths and surfaces– MIN-CUT on dual graph
• Apply these algorithms to stereo vision
![Page 28: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/28.jpg)
Flatland Stereo
Camera Left Camera Right
Geometric interpretation of Cox et al. 96
pixel
![Page 29: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/29.jpg)
Flatland Stereo
Camera Left Camera Right
Geometric interpretation of Cox et al. 96
pixel
![Page 30: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/30.jpg)
Flatland Stereo
Camera Left Camera Right
Cost: unmatched/discontinuity, β
![Page 31: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/31.jpg)
Flatland Stereo
Camera Left Camera Right
Cost: correspondence quality
![Page 32: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/32.jpg)
Flatland Stereo
Camera Left Camera Right
![Page 33: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/33.jpg)
Camera Left Camera Right
MatchUnmatched
Uniqueness & monotonicity solution is directed pathFlatland Stereo
![Page 34: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/34.jpg)
Camera Left Camera Right
MatchOcclusion, discontinuity
Note: unmatched pixels also function as discontinuities
Flatland Stereo
![Page 35: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/35.jpg)
Flatland to Fatland
Camera Left Camera Right
![Page 36: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/36.jpg)
Flatland to Fatland
Camera Left Camera Right
![Page 37: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/37.jpg)
2 cameras, 3d
![Page 38: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/38.jpg)
2 cameras, 3d
![Page 39: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/39.jpg)
One Cuboid Among Many
Solve for minimal surface
![Page 40: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/40.jpg)
Geometric interpretation IG98
![Page 41: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/41.jpg)
Three Camera
Rectification (Ayache ‘88)
![Page 42: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/42.jpg)
Three Camera
![Page 43: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/43.jpg)
Three Camera
![Page 44: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/44.jpg)
Three Camera
![Page 45: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/45.jpg)
Three Camera
![Page 46: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/46.jpg)
One cuboid
![Page 47: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/47.jpg)
Dual graph of one cuboid
![Page 48: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/48.jpg)
One Cuboid Among Many
Solve for minimal surface
![Page 49: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/49.jpg)
More divisions of middle cell
![Page 50: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/50.jpg)
More expressive decomposition
![Page 51: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/51.jpg)
Complexity
• Vertices and edges: 20 n d– n: pixels per image– d: max disparity
• Time complexity O((nd)2 log(nd))
• About 1 min
![Page 52: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/52.jpg)
ResultsLL image
RC
KZ01
MS
![Page 53: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/53.jpg)
LL image
RC
KZ01
MS
![Page 54: Minimal Surfaces for Stereo Chris Buehler, Steven J. Gortler, Michael F. Cohen, Leonard McMillan MIT, Harvard Microsoft Research, MIT.](https://reader037.fdocuments.us/reader037/viewer/2022103004/56649cf65503460f949c5a4d/html5/thumbnails/54.jpg)
Future
• Application of MS to n cameras– Monotonicity/oriented manifold enforces more
than uniqueness– see Kolmogorov & Zabih (today 11:00am)
• Other applications of MS