Image-Based Visual Hulls
description
Transcript of Image-Based Visual Hulls
![Page 1: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/1.jpg)
1
Image-Based Visual Hulls
Paper by Wojciech Matusik, Chris Buehler, Ramesh Raskar,
Steven J. Gortler and Leonard McMillan[http://graphics.lcs.mit.edu/~wojciech/vh/]
Vortrag von Simon DellenbachGDV Fachseminar 2001
![Page 2: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/2.jpg)
2
Overview (1)
• Motivation• Basics
– Viewpoint Model– Visual Hull– Epipolar Geometry
• Creating Image-Based Visual Hulls
![Page 3: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/3.jpg)
3
Overview (2)
• Rendering IBVH• System Implementation• Summary & Results• Future Work• Personal Opinion
![Page 4: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/4.jpg)
4
Motivation (1)
• Traditional computer graphics, rendering..– static synthetic scenes (CG Images)– dynamic synthetic scenes (CG Animations)– static acquired scenes (Image-Based Rendering)
• Acquire and render dynamic scenes in real-time:– appropriate representation– rendering system
![Page 5: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/5.jpg)
5
Motivation (2)
![Page 6: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/6.jpg)
6
Viewpoint Model - Basics (1)
![Page 7: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/7.jpg)
7
Visual Hull - Basics (2)
• Geometric shape obtained using silhouettes of object seen from number of views:– extruded silhouette = cone-like volume limiting
the extent of object– intersection of volumes results in a visual hull– more views better approximation of object– limitation: concavities can’t be captured
(e.g. an open box looks like a solid cube)
![Page 8: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/8.jpg)
8
Visual Hull - Basics (3)
![Page 9: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/9.jpg)
9
Epipolar Geometry - Basics (4)
• The tree points [COP1,COP2,P] form an epipolar plane
• Intersection of this plane with image planes results in epipolar lines
• The line connecting the two centers of projection [COP1,COP2] intersects the image planes at the conjugate points e1 and e2 which are called epipoles
![Page 10: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/10.jpg)
10
Epipolar Geometry - Basics (5)
![Page 11: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/11.jpg)
11
Creating Image-Based Visual Hulls (1)
• Algorithm input:– set of k silhouettes (binary images) with
associated viewpoints– desired viewpoint (in this case, constructed
visual hull is viewpoint-dependent)• Algorithm output:
– sampled image of the visual hull, each pixel containing a list of occupied intervals of space
![Page 12: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/12.jpg)
12
Creating Image-Based Visual Hulls (2)
• The Basic Algorithm:– cast ray into space for each pixel in the
desired view of the visual hull– intersect ray with the k silhouette cones
k lists of intervals; intersect together single list of intersections of the viewing ray with the visual hull
![Page 13: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/13.jpg)
13
Creating Image-Based Visual Hulls (3)
![Page 14: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/14.jpg)
14
Creating Image-Based Visual Hulls (4)
• Trick: due to Epipolar Geometry interval calculation can be done in image space of reference images:– 3D: intersecting silhouette cone with viewing
ray– 2D: intersecting projected viewing ray with
silhouette
![Page 15: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/15.jpg)
15
Creating Image-Based Visual Hulls (5)
![Page 16: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/16.jpg)
16
Rendering IBVH (1)
• Reference images are used as textures• For each pixel:
– rank reference-image texture from “best” to “worst” according to angle, take reference with lowest
– avoid texturing surface points with an image whose line-of-sight is blocked by some other point of the visual hull
– consider visibility during shading based on visual hull (not actual geometry)
![Page 17: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/17.jpg)
17
Rendering IBVH (2)
![Page 18: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/18.jpg)
18
Rendering IBVH (3)
![Page 19: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/19.jpg)
19
System Implementation (1)
• Four calibrated and triggered digital cameras
• One desktop PC per camera for capturing and pre-processing video frames (image segmentation)
• Silhouette and texture information sent to central server for IBVH processing
![Page 20: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/20.jpg)
20
System Implementation (2)
• Server runs IBVH intersection and shading algorithms
• IBVH objects can be combined with OpenGL background
• System runs in ‘real time’ with heavy optimization (like caching strategies for silhouette intersection)
![Page 21: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/21.jpg)
21
System Implementation (3)
![Page 22: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/22.jpg)
22
Summary & Results
• Use visual hull as object shape approximation• Using silhouette information from reference
views to generate view dependent visual hull• Reference images are used as ‘textures’
• Results:Videoclips
![Page 23: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/23.jpg)
23
Future Work
• Find Techniques for blending between textures to produce smoother transitions
• Scale up system by using larger number of cameras
• Split workload on multiple servers, as algorithm parallelizes fairly much
• Speed up viewing ray silhouette intersections (most expensive part of the computation)
![Page 24: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/24.jpg)
24
Personal Opinion (1)
• Pros:– simple technique / low-cost hardware– image-based representation partially
compensates simplification problems– epipolar geometry reduces 3D-intersection
problems to 2D-intersections
![Page 25: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/25.jpg)
25
Personal Opinion (2)
• Cons:– texture flipping during viewpoint transitions
produces ugly results– shadows are considered as part of the object– preprocessing is really expensive
(85 ms for image foreground segmentation)
![Page 26: Image-Based Visual Hulls](https://reader036.fdocuments.us/reader036/viewer/2022081604/56815694550346895dc43cc9/html5/thumbnails/26.jpg)
26
The End
“If there are no questions,there won’t be any answers.”
? ? ?
???