Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction)...
Transcript of Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction)...
![Page 1: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/1.jpg)
Advanced Geometry
![Page 2: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/2.jpg)
Topics
1. Automatic content creation
2. Level-of-detail representations
2
[Liu et al. Eurographics 2015]
![Page 3: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/3.jpg)
Content creation
3
![Page 4: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/4.jpg)
Traditional manual approach
• Use of 3D modeling software
• Manually define geometry and materials
• Compose model of multiple primitives
• Requires expertise and time, expensive
4
Example image of modeled fish from Wikipedia.
![Page 5: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/5.jpg)
Automatic content creation
Main idea
•Generate 3D models without manual modeling
• Two options
1. Measuring real-world objects
2. Modifying existing 3D models
5
![Page 6: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/6.jpg)
Measuring real-world objects
6
![Page 7: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/7.jpg)
Traditional Acquisition Techniques• Small set of discrete
measurements• Used in different applications
Anthropometric measurements Surveying
• Main advantage Easy to acquire and process
measurements
• Main disadvantage Impossible to get a detailed
shape description
![Page 8: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/8.jpg)
3D Scanners• New technology
• 3D (animation) scanners
• Record 3D video
• Active research area
• Powerful tool
• Preserve artwork / historic artifacts
• Acquire populations of 3D shapes for analysis
[P. Jenke, WSI/GRIS Tübingen]
![Page 9: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/9.jpg)
Types of 3D Scanners
Scanning Techniques:o Time-of-flight
Time-of-flight laser scanner
Time-of-flight depth cameras (dynamic)
o Triangulation Laser line sweep
Structured light
o Stereo / computer vision Passive stereo
Active stereo / space time stereo
Other techniques
![Page 10: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/10.jpg)
Example Scan (time of flight laser scanner)
[data set: University of Hannover]
![Page 11: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/11.jpg)
Example scan (structured light scanner)
color-codedstructured light
courtesy of Phil Fong,Stanford University
motion compensatedstructured light
courtesy of Sören König,TU Dresden
![Page 12: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/12.jpg)
Example scan (active stereo scanner)
![Page 13: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/13.jpg)
Example scan (stereo reconstruction)
multi view matching (6 cameras)(photo-consistent space carving)
[Data set: Christan Theobald, MPII, 2006]
multi view matching (8 cameras)(piecewise smooth variational surfaceon presegmented imagessolved with Bayesian belief propagation)
[Data set: Zitnick et al.,Microsoft Research, Siggraph 2004]
![Page 14: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/14.jpg)
Processing the scans
To be useful, the scans need to be processed
oHole filling
oOutlier removal
o…
oMany methods and software libraries available
14
![Page 15: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/15.jpg)
Allows for direct content creation
15
![Page 16: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/16.jpg)
Modifying existing 3D models
16
![Page 17: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/17.jpg)
Possibilities
• Common possibilities
o Extrapolation
o Interpolation
o Structure-aware modification
• Many possibilities, active area of research
17
![Page 18: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/18.jpg)
Exercise
Propose a way to interpolate and extrapolate between two 3D shapes
• How do you represent the shapes?
• What interpolation / extrapolation equation to use?
• What are the advantages and disadvantages of the method?
18
![Page 19: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/19.jpg)
Shape interpolation and extrapolation
• Possible in shape space using correspondence information
19
![Page 20: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/20.jpg)
Statistical shape spaces possible
• Learn statistical distribution of geometry of shape
• Use this information for synthesis
20
![Page 21: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/21.jpg)
Structure-aware modification
ChallengeoDirect point-to-point correspondence cannot be established
oPoint-wise or triangle-wise modification not meaningful
oActive area of research
21
Liu et al. Eurographics 2015
![Page 22: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/22.jpg)
Take advantage of symmetry information
22
Kalojanov et al. SGP 2012
![Page 23: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/23.jpg)
Decompose into basic building blocks
23
Liu et al. Eurographics 2015
![Page 24: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/24.jpg)
Level-of-detail representations
24
![Page 25: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/25.jpg)
Key idea
25
Distance from camera = less geometric detail required
![Page 26: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/26.jpg)
Subdivision surfaces
• We saw them already
• AdvantagesoVery strong geometric compression (4 triangles become 1)
o Theoretical convergence properties
• DisadvantagesoNot applicable to downsample shapes that do not have this structure
26
![Page 27: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/27.jpg)
Edge collapse
• Models can be downsampled by sequence of edge collapses
27
[Hoppe, Progressive Meshes, SIGGRAPH 1996]
![Page 28: Synthèse d'images animées 1 - Grenoble INP - Chamilo · Example scan (stereo reconstruction) multi view matching (6 cameras) (photo-consistent space carving) [Data set: Christan](https://reader035.fdocuments.us/reader035/viewer/2022081612/5f36578a8b2eed79a17b60db/html5/thumbnails/28.jpg)
Example result
28
[Hoppe, Progressive Meshes, SIGGRAPH 1996]