View Morphing by Steven M. SeitzCharles R. Dyer
description
Transcript of View Morphing by Steven M. SeitzCharles R. Dyer
![Page 1: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/1.jpg)
View Morphingby
Steven M. Seitz Charles R. Dyer
Irwin Chiu HauComputer ScienceMcGill University
Winter 2004
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
![Page 2: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/2.jpg)
Overview
• What is view morphing?• How to do view morphing?• Results• Conclusion
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Mona Lisa view morphsSource: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm
![Page 3: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/3.jpg)
View Morphing
What is view morphing?– What is it?
– Why do we care about it?
– Where do we see them?
– Image Morphing vs View interpolation vs View Morphing
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Virtual Cameras
Source:View Morphing;Steven M. Seitz,Charles R. Dyer
![Page 4: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/4.jpg)
View Morphing: Key Idea
Beier-Neely morph is NOT shape-preserving!– distortions– un-natural
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
A Shape-Distorting MorphSource: Steven M. Seitz, Charles R. Dyer
![Page 5: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/5.jpg)
View Morphing: Key IdeaView morphing uses 3D shape preserving morph!
– no distortions
– natural
A morph is 3D shape preserving if the results of two
different views represent new views of the same object
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
A Shape-Distorting MorphSource: Steven M. Seitz, Charles R. Dyer
View morphing fixesthese intermediate
steps!
![Page 6: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/6.jpg)
Why do we care?
View morphing is efficient
Produces new views without – 3D modelling
– Taking additional photos
View morphing creates impressive effects– Camera motion
– Image morphing
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
trueSpaceSource: www.caligari.com
![Page 7: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/7.jpg)
Image Morphing vs View interpolation vs View Morphing
View Morphing is an extension toImage Morphing (Beier and Neely, 1992)
– Produces physically plausible new views of a scene
View Morphing (Seitz and Dyer, 1996) is an improvement over View Interpolation (Chen and Williams, 1993)
– Addresses non-rigid transformations problems– Does not require depth values– Creates realistic image transitions
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
![Page 8: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/8.jpg)
Overview
• What is view morphing?• How to do view morphing?• Results• Conclusion
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Mona Lisa view morphsSource: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm
![Page 9: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/9.jpg)
How to do View Morphing?
View morphing in three steps1. Prewarp two images
2. Compute a morph between the prewarped images
3. Postwarp each in between images produced by the morph
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
View Morphing ProcedureSource: Steven M. Seitz, Charles R. Dyer
![Page 10: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/10.jpg)
How to do View Morphing?
View morphing in three steps1. Prewarp two images
2. Compute a morph between the prewarped images
3. Postwarp each in between images produced by the morph
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
View morphing in 1 steps: Assume parallel views!1. Compute a morph between the parallel images
![Page 11: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/11.jpg)
Parallel Views
Basic Theory
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
image point p0 = (x0,y0)scene point P = (X,Y,Z)p0 = Π0 Pwhere Π0 is a projection matrix
Morphing Parallel ViewsSource: Steven M. Seitz, Charles R. Dyer
image point p1 = (x1,y1)scene point P = (X,Y,Z)p1 = Π1 Pwhere Π1 is a projection matrix
![Page 12: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/12.jpg)
Parallel Views
Basic Theory
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
image point p = (x,y)scene point P = (X,Y,Z)p = Π Pwhere Π is a homogenous projection matrix
Π = [ H | - HC]H: position and orientation of image planeC: euclidean position of the cameraMorphing Parallel Views
Source: Steven M. Seitz, Charles R. Dyer
![Page 13: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/13.jpg)
Parallel Views
Mathematics for View Interpolation
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Πs: Linear interpolation of Π0 and Π1
Πs = (1 – s) Π0 + s Π1
s = [0,1]
Cs = (sCx, sCy, 0)fs = (1 – f) f0 + s f1
f : focal lenghC : center of a camera
![Page 14: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/14.jpg)
View Interpolation Recap
View Interpolation vs View Morphing
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Morphing Parallel ViewsSource: Steven M. Seitz, Charles R. Dyer
View InterpolationSource: 3D Games byAlan Watt and Fabio Policarpo
![Page 15: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/15.jpg)
How to do View Morphing?
View morphing in three steps1. Prewarp two images
2. Compute a morph between the prewarped images
3. Postwarp each in between images produced by the morph
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
View morphing in 1 steps: Assume parallel views!1. Compute a morph between the parallel images
![Page 16: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/16.jpg)
Non-Parallel Views
The General Case
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
This is where the 3-Step Algorithmcomes into play1. Prewarping: I0 to Î0 and I1 to Î1 2. Morphing: Î0 and Î1 into Îs
3. Postwarping: Îs into Is
Morphing in Three StepsSource: Steven M. Seitz, Charles R. Dyer
![Page 17: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/17.jpg)
Non-Parallel Views
Mathematics for Image Reprojection
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
ĤH-1p = pH and Ĥ are 3x3 matrices that represent the position and the orientation of their image planes
The resulting 3x3 matrix, ĤH-1 is a projective transformation that reprojects the image plane I into Î
Morphing in Three StepsSource: Steven M. Seitz, Charles R. Dyer
^
![Page 18: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/18.jpg)
Limitations
• Singular Views cannot be reprojected to form parallel views
• Singular configurations are settings where one of the camera resides in the field of view of another camera
• Still works, just kind of conceptually hacky
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Parallel viewSource: Steven M. Seitz, Charles R. Dyer
Singular viewSource: Steven M. Seitz, Charles R. Dyer
![Page 19: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/19.jpg)
Traditional Problems
Change in visibility creates1. Folds
– Occurs when a visible area becomes occluded
2. Holes– Occurs when an occluded area becomes visible
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Penumbra, umbra and hole regionsSource: Chen and Williams
FoldSource: Irwin Chiu Hau
Area
Point
![Page 20: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/20.jpg)
Producing The Morph
We have talked about theory behind algorithm1. How to project/unproject images to parallel planes
2. How to warp between parallel planes
3. Theoretical problems
Now, let’s do an actual View Dependent Morph!
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
![Page 21: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/21.jpg)
Producing The Morph
We need:1. Two images I0 and I1
2. Two perspective projection matrices Π0 and Π1
3. Correspondence between pixels
Note that a sequence of projection matrices Πs is required to
control the entire animation, but Πs can be computed automatically
if we know Hs .
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
![Page 22: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/22.jpg)
Controlling The Morph
Hs can be obtained indirectly by establishing constraints– Recall Hs is position and orientation of image plane – One way is to specify four control points
Note: Control points implicitly determine the postwarpingtransformation
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
View Morphing ProcedureSource: Steven M. Seitz, Charles R. Dyer
Four control points form the red bounding box to determine the postwarping stage
Yellow lines are set of features to determine the prewarping stage
![Page 23: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/23.jpg)
View Morphing sans Prewarping
Prewarping is not necessary for:– Objects that aren’t closely related
• Prewarping is less effective
• Computation is unstable
– Images that are approx. orthographic (eg. telephoto)
However, postwarping should not be left out to:– Reduce image plane distorsions
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
![Page 24: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/24.jpg)
Overview
• What is view morphing?• How to do view morphing?• Results• Conclusion
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Mona Lisa view morphsSource: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm
![Page 25: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/25.jpg)
Results
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Facial view morphsSource: Steven M. Seitz, Charles R. Dyer
![Page 26: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/26.jpg)
Results
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Facial view morphsSource: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm
![Page 27: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/27.jpg)
Results
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Mona Lisa view morphsSource: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm
![Page 28: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/28.jpg)
Results
Image Morphing vs View Morphing
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
Image Morphing vs View MorphingSource: Steven M. Seitz, Charles R. Dyer
![Page 29: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/29.jpg)
Conclusions
Things to remember
View Morphing• Powerful extension to image morphing• Produces new views of a scene• 3D shape preserving
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
![Page 30: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/30.jpg)
References
• View Morphing - Seitz and Dyer, 1996• View Interpolation - Chen and Williams, 1993• Image Morphing - Beier and Neely, 1992• 3D Games: Realtime rendering and Sofware Technology
- Alan Watt and Fabio Policarpo
Comp 767: Advanced Topics in Graphics: Image-Based Rendering
![Page 31: View Morphing by Steven M. SeitzCharles R. Dyer](https://reader035.fdocuments.us/reader035/viewer/2022062500/56815870550346895dc5cf7f/html5/thumbnails/31.jpg)
Questions?
Comp 767: Advanced Topics in Graphics: Image-Based Rendering