Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi...

Click here to load reader

  • date post

    20-Dec-2015
  • Category

    Documents

  • view

    220
  • download

    3

Embed Size (px)

Transcript of Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi...

  • Slide 1
  • Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi Ramamoorthi http://www.cs.columbia.edu/~cs4162
  • Slide 2
  • To Do / Motivation Work hard on assignment 4 This last series of lectures covers (at a high level) some more advanced topics and areas of current research interest in modern rendering
  • Slide 3
  • Course Outline 3D Graphics Pipeline Rendering (Creating, shading images from geometry, lighting, materials) Modeling (Creating 3D Geometry)
  • Slide 4
  • Next few slides courtesy Paul Debevec; SIGGRAPH 99 course notes
  • Slide 5
  • Slide 6
  • Slide 7
  • Image-Based Modeling and Rendering Generate new views of a scene directly from existing views Pure IBR (such as lightfields): no geometric model of scene Other IBR techniques try to obtain higher quality with less storage by building a model
  • Slide 8
  • IBR: Pros and Cons Advantages Easy to capture images: photorealistic by definition Simple, universal representation Often bypass geometry estimation? Independent of scene complexity? Disadvantages WYSIWYG but also WYSIAYG Explosion of data as flexibility increased Often discards intrinsic structure of model?
  • Slide 9
  • Slide 10
  • Slide 11
  • IBR: A brief history Texture maps, bump maps, env. maps [70s] Poggio et al. MIT: Faces, image-based analysis/synthesis Modern Era Chen and Williams 93, View Interpolation [Images with depth] Chen 95 Quicktime VR [Images from many viewpoints] McMillan and Bishop 95 Plenoptic Modeling [Images w disparity] Gortler et al, Levoy and Hanrahan 96 Light Fields [4D] Shade et al. 98 Layered Depth Images [2.5D] Debevec et al. 00 Reflectance Field [4D] Inverse rendering methods (Sato,Yu,Marschner,Boivin,) Fundamentally, sampled representations in graphics
  • Slide 12
  • Outline Overview of IBR Basic approaches Image Warping Light Fields Survey of some recent work
  • Slide 13
  • Warping slides courtesy Leonard McMillan
  • Slide 14
  • Slide 15
  • Slide 16
  • Slide 17
  • Slide 18
  • Outline Overview of IBR Basic approaches Image Warping [2D + depth. Requires correspondence/disparity] Light Fields [4D] Survey of some recent work
  • Slide 19
  • Slide 20
  • Plenoptic Function L(x,y,z, , ,t, ) Captures all light flow in a scene to/from any point (x,y,z), in any direction ( , ), at any time (t), at any frequency ( ) Enough information to construct any image of the scene at any time (x,y,z) (,)(,)(,)(,) [Funkhouser]
  • Slide 21
  • Plenoptic Function Simplifications Represent color as RGB: eliminate Static scenes: ignore dependence on t 7D 3 5D
  • Slide 22
  • Slide 23
  • Slide 24
  • Slide 25
  • Slide 26
  • Slide 27
  • Slide 28
  • Slide 29
  • Slide 30
  • Slide 31
  • Lumigraph Postprocessing Obtain rough geometric model Chroma keying (blue screen) to extract silhouettes Octree-based space carving Resample images to two-plane parameterization
  • Slide 32
  • Lumigraph Rendering Use rough depth information to improve rendering quality
  • Slide 33
  • Lumigraph Rendering Use rough depth information to improve rendering quality
  • Slide 34
  • Lumigraph Rendering Without using geometry Using approximate geometry
  • Slide 35
  • Unstructured Lumigraph Rendering Further enhancement of lumigraphs: do not use two-plane parameterization Store original pictures: no resampling Hand-held camera, moved around an environment
  • Slide 36
  • Unstructured Lumigraph Rendering To reconstruct views, assign penalty to each original ray Distance to desired ray, using approximate geometry Resolution Feather near edges of image Construct camera blending field Render using texture mapping
  • Slide 37
  • Unstructured Lumigraph Rendering Blending fieldRendering
  • Slide 38
  • Outline Overview of IBR Basic approaches Image Warping [2D + depth. Requires correspondence/disparity] Light Fields [4D] Survey of some recent work
  • Slide 39
  • LDIs Layered depth images [Shade et al. 98] Geometry Camera Slide from Agrawala, Ramamoorthi, Heirich, Moll, SIGGRAPH 2000
  • Slide 40
  • LDIs Layered depth images [Shade et al. 98] LDI
  • Slide 41
  • LDIs Layered depth images [Shade et al. 98] LDI (Depth, Color)
  • Slide 42
  • Slide 43
  • Surface Light Fields Miller 98, Nishino 99, Wood 00 Reflected light field (lumisphere) on surface Explicit geometry as against light fields. Easier compress
  • Slide 44
  • Acquiring Reflectance Field of Human Face [Debevec et al. SIGGRAPH 00] Illuminate subject from many incident directions
  • Slide 45
  • Example Images Images from Debevec et al. 00
  • Slide 46
  • Conclusion (my views) Real issue is compactness/flexibility vs. rendering speed IBR is use of sampled representations. Easy to interpolate, fast to render. If samples images, easy to acquire. Of course, for this course, some pretty fancy precomputed algorithms (because we want to handle complex lighting that changes) IBR in pure form not really practical WYSIAYG Explosion as increase dimensions (8D transfer function) Ultimately, compression, flexibility needs geometry/materials But lots of recent work (some in course) begins to correct these issues Right question is tradeoff compactness/efficiency Factored representations Understand sampling rates and reconstruction