CS 563 Advanced Topics in Computer GraphicsIntroduction To IBR

By Cliff Lindsay

Slide Show ’99 Siggraph[6]

What Is IBR?

IBR: Multidisciplinary field that includes computer vision

and graphics

Techniques that replace and/or augment polygon models

Primary data pre-rendered images and photographs as input

The Rendering Spectrum [3]

Where Did IBR Come From?

Photo-realism Modeling ability has been stifled by rendering

advancements Availability of inexpensive digital image

acquisition hardware Recent graphics accelerators trends Necessity to render object that can’t be

rendered using polygons

Tenants and Common Techniques of IBR

Rendering time is decoupled from scene complexity

Images are used as input Exploits coherence Pre-calculation of scene data/images

Computer Vision Computer Graphics = IBR.

Combining CG and Computer Vision

Vision Technology Lacks robust Algorithms The Graphics Industry Needs better modelling

Siggraph ’99 Course on Image-based Rendering[6]

How Does IBR Compare to Traditional Rendering?

Image Warping Vs. Matrix Transformations

Perspective Division Vs. Projective Normalization

What the !@#$ Is A Splat Kernel?

[1] Image from Leonard McMillan

Approaches

2D Approaches: Texture Mapping Sprite, Billboards, and Impostors Image Layering

3D approaches: LDI (2.5D) View Interpolation & Morphing Mosaics

4D approaches: The Lumigraph Light Fields

Texturing Mapping

Texture Space (u,v) 3D Object Space (x0, y0, z0) Screen Space (x, y) [5]

Texture mapping has close ties to Image Warping

Wide Industry Support (hardware and Software)

Filtering

Sprites, Billboards, and Impostors (Oh my!)

Sprites: Pure 2D image No warping, or projection (like mouse cursor)

Billboards: Sprite applied to a polygon Alpha channel usually employed Uses texture mapping for acceleration

Impostors: Billboards created on the fly. Can represent complex models Error metric associated w/ changed views

Billboards

Billboards: Oriented toward viewer Matrix transformations (classical pipeline) Special effects (lens flares, laser/light bursts, etc) Hard to render objects (clouds, fire, smoke)

Impostors

Impostor Techniques: Error Angle Off Screen Rendering Polygon Texturing Texture resolution need not exceed screen

resolution texres = screenres * objsize/(2 * distance *

tan(fov/2))

Billboard Example

Billboard Example

Lumigraph/Light Fields 0

Plenoptic function An image is a collection of radiance values a long a ray. Radiance value for all possible rays = Plenoptic function 4D (for our purpose)

[7]

[7]

Lumigraph/Light Fields 1

Represent an object by it’s extents Each point on a cube has multiple rays

eminating. Each wall has 2 planes (12 planes make a

cube)

Lumigraph/Light Fields 2

You parameterize a ray using the 2 planes L(s, t, u, v) = radiance for a ray Ray – plane intersection make it easy and fast

Lumigraph/Light Fields 3

Sample of the objects on the plane are not continuos

Gaps are Created

[10]

Lumigraph/Light Fields 4

Continuos luminance is a linear sum B – basis function for which we can calculate

at grid points If we use a constant value, the coefficient take

on the values of the grid points

[10]

Lumigraph/Light Fields 5

[10]

Lumigraph/Light Fields 6

Example Rendering

[10]

View Interpolation

Reference Image 1 Reference Image 2

CorrespondingPixels

Morph maps

Based on diagrams from Watt[8]

View Morphing - more to come next presentation!

View Morphing

View Morphing - more to come next presentation!

View Morphing[9]

View Morphing

I0 I1

I0 ' I1 'Is '

I s

1 1

2 2

3

View Morphing - more to come next presentation!

View Morphing[9]

Recent Developments & The Future of IBR

Surface Light fields High Dynamic Range Radiance Maps View-dependent texture-mapping (VDTM) IBO (Image Based Objects)

Conclusion

Rendering time is decoupled from scene complexity

Images are used as input

Pre-calculation of scene data/images

Additional Resources

http://citeseer.nj.nec.com/cs - NEC Digital Library

http://www.siggraph.org http://www.debevec.org/ (View Morphing, High

Dynamic Range Radiance Maps, Projective Texture-Mapping)

http://www-2.cs.cmu.edu/%7Eph/869/www/misc.html (a cool site with a bunch of IBR links)

http://www.peter-oel.de/ibmr-focus/ (Another cool site)

References

[1] McMillan, Leonard, “An Image-Based Approach to Three-Dimensional Computer Graphics ”, , http://graphics.laces.mitt.edu/~mcmillan/IBRwork/defense23.html, date unknown, Cited slide #6.

[2] McMillan, Leonard, Gortler, Steven, “Applications of Computer Vision to Computer Graphics”, ACM Siggraph, Vol. 33 no. 4, Nov. 99

[3] Akenine-Moller, Tomas, Haines, Eric, “Real-Time Rendering, 2nd Edition”, A K Peters, 2002[4] Watt, Alan, “3D Computer Games”, Addison-Wesley Pub Co, Volume 1, 2nd edition, 1999[5] Heckbert, Paul S., “Survey of Texture Mapping,” IEEE Computer Graphics & Applications,

Cited slide #10, November 1986,[6] Cohen, Michael, “Course on Image-based, Modeling, Rendering, and Lighting”,

Siggraph ‘99[7] Mcmillian, Leonard, Bishop, Gary, “Plenoptic Modeling: An Image-Based Rendering

System”, Proceedings of SIGGRAPH 95, (Los Angeles, CA August 6-11, 1995), pp. 39-46[8] Watt, Alan, “3D Computer Graphics”, Addison-Wesley Pub Co, 3nd edition (), 2000[9] Chen, S.E., Williams L., “View Interpolation for Image Synthesis”, ACM Siggraph ’95[10]Gortler, S, Cohen, M, Girzesczuk, R, Szeliski, R, “The Lumigraph”, ACM Siggraph, 1996