CS 563 Advanced Topics in Computer Graphics
View Interpolation and Image Warping
by Brad Goodwin
Images in this presentation are used WITHOUT permission
Over View
General Imaged-Based Rendering
Interpolation
Plenoptic Function
Layered Depth Image (LDI)
Introduction
Image Based Rendering (IBR) Composed of photometric observations Mix of fields (photogrammetry, vision, graphics) Texture mapping Environment mapping Realistic surface models Uses from virtual reality to video games Just render the 3D scene? Judge results?
Different types of rendering using different amounts of geometry
Interpolation
Morphing interpolating texture map and shape
Generation of a new image is independent of scene complexity
Morph adjacent images to view between based on viewpoints being closely
spaced Uses camera position, orientation and
range to deteremine pixel by pixel Images pre computed and stored as
morph maps
About this method
Method can be applied to natural images Only synthetic were tested with this paper Of course this paper was in ’93 so hopefully
someone’s tested them by now
Only accurately supports view independent shading Others could be used on maps but they are
discussed
Types of Images
Can be done with natural or sythetic images
Sythetic easy to get the range and camera data
Natural Use ranging camera Computed by photogrammetry or artist
General Setup
Morphing can interpolate different parameters Camera position Viewing angle Direction of view Hierarchical object transformation
Find correspondence of images Images arranged in graph structure
Find correspondence
Usually done by animator This method
Form of forward mapping uses camera and range to do it
Cross dissolving pixels(not view-independent)
Done for each source image Quadtree compression
Move groups of pixels Scene moves opposite camera Offset vectors for each pixel (“morph
map”) Small change more accurate when interpolated
Offset vectors Sampled every 20 pixels
Overlaps and holes
Overlaps Local image contraction - several samples
move to the same pixel in interpolated image Perpendicular to oblique
Holes Show when mapping source to destination Background color Interpolate four corners of the pixel instead
of center (filling and filtering) Interpolate adjacent offset vectors Or if part seen in interpolated but not
source
Block Compression
Pixels ten to move together so block compression algorithm is used to compress morph map.
Related to image depth complexity High complexity low compression ratio
View independent Priority
Established to determine points that are viewable Pixels are ordered from back to front based on Z-
coordinates established in morph map Eliminates need for interpolating the Z-
coordinates of every pixel and updating the Z-buffer in the interpolation process.
Applications
Virtual Reality Motion blur
Uses super-sampling of many images computationally which is expensive thus inefficient
Reduce cost of computing a shadow map Only for point light sources
Create 3D primitives without creating 3D primitives
Plenoptic Modeling
The Plenoptic function Latin root plenus – complete or full
optic - pertaining to vision Parameterized function for describing
everything that is visible from a given point in space
Used as a taxonomy to evaluate low-level vision
Adelson and Bergen postulate“…all the basic visual measurements can be
considered to characterize local change along one or tow dimensions of a single function that describes the sructure of the information in the light impinging on an observer.”
Parameters
azimuth and elevation angle
Plenoptic
Set of all possible environment maps for a given scene
Specify point and range for some constant t
A complete sample can be defined as a full spherical map
Plenoptic Modeling
Claimed that all image-based rendering approaches are just attempts to create a plenoptic function with just a sampling of it
Set up is the same as most approaches Set of reference images which are
warped to create instances of the scene from arbitrary view points
Sample Representation
Unit sphere Hard to store on a computer Example of all distorted maps
Six planar projections of a cube Easy to store 90 degree face requires expensive lens
system to avoid distortion Oversampling in corners
Have to choose Cylindrical Easily unrolled Finite height :problems with boundary
conditions No end caps
Aquiring Cylindrical Projections
Get the projections is simple Tripod that can continuously pan Ideally camera’s panning motion should
be exact center of tripod When panning objects are far away slight
misalignment is tolerated
Panning takes place entirely on the x-z plane
Both images should have points within each other.
Find the projection of the output camera on input cameras image plane
That is the intersection of the line joining the two camera locations with the input camera’s image plane
Line joining the two cameras is the epipolar line
Intersection with the image plane is the epipolar point
Map image point to output cylinder Same techique for comparing points used with face
mapping from last week
Layered Depth Images
Paper presents some methods to render multiple frames per second on a PC
Sprites – are texture maps or images with alphas (transparent pixels) rendered onto planar surfaces
One method warps Sprits with Depth Warps depth values and uses this information to add
parallax correction to a standard sprite renderer
LDI Single input camera Contains multiple pixels along each line of sight Size of representation grows linearly with the depth
complexity of the scene Uses McMillan’s warp odering algorithm because data is
represented in a single image coordinate system.
References Chen S E and Williams L, "View Interpolation for Image
Synthesis", Proc. ACM SIGGRAPH '93 McMillan L, and Bishop, "Plenoptic Modeling: An Image-based Rendering System", Proc. ACM SIGGRAPH '95
Shade, Gortler, He and Szeliski, "Layered-Depth Images", Proc. ACM SIGGRAPH '98
McMillan L. and Gortler S,"Applications of Computer Vision to Computer Graphics: Image-Based Rendering - A New Interface Between Computer Vision and Computer Graphics, ACM SIGGRAPH Computer Graphics Newsletter, vol 33, No. 4, November 1999
Shum, Heung-Yeung and Kang, Sing Bing, A Review of Image-based Rendering Techniques, Microsoft Research
Watt, 3D Graphics 2000, Image-based rendering and phto-modeling (Ch 16)
http://www.widearea.co.uk/designer/anti.html http://www.dai.ed.ac.uk/CVonline/LOCAL_COPIES/EPSRC_S
SAZ/node18.html http://www.cs.northwestern.edu/~watsonb/school/teachin
g/395.2/presentations/14
Questions???????
Top Related