Multiview Rendering
description
Transcript of Multiview Rendering
MULTIVIEW RENDERING
Juraj ObertFEL ČVUT
supervised by Ing. Jan Buriánek
2
Outline
Motivation Related Work Rendering Techniques Requirements PerspectiveCube Results Future Work Discussion
Multiview Rendering - Juraj Obert - FEL CVUT
3
Motivation
Perspective (pinhole) camera is defined by a set of rays, which are (1) incident with the center of projection and (2) the optical axis is perpendicular to the image plane.
Relaxation of the latter requirement gives rise to central cameras, which are defined as sets of rays, which meet in one point.
Multiview Rendering - Juraj Obert - FEL CVUT
Object
Inverted image
Pinhole
Optical axis
4
Motivation
Multiview Rendering - Juraj Obert - FEL CVUT
Perspective (pinhole) camera is defined by a set of rays, which are (1) incident with the center of projection and (2) the optical axis is perpendicular to the image plane.
Relaxation of both requirements gives rise to non-central cameras, which are defined as sets of arbitrary rays in 3D space. In traditional rendering systems, perspective cameras are rather well-explored. Our goal was to create a system for working with non-perspective (also termed multiperspective) projections and images.
5
Related Work – Multiperspective Panoramas
Sweeping a window across the image simulates movement in 3D space
Used in Walt Disney’s Pinocchio in 1940 and researched in 1997 by Wood et al.
Fostered a lot of new research
Multiview Rendering - Juraj Obert - FEL CVUT
6
Related Work – X-Slits
X-Slits consist of rays incident with two non-intersecting lines in 3D space Yield images, which are fundamentally different from perspective images,
but look alike (hardly distinguishable for inexperienced users) Can simulate perspective projections by using spatiotemporal volumes
Multiview Rendering - Juraj Obert - FEL CVUT
7
Related Work – Digital Cubism
Multiview Rendering - Juraj Obert - FEL CVUT
8
Rendering Techniques
Spatiotempor
al volumes
•Stack video frames to create a spatiotemporal volume•Define a projection surface•Shade the projection surface by slicing the spatiotemporal volume•Amenable to programmable graphics hardware•Simple support for animation•Requires input video
Ray-
based co
mposition
•Each pixel corresponds to a different camera ray•Rays are arbitrary, no explicit patterns such as fans of rays as in the case of perspective projections•Implementation on graphics hardware an open research problem•Aliasing problems and user interface problems
Multiview Rendering - Juraj Obert - FEL CVUT
9
Spatiotemporal Volume Extract frames from input video file, stack them and build
a volume texture (x, y, t) Three important slicing planes:
x-y original frames y-t pushbroom image x-t epipolar plane image
Can be used to generate various classes of multiperspective images, such as X-Slits, cyclographs, etc.
Our goal was to develop a tool, which would allow a user to do this in real time
Multiview Rendering - Juraj Obert - FEL CVUTx
y
t
10
Requirements
Production requirements Capacity to handle huge amounts of data Utilization of graphics hardware Presets for commonly used camera classes Interactivity Extensibility
Naive approach Decompress video file, create a volume texture containing every single frame
and store it in video memory Example:
19 seconds, 640 x 480, RGB, 24 FPS 400 MB 2 hours, 1280 x 720, RGB, 24 FPS 450 GB !!!
Video memory capacity of latest ( March 2007) NVIDIA graphics cards – 768 MB Video memory capacity of upcoming (June 2007) ATI graphics cards – 1024
MB => naive approach will not work for high-definition content !!!
Multiview Rendering - Juraj Obert - FEL CVUT
11
PerspectiveCube - Overview Octree-based spatial organization
of spatiotemporal volumes
GPU memory management based on adaptive assembly of shuffle textures
On-demand loading
Two-level caching mechanism and multithreaded rendering engine for interactivity
Predefined camera types and animation curves
Robust, scalable and GPU-based
Multiview Rendering - Juraj Obert - FEL CVUT
12
PerspectiveCube – Rendering
Multiview Rendering - Juraj Obert - FEL CVUT
13
PerspectiveCube - Caching
Multiview Rendering - Juraj Obert - FEL CVUT
14
Perspective Cube – Demo
Multiview Rendering - Juraj Obert - FEL CVUT
15
Perspective Cube – Demo
Multiview Rendering - Juraj Obert - FEL CVUT
16
Perspective Cube – Demo
Multiview Rendering - Juraj Obert - FEL CVUT
17
Results - Redmond
Pushbroom
X-Slits
Redmond X-Slits right
Original
Multiview Rendering - Juraj Obert - FEL CVUT
x
t
Resolution: 360 x 240Length: 148 secondsFPS: 1524 bpp--------------------------------STV size 548 MBShuffle size ~ 128 MB
18
Results – Langweil’s Prague
Original Pushbroom
Multiview Rendering - Juraj Obert - FEL CVUT
Resolution: 720 x 480Length: 30 secondsFPS: 15
24 bppSTV size 445 MBShuffle size ~ 64 MB
19
Future Work Explicit camera ray representation
Depart from spatiotemporal volumes and use ray-based techniques More freedom vs. higher productivity
Non-planar projection surfaces Even though already supported in code, practical applications still require
thorough research Problems with unwrapping
Integration with existing modeling systems Access to ray tracing engine, which is necessary for ray-based rendering Unwrapping for non-planar projection surfaces
User interfaces Controls for perspective camera manipulation are too restrictive
for multiperspective cameras How would one map one ray per pixel ?
Performance Effect of used video compression schemes, frame extraction scheduling
algorithm, video memory management Bottleneck is shuffle texture assembly and frame decompression
Multiview Rendering - Juraj Obert - FEL CVUT
20
Conclusion
Ray-based techniques amenable to ray tracing engines, but not to programmable graphics devices
PerspectiveCube – system for GPU slicing of spatiotemporal volumes Octree-based spatial organization and smart texture assembly algorithm
for video memory waste minimization Interactivity due to two-level caching mechanism and multithreaded
rendering Scalability in terms of video memory size Support for animation Presets for commonly used multiperspective camera types
In the future Push for ray-based techniques as ray tracing hardware gets developed Don’t necessarily require that programmable graphics devices be used,
CPU can be faster sometimes
Multiview Rendering - Juraj Obert - FEL CVUT
21
Thank you
Questions ?
Multiview Rendering - Juraj Obert - FEL CVUT