Virtual Reality Features of NVIDIA GPUs
42
Mark Kilgard, July 27, 2016 Virtual Reality Features of NVIDIA GPUs
-
Upload
mark-kilgard -
Category
Technology
-
view
119 -
download
0
Transcript of Virtual Reality Features of NVIDIA GPUs
Mark Kilgard, July 27, 2016
Virtual Reality Features ofNVIDIA GPUs
2
Mark Kilgard
Principal System Software EngineerOpenGL driver implementation & API evolutionOther interests
Programmable shading languages, original Cg GPU shading language
Web, vector graphics & path rendering, GPU rendering for 2D graphics
Also known for creating OpenGL Utility Toolkit (GLUT)
My Background
3
GPU Challenges for Virtual Reality Rendering
1. Must render unique view for each eyeTwo scene renders per frame
2. Wide field-of-view renderedMore pixels, distributed towards periphery
3. HMD lens magnifies rendered sceneNon-linear inverse lens warp
Allows the HMD lens magnification to provide convincing sense of presence
4
Magnification Effect of HMD Lens
Graph paper viewed & magnified through HMD lens
5
Head Mounted Display Lens Effect
Barrel Distorted Imageson HMD Screen
Magnified Overlapping Wide Field-of-viewBinocular View
6
Focus on Virtual Reality Features of Pascal GPUs
Virtual Reality rendering features in Pascal GPUsGeForce 1080/1070/1060 NEW Quadro P6000 & P5000
As exposed in standard 3D APIs
7
Background: NVIDA GPU Architecture Road Map
Our interest NVIDIA GPU architectures of interest: Pascal
What is Pascal mentioned on last slide?
8
Conventional 3D Single-eye Scene
9
Conventional 3D Single-eye Scene
10
Wide Stereo Views of Same Scene
Left Right
11 Left Right
12 Right Left
13
Left & Right Views AreSimilar but Different
Left and Right eye views have different occlusionWhile left & right eye views look similar, must be draw independently
Really only different in a small translation in X
NVIDIA Single Pass Stereo supports generating left & right eye positionsGPU automatically makes left & right eye versions of each primitiveThen renders each view to a different framebuffer sliceExposed in OpenGL by NV_stereo_view_rendering extensionExposed for Direct3D 11 by NVAPI SinglePassStereoMode feature
14
Image Difference of Two Views
− + 0.5 =
Left eye view Right eye viewClamped difference image
15
Left & Right Views Rendered Efficiently in One Pass
16
Lens Matched Shading
Left Right
17
Visualizing Lens Matched Shading
18
Lens Matched Shading
Left Right
19
Visualization of Lens Matched Shading Rendering
20
Still Needs Non-linear Warp
Lens Matched Shadingrendering
Warped image suitablefor HMD display
Non-linearimage
21
Still Needs Non-linear Warp
Lens Matched Shadingrendering
Warped image suitablefor HMD display
Non-linearimage
BUT substantiallyless pixel motion
and distortionthan conventional
VR re-warping
22
Example HMD Post-rendering Warp
23
Lens Matched Shadingwith Window Rectangle Testing
24
Lens Matched Shadingwith Window Rectangle Testing
Nothing in black corners is shaded or even rasterizedYellow lines show overlaid 8 inclusive window rectanglesSame 8 window rectangles “shared” by each view’s texture array layer
25
Lens Matched Shadingwith Window Rectangle Testing
26
What Didn’t Have to be Rendered
17% of left view pixels 17% of right view pixels
27
Window Rectangle TestingDuring Rasterization
New multi-vendor OpenGL extension
EXT_window_rectangles
Provides a set of window-space rectangles, 8 for NVIDIA GPUs
Rasterizer either includes or excludes rasterization from rectangles
Discards at speed-of-lightAs if free to rasterize to irregular rectangular regions
WindowRectangles
Test
NEWstage
OpenGL’s fragment processing pipeline
28
Warped Lens Matched Scene
Warped version of Lens Matched Shading to match HMD lens
29
Warped Lens Matched Shading
30
Visualizing Warp Window Rectangles
31
What Didn’t Have to be Warped
17% of left view pixels 17% of right view pixels
32
VR Rendering Pipeline in Images
LMS Right Eye View Warped Right Eye View
LMS Left Eye View Warped Left Eye View
Scene
LMS = Lens Matched Shading
33
VR Rendering Pipeline in Images
LMS Right Eye View Warped Right Eye View
LMS Left Eye View Warped Left Eye View
Scene
Displayed within HMD
Single Rendering PassSingle Pass Stereo +
Lens Matched Shading +Window Rectangle Testing
Drawn with Single TriangleFragment Shader WarpingWindow Rectangle Testing
Perception to user is linear rendering HMD lens “undoes” warping to provide
a perceived wide field-of-view
Pascal does all thisefficiently in a singlerendering pass!
8 viewports, 1 pass
34
Traditional 3D Rendering Pipeline
Transform3D View Downsample Display
FrameApp
Processing
if multisampled
uses pinhole camera projection,rasterizing to uniform rectilinear pixel grid
monitor directlydisplays uniform rectilinear pixel grid
Rasterize3D View
35
Virtual Reality Rendering PipelineRendering Left & Right Eye Views
TransformLeft View Downsample
DisplayFrame to HMD
AppProcessing
RasterizeLeft View
TransformRight View DownsampleRasterize
Right View
HMDWarp
HMDWarp
warps uniform rectilinear grid based on HMD’s device profile
TransformRight View
RasterizeRight View
TransformRight View DownsampleRasterize
Right ViewTransformRight View
HMDWarpDownsampleRasterize
Right ViewTransformRight View
second view rendering
36
Virtual Reality Rendering PipelineWith Lens Matched Shading
TransformLeft View Downsample
DisplayFrame to HMD
AppProcessing
RasterizeLeft View
TransformRight View DownsampleRasterize
Right View
HMDWarp
HMDWarp
LensMatchedShading
LensMatchedShading
replicates primitiveto 4 projected quadrants
less pixel resamplingeffort & error operating onLMS rasterized images
37
Virtual Reality Rendering PipelineWith Single Pass Stereo
TransformLeft & Right
View
Downsample
DisplayFrame to HMD
AppProcessing
RasterizeLeft View
DownsampleRasterizeRight View
HMDWarp
HMDWarp
one transform instance,outputs left & right position
38
Virtual Reality Rendering PipelineSingle Pass Stereo + Lens Matched Shading
Downsample
DisplayFrame to HMD
RasterizeLeft View
DownsampleRasterizeRight View
HMDWarp
HMDWarp
LensMatchedShading
LensMatchedShading
TransformLeft & Right
View
AppProcessing
+ Advantages1. Single transform for both views2. Closer match between rendering & lens images3. Fewer overall pixels for similar quality
use window rectangle testingto bound lens region, duringboth rasterization & warp
39
Pascal GPU Features for VRUnique to NVIDIA
Pascal renders left & right eyeIn single rendering pass
Halves CPU load for rendering
Better concurrent texture locality
OpenGL: NV_stereo_view_renderingDirect3D: NVAPI SinglePassStereoMode
Better matches rendering to HMD lensSingle pass
Four separately projected quadrants
Less rendering, less resampling error
OpenGL: NV_clip_space_w_scalingDirect3D: NVAPI ModifiedWSupport
40
Other NVIDIA Virtual Reality Initiatives
41
More Information
VRWORKS 2.0 Software Development KitAvailable NOW
Pascal OpenGL extensions for Virtual RealitySpecification now public
In official OpenGL.org extension registry
OpenGL Extension Wrangler 2.0 (GLEW) support
Example code in VRWORKS 2.0 SDK
VRWORKS also documents NVAPI usageFor Direct3D 11
Includes Examples code
