Enhancing Traditional Rasterization Graphics with Ray...
Transcript of Enhancing Traditional Rasterization Graphics with Ray...
-
Enhancing Traditional Rasterization Graphics with Ray Tracing
October 2015
-
James Rumble Developer Technology Engineer, PowerVR Graphics
-
3 @PowerVRInsider │ #idc15 facebook.com/imgtec
Overview
• Ray Tracing Fundamentals
• PowerVR Ray Tracing Pipeline
• Using Ray Tracing
-
4 @PowerVRInsider │ #idc15 facebook.com/imgtec
Overview
• Ray Tracing Fundamentals
• PowerVR Ray Tracing Pipeline
• Using Ray Tracing
-
5 @PowerVRInsider │ #idc15 facebook.com/imgtec
Introduction to Ray Tracing
Ray Tracing is not a new concept!
• Synonymous with photorealistic (offline) renders
• Traditionally too resource intensive for broad industry applications
Ray tracing is emerging as the next big thing in real-time graphics
• Used in a limited fashion i.e. reflections and shadows
• Often limited to very few rays and low resolution/course grained scene representations
• The benefits ray tracing can offer are obvious
-
6 @PowerVRInsider │ #idc15 facebook.com/imgtec
Ray Tracing Fundamentals
RAY = {origin, direction, clipping distance}
• Triangles
• Rays
• Intersection Testing
• Process of determining if a given ray
has intersected a given triangle
-
7 @PowerVRInsider │ #idc15 facebook.com/imgtec
Tracing Rays
• Commonly visualised and explained as a
process of modelling light
• Rays used to model real-world behaviour
of light
• Forward Ray Tracing:
• Rays traced from the light source
• Many rays never reach our eyes
• Backward Ray Tracing:
• Rays traced from the “eye” towards light
Forward Ray Tracing vs Backward Ray Tracing
-
8 @PowerVRInsider │ #idc15 facebook.com/imgtec
Tracing Rays
• Commonly visualised and explained as a
process of modelling light
• Rays used to model real-world behaviour
of light
• Forward Ray Tracing:
• Rays traced from the light source
• Many rays never reach our eyes
• Backward Ray Tracing:
• Rays traced from the “eye” towards light
Forward Ray Tracing vs Backward Ray Tracing
-
9 @PowerVRInsider │ #idc15 facebook.com/imgtec
What is Ray Tracing? Backwards Ray Tracing
• How is ray tracing
commonly used?
• ‘Primary’ rays fired from
“eye” through pixels of
the screen
• Finds closest triangle
intersection
• ‘Secondary’ Rays used for
additional effects
-
10 @PowerVRInsider │ #idc15 facebook.com/imgtec
Ray Tracing Basics
• Now using a concrete example…
• Cast primary rays out into the world
through each pixel
• Find the closest triangle intersection
• ‘Secondary’ Rays used for
additional effects (direct lighting)
Backwards Ray Tracing
-
11 @PowerVRInsider │ #idc15 facebook.com/imgtec
What is Ray Tracing?
• Can be used in a more abstract
way than ‘simply’ modelling light
• Can be used to calculate spatial
queries between rays and
objects/triangles
• i.e. Does ray x intersect object 1
• A ray can be fired as long it has
been given an origin and direction
Fire a ray from a box towards a
neighbouring object. It intersects
the plane in between
Fire a ray from a box towards a
neighbouring object. It reaches
the other box and is marked as
intersected
Spatial Queries
-
12 @PowerVRInsider │ #idc15 facebook.com/imgtec
What is Ray Tracing?
• Ray Tracing provides a mechanism for determining inter-object
visibility using spatial queries
• Objects can be aware of both their own properties and position but
now also those of other objects in the scene through the use of
rays
• Provides elegant solutions to many traditionally difficult to render
or process effects
-
13 @PowerVRInsider │ #idc15 facebook.com/imgtec
Overview
• Ray Tracing Fundamentals
• PowerVR Ray Tracing Pipeline
• Using Ray Tracing
-
14 @PowerVRInsider │ #idc15 facebook.com/imgtec
Introducing PowerVR Ray Tracing
• PowerVR Ray Tracing GPUs are built on top of the latest
generation of PowerVR GPUs
• Additional ray tracing specific hardware designed to integrate
efficiently and communicate perfectly with the rest of the GPU
architecture
• Real-time, interactive ray traced graphics that can comfortably
scale from mobile and embedded platforms to gaming consoles
and mainstream gaming PCs
-
15 @PowerVRInsider │ #idc15 facebook.com/imgtec
PowerVR Ray Tracing Pipeline
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Geometry
Data
Memory
Scene
Hierarchy
Ray Traversal Phase
Primary Ray
Generation
Intersection
TestsRay Shaders Accumulation
-
16 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Organises triangles based on their size and position
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Memory
Scene
Hierarchy
Geometry Submission
Scene
ComponentsScenes
Geometry
Data
-
17 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Generation Geometry submission
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Memory
Scene
Hierarchy
Ray Traversal Phase
Primary Ray
Generation
Intersection
TestsRay Shaders Accumulation
Geometry Submission
Scene
ComponentsScenes
Geometry
Data
-
18 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Generation Scene Components
Windshield
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Car Body
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Headlights
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Tyres
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Chrome
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
-
19 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Generation
• Contain groups of scene components
• Ray traversal occurs within scene
World Scenes
Windshield
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Car Body
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Headlights
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Tyres
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Chrome
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Windshield
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Car Body
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Headlights
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Tyres
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Chrome
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Windshield
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Car Body
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Headlights
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Tyres
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
Chrome
Scene Component:
- Program:
- Vertex Shader
- Ray Shader
- Uniforms
- Geometry Data
- Buffers
- Textures
-
20 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Generation Vertex processing
Scene Hierarchy Building Phase
Hierarchy
Generation
Memory
Scene
Hierarchy
Ray Traversal Phase
Primary Ray
Generation
Intersection
TestsRay Shaders Accumulation
Geometry
Data
Vertex
Processing
-
21 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Generation
• Vertex shader used to transform geometry to world space
Vertex processing
-
22 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Generation Hierarchy generation
Scene Hierarchy Building Phase
Memory
Scene
Hierarchy
Ray Traversal Phase
Primary Ray
Generation
Intersection
TestsRay Shaders Accumulation
Geometry
Data
Vertex
Processing
Hierarchy
Generation
-
23 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Hierarchy Generation Hierarchy generation
• Vertex shader output is passed to a fixed function scene generator which
calculates the scene hierarchy
-
24 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Geometry
Data
Memory
Scene
Hierarchy
Ray Traversal Phase
Primary Ray
Generation
Intersection
TestsRay Shaders Accumulation
-
25 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal Primary ray generation
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Geometry
Data
Memory
Scene
Hierarchy
Ray Traversal Phase
Intersection
TestsRay Shaders Accumulation
Primary Ray
Generation
-
26 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal
• Frame Shaders are run once
per pixel to emit primary rays
into the scene
Primary ray generation
-
27 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal Intersection tests
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Geometry
Data
Memory
Scene
Hierarchy
Ray Traversal Phase
Ray Shaders AccumulationPrimary Ray
Generation
Intersection
Tests
-
28 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal Intersection tests
• Simple approach – each ray tested
against all triangles:
• Brute force
• m * n intersection tests (m – rays, n – objects)
• Use accelerated data structure to
reduce intersection tests
• Application developer does not need to worry
about handling this
• Accelerated triangle and box testing
-
29 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal
• Simple approach – each ray tested
against all triangles:
• Brute force
• m * n intersection tests (m – rays, n – objects)
• Use accelerated data structure to
reduce intersection tests
• Fixed function hardware accelerated triangle and
box testing
Intersection tests
-
30 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal Ray shaders and accumulation
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Geometry
Data
Memory
Scene
Hierarchy
Ray Traversal Phase
Intersection
Tests
Primary Ray
GenerationRay Shaders Accumulation
-
31 @PowerVRInsider │ #idc15 facebook.com/imgtec
Scene Ray Traversal
• Ray Shaders:
• Defined for each scene
component and invoked when
a ray intersects a triangle
• Defines material properties of
component (fully
programmable)
• Can emit further rays causing
additional intersection tests
Ray shaders and accumulation
accumulate(0.85, 0.85, 0.75);
reflect(incidentDirection, normal)
-
32 @PowerVRInsider │ #idc15 facebook.com/imgtec
Types of Rays are important
• Rays are a communication medium making inter-object visibility
possible
• An engine will likely use multiple “types” of ray
• Primary Rays – If it hits anything shade it / emit additional rays
• Shadow Rays – if it doesn’t hit a light – kill it
• etc.
-
33 @PowerVRInsider │ #idc15 facebook.com/imgtec
Coherence-Gathering
• What if these rays could be automatically coalesced?
-
34 @PowerVRInsider │ #idc15 facebook.com/imgtec
Pipeline Summary
Scene Hierarchy Building Phase
Vertex
Processing
Hierarchy
Generation
Memory
Scene
Hierarchy
Ray Traversal Phase
Primary Ray
Generation
Intersection
TestsRay Shaders Accumulation
Geometry Submission
Scene
ComponentsScenes
Geometry
Data
-
35 @PowerVRInsider │ #idc15 facebook.com/imgtec
Overview
• Ray Tracing Fundamentals
• PowerVR Ray Tracing Pipeline
• Using Ray Tracing
-
36 @PowerVRInsider │ #idc15 facebook.com/imgtec
How do we use it?
Hybrid Shadows,
Reflections, etc.
Production-
Quality Renders Ambient
Occlusion
Physics &
Collision
Detection
A.I. & Line of
Sight
Calculations
Global Illumination Virtual Reality
Lens correction, Ultra-low latency
rendering, Lenticular Displays
Asset creation /
compression
Rapid photo-
quality output
Order-Independent
Transparency
Augmented
Reality
-
37 @PowerVRInsider │ #idc15 facebook.com/imgtec
Render the scene
into a deep frame
buffer (G-Buffer)
Compute Lighting on
each G-Buffer pixel
(No Overdraw)
Assemble the final
frame and perform
screen-space effects
G-Buffer
Material IDs
Normals
World-Space Positions or depth buffer
Final Result Frame
Lighting Buffer(s)
Lighting calculations executed in screen
space
Modern Raster-Based Game Engine Deferred shading used in many modern game engines
-
38 @PowerVRInsider │ #idc15 facebook.com/imgtec
Ray Trace Using the G-Buffer Who needs primary rays when we have rasterization
G-Buffer
Material IDs
Normals
World Space Positions or Depth values
(1.0,0.0,0.0)
(0.5,0.6,-0.2)
(14.3,81.6,-0.4)
-
39 @PowerVRInsider │ #idc15 facebook.com/imgtec
Ray Trace Using the G-Buffer Who needs primary rays when we have rasterization
G-Buffer
Material IDs
Normals
World Space Positions or Depth values
(1.0,0.0,0.0)
(0.5,0.6,-0.2)
(14.3,81.6,-0.4)
-
40 @PowerVRInsider │ #idc15 facebook.com/imgtec
Render the scene
into a deep frame
buffer (G-Buffer)
Trace Rays to
determine lighting
Assemble the final
frame and perform
screen-space effects
Hybrid Ray Tracing Use the G-Buffer to set up your rays
G-Buffer
Material IDs
Normals
World-Space Positions or depth buffer
Final Result Frame
Lighting Buffer(s)
-
41 @PowerVRInsider │ #idc15 facebook.com/imgtec
Ray traced Cascaded shadow maps
Ray Traced Techniques Analytical 1 Ray Per Pixel Soft Shadows
http://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdf
http://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idc15_SF_EnhancingTraditionalRasterizationGraphicsWithRayTracing.pdf
-
42 @PowerVRInsider │ #idc15 facebook.com/imgtec
Direct Lighting Only Ray Traced Global Illumination
Ray Traced Techniques Low Overhead Probe – based Global Illumination
http://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdf
http://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdfhttp://cdn.imgtec.com/sdk-presentations/idf15_SF_LowOverheadProbeBasedGlobalIlluminationUsingRayTracing.pdf
-
43 @PowerVRInsider │ #idc15 facebook.com/imgtec
Summary
• An introduction to ray tracing
• The PowerVR Ray Tracing Pipeline
• Hybrid Ray Tracing
• Ray Traced Technique Teaser