Queensland University of Technology CRICOS No. 000213J INB382/INN382 Real-Time Rendering Techniques...

Queensland University of Technology CRICOS No. 000213J INB382/INN382 Real-Time Rendering Techniques Lecture 9: Global Illumination Ross Brown

CRICOS No. 000213J a university for the world real R Lecture Contents Global Illumination Radiosity Pre-computed Radiosity Transfer Spherical Harmonics

CRICOS No. 000213J a university for the world real R Global Illumination There are one-story intellects, two-story intellects, and three-story intellects with skylights. All fact collectors with no aim beyond their facts are one-story men. Two-story men compare, reason and generalize, using labours of the fact collectors as well as their own. Three- story men idealize, imagine, and predict. Their best illuminations come from above through the skylight. - Oliver Wendell Holmes

CRICOS No. 000213J a university for the world real R Lecture Context As mentioned before, we have covered the direct forms of illumination on object surfaces Last week we moved further into more indirect illumination issues by modelling effects such as reflection and refraction We then presented a more generalised approach known as ray tracing simple but costly not practical for real-time just yet We now cover two relevant forms of global illumination in real-time rendering Radiosity and Pre- computed Radiance Transfer

CRICOS No. 000213J a university for the world real R Global Illumination In most rendering, the use of a local lighting model is the norm Meaning that only the surface data at the visible point is used in the lighting calculations This is useful for object-precision GPU systems you can easily parallelise SIMD instructions on a stream of vertex and pixel data Data is discarded once rendered Problematic for global illumination as we showed last week

CRICOS No. 000213J a university for the world real R Global Illumination Beginning We have covered reflection, refraction and area lighting, which are global illumination techniques because the rest of the scene influences the lighting on the object Lighting needs to be thought of as the paths that the photons take from light sources to the eye Local only acounts for one reflection from the light source to the object, onto the eye

CRICOS No. 000213J a university for the world real R Global Illumination Beginning Global illumination calculates the results of the paths of photons from any reflecting object in the room specular, transparent or diffuse The contributions are summed for the object being rendered This process is contained within the rendering equation developed by Kajiya [2] more later You can thus sum all the possible light paths to the surface the more paths, the greater the level of realism

CRICOS No. 000213J a university for the world real R Radiosity and Ray Tracing Thus global illumination is divided into two main approaches Radiosity calculation of light as radiation transfer through volume of air Ray Tracing calculation of light as a set of discrete ray samples through a volume We have covered the general idea of ray tracing Now we illustrate the main principles of Radiosity using a series of lectures notes from Brown University [1]

CRICOS No. 000213J a university for the world real R Color Bleeding Soft Shadows No Ambient Term View Independent Used in other areas of Engineering Radiosity for Inter-Object Diffuse Reflection

CRICOS No. 000213J a university for the world real R Pretty Pictures Reality (actual photograph) Minus Radiosity Rendering Equals the difference (or error) image http://www.graphics.cornell.edu/online/box/compare.html Mostly due to mis-calibration

CRICOS No. 000213J a university for the world real R 1.Model scene as patches 2.Each patch has an initial luminance value: all but luminaires (light sources) are probably zero 3.Iteratively determine how much luminance travels from each patch to each other patch until entire system converges to stable values We can then render scene from any angle without recomputing these final patch luminances they are viewer independent! The Radiosity Technique: An Overview

CRICOS No. 000213J a university for the world real R Overview of Radiosity The radiometric term radiosity means rate at which energy leaves a surface Sum of rates at which surface emits energy and reflects (or transmits) energy received from all other surfaces Radiosity simulations are based on thermal engineering model of emission and reflection of radiation using Finite Element Analysis (FEA) First determine all light interactions in a view-independent way, then render one or more views Consider room with only floor and ceiling: floor ceiling

CRICOS No. 000213J a university for the world real R Overview of Radiosity Suppose ceiling is actually a fluorescent drop- panel ceiling which emits light Floor gets some of this light and reflects it back Ceiling gets some of this reflected light and sends it back Simulation mimics these successive bounces through progressive refinement each iteration contributes less energy so process converges

CRICOS No. 000213J a university for the world real R Kajiyas [2] Rendering Equation Light energy travelling from point i to j is equal to light emitted from i to j, plus the integral over S (all points on all surfaces) of reflectance from point k to i to j, times the light from k to i, all attenuated by a geometry factor L(i j) is the amount of light travelling along the ray from point i to point j L e is the amount of light emitted by the surface (luminance) f(k i j, ) is the Bidirectional Reflectance Distribution Function (BRDF) of the surface. Describes how much of the light incident on the surface at i from the direction of k leaves the surface in direction of j is wavelength of light, use a different function for R, G, & B G(i j) is a geometry term which involves occlusion, distance, and the angle between the surfaces

CRICOS No. 000213J a university for the world real R Kajiyas Rendering Equation More complete model of light transport than either raytracing or polygonal rendering, but does omit some things, eg., subsurface scattering How do we evaluate this function? very difficult to solve complicated integral equations analytically

CRICOS No. 000213J a university for the world real R A scene has: geometry luminaires (light sources) observation point Light transport to camera must be computed for every incoming angle to observation point, bouncing off all geometry (in all directions combined) This is far too hard Rendering a Scene

CRICOS No. 000213J a university for the world real R Rendering a Scene Both raytracing and radiosity are crude approximations to rendering equation Raytracing: consider only a very small, finite number of rays and ignore diffuse inter-object reflections, perhaps using ambient hack to approximate that lost component. Can use either a BRDF or a simpler (e.g., Phong) model for luminaires Radiosity: approximate integral over differential source areas dA(i) with finite sum over finite areas; consider light transport not on basis of individual rays but on basis of energy transport between finite patches (e.g., quads or triangles that result from (adaptive) meshing) remember Lecture 5???

CRICOS No. 000213J a university for the world real R Adaptive Meshes for Radiosity

CRICOS No. 000213J a university for the world real R What Can We Do? Design an alternative simulation of real transfer of light energy With any luck, will be more accurate, but accuracy is relative hall of mirrors is specular raytracing museum with latex-painted walls is diffuse radiosity Best solutions are hybrid techniques: use raytracing for specular components and radiosity for diffuse components

CRICOS No. 000213J a university for the world real R What Can We Do? Radiosity approximates global diffuse inter- object reflection by considering how each pair of surface elements (patches) in scene send and receive light energy, an O(n 2 ) operation best accomplished by progressive refinement

CRICOS No. 000213J a university for the world real R Energy = light energy = radiosity for our purposes it should really be rate, ie., energy/unit time E i : initial amount of energy radiating from i th patch B i : final amount of energy radiating from i th patch B j : final amount of energy radiating from j th patch F j-i : fraction of energy B i emitted by j th patch that is gathered by i th patch (relationship between i th and j th patches based on geometry: distance, relative angles) F j-i B j : total amount of patch j 's energy sent to patch i i : fraction of incoming energy to a patch that is then exported in next iteration Some Important Symbols

CRICOS No. 000213J a university for the world real R Amount of light/radiosity/energy a patch finally emits is initial emission plus sum of emissions due to other n-1 patches in scene emitting to this patch - recursive definition. Note: F 1-1 is zero only for planar patches! Why is it zero for only planar patches? or Sender 1 Sender 2 Sender n Receiver patch i Lets Arrange Those Symbols

CRICOS No. 000213J a university for the world real R Arranging Those Symbols Thus: Rewrite as a vector product: And the whole system:

CRICOS No. 000213J a university for the world real R Decompose the first matrix as: Can be rewritten: (I D( )F)B = E where D( ) is diagonal matrix with i as its ith diagonal entry, and F is called a Form Factor Matrix and is based on geometry between patches If we know E, D( ), and F, we can determine B If we let A = I D( )F Arranging Those Symbols Some More

CRICOS No. 000213J a university for the world real R Arranging Those Symbols Even More Then we are solving (for B ) the equation AB = E This is a linear system, and methods for solving these are well-known, e.g. Gaussian elimination or Gauss-Seidel iteration (although which method is best depends on nature of matrix A) part of introductory linear algebra courses Typically want B, knowing E and A

CRICOS No. 000213J a university for the world real R Let us look at the floor ceiling problem again Both ceiling and floor act as lights emitting and reflecting light uniformly over areas (all surfaces considered such in radiosity) C emits 12 F gets 1/3 C F reflects 50% C gets 1/3 F C reflects 75% Progressive Refinement

CRICOS No. 000213J a university for the world real R Progressive Refinement Let ceiling emit 12 units of light per second Let floor reflect get 1/3 of light from ceiling (based on geometry) reflect 50% of what it gets Let ceiling get 1/3 of floors light (based on geometry), and reflect 75% of what it gets Writing B 1 for ceilings total light, and B 2 for floors, and E 1 and E 2 for light generated by each: Ceiling: Floor:

CRICOS No. 000213J a university for the world real R Progressive Refinement thus: B 1 = E 1 + 1 (F 2 -1(E 2 + 2 (F 1 -2B 1 ))) becoming: B 1 = E 1 + 1 F 2 -1E 2 + 1 2 F 1 -2F 1 -2B 1 which simplifies to: In general, this algebra is too complex, but we can find the solution iteratively using progressive refinement

CRICOS No. 000213J a university for the world real R Iterative method 1, gathering energy: send out light from emitters everywhere, accumulate it, resend from all patches Each iteration uses radiosity values from previous iteration as estimates for recursive form. Iterate by rows. B k = E + D(r)FB k-1 B 1 = E B 2 = E + D(r)FB 1 B 3 = E + D(r)FB 2 Where B k is your k th guess at radiosity values B 1, B 2 Progressive Refinement

CRICOS No. 000213J a university for the world real R Results for our example (notice what happens to values): {12, 0} = {B 1, B 2 }= {E 1, E 2 } {12, 2} = {12.5, 2} = {12.5, 2.08373} = {12.5208, 2.08373} {12.5208, 2.08681} {12.5217, 2.08681} {12.5217, 2.08695} Progressive Refinement tolerance; // ie. Matrix has not changed Radiosity Pseudocode">

CRICOS No. 000213J a university for the world real R Algorithm for fast progressive refinement through shooting: U 0 = e; // Set Matrix to initial emission values U = unshot energy B 0 = e; t = 1; do { i = index_of(MAX(u t-1 )); // Max row precalculate hemicube i ; // Shoot the radiance to each row for (j = 1; j < n; ++) { b j t = u i t-1 F i-j A i /A j + b j t-1 ; u j t = u i t-1 F i-j A i /A j + u j t-1 ; } u i t = u i t-1 F j-j ; ++t; } while B t -B t-1 > tolerance; // ie. Matrix has not changed Radiosity Pseudocode

CRICOS No. 000213J a university for the world real R Assumption that radiation is uniform in all directions Assumption that radiosity is piecewise constant usual renderings make this assumption, but then interpolate cheaply to fake a nice-looking answer this introduces quantifiable errors Computation of form factors F i-j can be tough especially with intervening surfaces, etc. Assumption that reflectivity is independent of directions to source and destination Limitations of Radiosity

CRICOS No. 000213J a university for the world real R Limitations of Radiosity No volumetric objects (though there are equations and algorithms for calculating surface-to-volume form factors) No transparency or translucency Independence from wavelength no fluorescence or phosphorescence Independence from phase no diffraction Enormity of matrices! For large scenes, 10K x 10K matrices are not uncommon (shooting reduces need to have it all memory resident)

CRICOS No. 000213J a university for the world real R More Comments Even with these limitations, it produces great pictures For n surface patches, we have to build an n x n matrix and solve Ax = b, which takes O(n 2 ), this gets rather expensive for large scenes Could we do it in O(n) instead? The answer, for lots of nice scenes, is Yes The Google search engine uses an system much like radiosity to rank its pages Site rankings are determined not only by the number of links from various sources, but by the number of links coming into those sources (and so on) After multiple iterations through the link network, site rankings stabilize Site importance is like luminance, and every site is initially considered an emitter

CRICOS No. 000213J a university for the world real R One approach is importance driven radiosity: if I turn on a bright light in the graphics lab with the door open, itll lighten my office a little but not much By taking each light source and asking whats illuminated by this, really? we can follow a shooting strategy in which unshot radiosity is weighted by its importance, i.e., how likely it is to affect the scene from my point of view No longer a view-independent solutionbut much faster Making Radiosity Fast

CRICOS No. 000213J a university for the world real R Precomputed Radiance Transfer PRT Imagine that time is stationary. Picture a volume of space filled with photons each cube of space can be said to have a constant photon density. Picture this field of photons in linear motion We need to find out how many photons collide with a stationary surface for each unit of time, a value called flux which measured in joules/second or watts Divide the flux by the differential area, we get a value called the irradiance, measured in watts/meter 2 We thus model this using Spherical Harmonic functions rest of PRT based on paper - Spherical Harmonic Lighting: The Gritty Details, Robin Green[4]

CRICOS No. 000213J a university for the world real R Spherical Harmonics (SH) SH lighting paper assumes knowledge of the use of Basis Functions. Basis Functions are small pieces of signal that can be scaled and combined to produce an approximation to original function Process of working out how much of each basis function to sum is called Projection. To approximate a function using basis functions we must work out a scalar value that represents how much the original function f(x) is like the each Basis Function B i (x). We do this by integrating the product f(x)B i (x) over the full domain of f use summation for an approximation

CRICOS No. 000213J a university for the world real R Legendre Polynomials The associated Legendre polynomials are at the heart of the Spherical Harmonics, a mathematical system analogous to the Fourier transform but defined across the surface of a sphere SH functions in general are defined on Imaginary Numbers but we are only interested in approximating real functions over the sphere (i.e. light intensity fields) Lecture will be working only with the Real Spherical Harmonics

CRICOS No. 000213J a university for the world real R SH series for varying values of l and m

CRICOS No. 000213J a university for the world real R Projecting into SH Space Note how the first band is just a constant positive value If you render a self-shadowing model using just the 0- band coefficients the resulting looks just like an accessibility shader with points deep in crevices (high curvature) shaded darker than points on flat surfaces See Lecture 8 The l = 1 band coefficients cover signals that have only one cycle per sphere and each one points along the x, y, or z-axis Linear combinations of just these functions give us very good approximations to the cosine term in the diffuse surface reflectance model

CRICOS No. 000213J a university for the world real R Projection Process Process for projecting a spherical function into SH coefficients is simple Calculate a single coefficient for a specific band you just integrate the product of your function f and the SH function y Working out how much your function is like the basis function Slide 58

CRICOS No. 000213J a university for the world real R Example Complex SH Function Approximations

CRICOS No. 000213J a university for the world real R Approximating Lighting using SH To project a function into SH coefficients we want to integrate the product of the function and an SH function We must evaluate this integral using Monte Carlo integration where x j is our array of pre- calculated samples (Slide 59) and the function f is the product f(x j ) = light(x j )y i (x j ). An example lighting function displayed as a color (left) and a spherical plot (right) Example (left) of a physically sampled scene using a spherical silver ball and a camera

CRICOS No. 000213J a university for the world real R Spherical Harmonic Sampling Basis Uses a set of ortho-normal basis functions on the surface of a sphere just like X,Y,Z axes Rendering equation that we want to integrate over the surface of a sphere So all we need to do is generate evenly distributed points (more technically called unbiased random samples) over the surface of a sphere. Taking a pair of independent canonical random numbers x and y we can map this square of random values into spherical coordinates using the transform This forms a light sample basis for the later calculations

CRICOS No. 000213J a university for the world real R Generating SH Coefficients Applying this process to the light source we defined earlier with 10,000 samples over 4 bands gives us this vector of coefficients: [ 0.39925, - 0.21075, 0.28687, 0.28277, - 0.31530, - 0.00040, 0.13159, 0.00098, - 0.09359, - 0.00072, 0.12290, 0.30458, - 0.16427, - 0.00062, - 0.09126 ] Reconstructing the SH functions for checking purposes from these 15 coefficients is simply a case of calculating a weighted sum of the basis functions: An example approximated lighting function displayed as a color and a spherical plot

CRICOS No. 000213J a university for the world real R Can also model shadows with SH The transfer function (occlusion factor) in the lighting model can also be stored as a 16 coefficient SH function As you can see this is a different way of recording shadowing at a point on a model without forcing us to do the final integral Can rotate object and still get correct values for shadows on surface without recasting the rays an improvement on last lecture approach Light Distribution Final Light Distribution Transfer (Occlusion)

CRICOS No. 000213J a university for the world real R Indirect Lighting Steps Geometrically, the idea of interreflected light is simple Each point on the model already knows how much direct illumination it has, encoded in the form of an SH transfer function Fire rays to find sample points that can reflect light back onto our position and add a cosine weighted copy of that transfer function back into our own For example, point A in the illustration above has fired a ray and hit point B

CRICOS No. 000213J a university for the world real R Shadowed Indirect Lighting A rendering using 5th order diffuse shadowed SH transfer functions Note the soft shadowing from the constant hemisphere light source NB: This is how light probes work in Unity 5

CRICOS No. 000213J a university for the world real R Real-Time Rendering using SH Now we have a set of SH coefficients for each vertex, how do we build a renderer using current graphics hardware that will give us real-time frame rates? Basic calculation for SH lighting is the dot product between an SH projected light source and the SH transfer function vertex Approximate the complete solution over an object by filling in the gaps between vertices using Gouraud shading Typically only need 4 coefficients per vertex one extra 4 value colour

CRICOS No. 000213J a university for the world real R Real-Time Rendering using SH Can rotate the SH coefficients so that the object is able to have its first pass lighting updated in real time Now have a cheap rendering technique for generalised area light sources in a scene Can add a specular term and/or environment maps to give required appearance

CRICOS No. 000213J a university for the world real R Direct X Browser PRT Demonstration Within the DirectX SDK is a demo browser Run the PRT demo to see similar to the movie on the left Choose scene 4 to obtain the head figure

CRICOS No. 000213J a university for the world real R Radiosity and Unity 5 For years this has been a grand challenge of CG, to run radiosity in real time Enlightens implementation [3] performs a partial version in real time on a tablet in Unity 5!!! See here https://www.youtube.com/w atch?v=Wrt5aLHI8MEhttps://www.youtube.com/w atch?v=Wrt5aLHI8ME

CRICOS No. 000213J a university for the world real R Unity 5 GI Basics [5] Unity GI performed in background and baked on for static objects Precomputed Realtime GI encodes all possible bounces into lightmap data structure texture [6] Directional Light is bounced into the scene via this data structure in real time due to no position requirements

CRICOS No. 000213J a university for the world real R Dynamic Objects and GI Dynamic objects cannot cast light into the scene But they can use light probes to sample the bouncing light in the scene to be lit themselves Light probes are placed where dynamic objects are moving and are a spherical panoramic view of the environment

CRICOS No. 000213J a university for the world real R Dynamic Objects and GI Light probes are stored as Spherical Harmonic approximations Interpolated internally to light dynamic object at that point Indirect, probe and direct light blended to provide final lighting model in scene

CRICOS No. 000213J a university for the world real R Unity 5 Demonstration Video Video: https://youtu.be/G-dpRCdfFc8

CRICOS No. 000213J a university for the world real R References 1.John F. Hughes, Andries van Dam, Introduction to Radiosity, www.cs.brown.edu/courses/cs123/lectures.shtml 29/04/2007 2.J Kajiya. The Rendering Equation. SIGGRAPH 1984, pp. 143- 150 3.www.geomerics.com 29/04/2007 4.Robin Green, Spherical Harmonic Lighting: The Gritty Details - www1.cs.columbia.edu/~cs4162/slides/spherical-harmonic- lighting.pdf www1.cs.columbia.edu/~cs4162/slides/spherical-harmonic- lighting.pdf 5.http://docs.unity3d.com/Manual/GIIntro.html 04/05/2015http://docs.unity3d.com/Manual/GIIntro.html 04/05/2015 6.http://www.geomerics.com/wp- content/uploads/2014/03/radiosity_architecture.pdfhttp://www.geomerics.com/wp- content/uploads/2014/03/radiosity_architecture.pdf

Queensland University of Technology CRICOS No. 000213J INB382/INN382 Real-Time Rendering Techniques...

Documents

Transcript of Queensland University of Technology CRICOS No. 000213J INB382/INN382 Real-Time Rendering Techniques...