Introduction to Real-Time Rendering

Koray Hagen

The agenda

1. Mathematics in three dimensions

2. The graphics pipeline

3. Simulation of light

4. Closing thoughts

Mathematics In Three DimensionsCoordinate systems, primitives, and affine transformations

Coordinate Systems

Primitive Types and Topologies

• A primitive is the most basic type of three dimensional object.

• Primitives are defined by a set of vertices.

• The type of primitive you end up with is defined by the method of connections used between vertices.

• The relation of these connections is referred to as the topology.

Primitive Topology – Point List

• A point is created for each vertex in the vertex set.

Primitive Topology – Line List

• A line is created for each vertex pair in the vertex set.

Primitive Topology – Line Strip

• A line is created between each of the vertices

• The line is continuous

Primitive Topology – Triangle List

• A triangle is created for each vertex triplet in the vertex set

Primitive Topology – Triangle Strip

• A triangle is created for each vertex triplet in the vertex set

• For each subsequent vertex in the vertex set, a triangle is created using that vertex and the previous two vertices in the vertex set

Triangles Are Special

• All vertices in a triangle are coplanar, meaning that the triangle is a planar shape.

• A triangle is the simplest primitive that creates a plane.

• We can approximate any other primitive using triangles.

Polygon MeshesConstructing and representing three dimensional objects

Constructing a Polygon Mesh

• All three dimensional objects are made up of a collection of triangle primitives.

• This collection of primitives is referred to as a polygon mesh.

• Each primitive is made up of three vertices. This means that a three dimensional mesh is a simply a large vertex set.

Model Space

• In order to correctly position all vertices we first need a frame of reference.

• Each mesh has its own coordinate system called a space.

• All vertices defined in the model are defined according to that space, and so that space is known as model space.

World Space

• A scene of objects also has its own frame of reference known as the world space.

• Every mesh within this scene has a position relative to the scene’s coordinate system.

• In order to transform a coordinate from model space to world space we will need the world matrix. But more on this later.

Transforming 3D Objects

• To deal with rotation and positioning of objects, we need to transform an object’s coordinate system.

• Since an object is defined around its coordinate system, transforming it will result in every point defined around that system being adjusted as well.

TransformationsLinear, affine, and projective transforms

Linear Transforms

• A linear transform is one that preserves vector addition and scalar multiplication.

• Scaling and rotation can be represented as linear transforms but translation cannot:

• Linear transforms can be represented by a 3 x 3 matrix in three dimensional space, but translation cannot so we must use homogeneous coordinates and the use of 4 x 4 matrices.

Homogeneous Coordinates

• Homogeneous coordinates for an Rn space are defined in an Rn+1 space. So for three dimensions, we use four dimensional coordinates (x, y, z, w)

• Homogeneous simply means “same type” and in this case it refers to the fact we can define both points and vectors using the same notation.

• The w element denotes a point if is set to 1, or a vector if it is set to 0– Point (x, y, z, 1)– Vector (x, y, z, 0)

• We are now working in four dimensional space, so we will need to use a 4 x 4 matrix to represent all transforms.

Affine Transforms

• To deal with the non-linearity of the translation transform we make use of affine transformations.

• An affine transformation is a transformation which preserves straight lines (i.e. all points lying on a line initially still lie on a line after a transformation – also called collinearity) and ratios of distances between points lying on a straight line.

• Affine transformations are represented by 4 x 4 matrices using homogeneous coordinates, which are then multiplied by all the points that need to be transformed.

Rotation

• The rotation transform rotates a vector by an angle around a given axis passing through the origin.

• Angles are represented in radians

Translation

• Translation is the change of location and is represented by the matrix T which translates an object by the vector t where t is (tx, ty, tz).

• First row represents a change on the X axis• Second row is a change on the Y axis• Third row is a change on the Z axis

• Translation is a rigid body transform

Scaling

• Scaling is used to enlarge or shrink an object using scaling factors represented by the matrix S which scales an object by the vector s where s is (sx, sy, sz).

• First row represents a change on the X axis• Second row is a change on the Y axis• Third row is a change on the Z axis

Concatenation of Transforms

• Matrix multiplication is non-commutative, so the order of concatenation of affine transformation matrices is very important.

• As an example, if we wanted to rotate, then translate, and finally scale, the complete transform would be:

– C = R T S

View Transform

• The viewing transform locates the viewer in world space, transforming vertices into camera space.

• In camera space, the camera, or viewer, is at the origin looking in the positive z direction – which utilizes a left handed coordinate system – or the negative z direction – which utilizes a right handed coordinate system.

• The view matrix relocates the objects in the world around the camera’s position – the origin of the camera space – and orientation.

• To calculate the view matrix, we can combine a translation matrix with rotation matrices for each axis, and typically in most graphics libraries the following general matrix equation applies.

– V = T Rz Ry Rx

Projective Transforms

• The projection matrix is typically a scale and perspective projection. You can think of the projection transformation as controlling the camera's internals; it is analogous to choosing a lens for the camera. This is the most complicated of the three transformation types.

• The projection transformation converts the viewing frustum into a cuboid shape. Because the near end of the viewing frustum is smaller than the far end, this has the effect of expanding objects that are near to the camera; this is how perspective is applied to the scene.

• Perspective Projection Orthographic Projection

What tools do we have now?

• The understanding of primitives and topology, and how three dimensional objects in space are constructed.

• The application of homogeneous coordinates , linear, and affine transformations, which allows us to define relative and concrete positions and transforms in space for objects within a scene.

• How projective transformation allows us to view three dimensional objects in a scene on a two dimensional plane (i.e. our monitor’s screen as an example).

The Graphics PipelineProgrammable shaders and GPU architecture

Shaders

• A shader is a computer program that runs on the graphics processing unit and is used to do shading - the production of appropriate levels of light and darkness within an image - or, in the modern era, also to produce special effects or do post-processing.– Vertex shaders run a shader program per-vertex that is streamed.– Pixel shaders, run per-pixel on a given frame.

• The position, hue, saturation, brightness, and contrast of all pixels, vertices, or textures used to construct a final image can be altered on the fly, using algorithms defined in the shader, and can be modified by external variables or textures introduced by the program calling the shader.

Theoretical Graphics Pipeline

• The Application Stage:– Programs are written in a high level language such as C++, using a graphics library such

as DirectX or OpenGL. This program executes on the CPU.

– The portion of the program controls and configure future stages of the pipeline; including but not limited to blending, clipping, testing, etc.

– It is responsible for creating, initializing, and updating all scene elements such as lights, cameras, geometry, textures, etc.

– This stage sends geometric data (vertex buffers) down the pipeline using draw calls.

Theoretical Graphics Pipeline

• The Geometry Stage:– This stage modifies the geometric data (vertices) sent down the pipeline by applying

geometric transformations, vertex lighting, etc.

– The vertex and geometry shaders play a large role in the geometry stage. They are written in shader languages such as GLSL for OpenGL, HLSL for DirectX, and other languages such as CG.

– This stage is responsible for view projection, clipping, and screen mapping.

Theoretical Graphics Pipeline

• The Rasterizer Stage:– This stage is responsible for producing the displayed image. It does this by converting 3D

geometric data into a 2D color image.

– The pixel shader calculates color values for pixels. It is also written (like the vertex shader) in higher level shader languages, and is executed by the GPU. It is responsible for all texturing and per pixel lighting operations.

– This stage also performs fragment testing, and pixel blending.

LightSimulating reflection and shading models

Reflection Model

• A reflection model is a mathematical representation of how a surface reflects any light rays hitting the surface – also known as incident rays.

• Reflection models take in a light ray and surface, and return the final light reflection at the surface of the incident ray.

• Reflectance Spectra (also known as the albedo)

Shading Model

• A shading model determines how a primitive’s surface is colored (shaded).

• If the shading is done per-vertex, then the colors are set at each vertex and the final color will be a linear interpolation of the vertex colors.

• If it is done per-pixel, then the color is set separately at each pixel on the surface, which has much higher fidelity, but is also far more expensive to calculate.

Light Intensity

• There are many different types of light sources, which will be covered in the next slide.

• It is important to note that light sources all have various types of interesting properties, yet they share a common property – their intensity.

• The intensity of a light source can be thought of as its brightness, although in the real world intensity is a far more complex subject. We will be treating brightness as an approximation of its true nature in the physical world.

• The intensity of a light can hold a value from 0 to 1 and is separate for each of the primary colors, red, green, and blue.

Sources of Light

• Point Lights– Radiates light in all directions (like a light bulb) and has both an intensity

and a position. They also exhibit attenuation, which means that the light intensity decreases with distance.

• Directional Lights– Defined by a direction and a light intensity, the light direction is the same at

any point on any surface, this occurs when you have a light source that is infinitely far away (like the sun).

• Spot Lights– These lights are point lights that only emit light in a directed cone (like a

flash light). There are various ways to define spotlights but at the most basic level they have a position, intensity and a cone direction.

Surface Reflectance

• Diffuse Reflection– The reflection of light from a surface such that an incident

ray is reflected at many angles rather than at just one angle. It originates from a combination of internal scattering of light, i.e. the light is absorbed and then re-emitted, and external scattering from the rough surface of the object.

• Specular Reflection– The mirror-like reflection of light from a surface, in which

light from a single incoming direction (a ray) is reflected into a single outgoing direction.

Surface Material

• Every surface is made of some material and each material reflects light differently– Think of how metal objects are shiny and wooden objects are matte. We need to have a way to

specify material parameters that can control how a surface reflects light.

• Every surface therefore has three reflectivity constants, and they control the intensity of the various reflections

Ka ambient reflectivity

Ks specular reflectivity

Kd diffuse reflectivity

a specular highlighting

Calculating Light Intensity

• The picture on the right contains all the necessary vectors to calculate the light intensity at point P– The normal vector N to the surface– The light vector L from the surface– The view vector V (camera position)

• Assuming that the surface is perfectly diffuse, we can use Lambert’s Law which states that the diffuse reflection’s intensity is proportional to the cosine of the angle (theta) between the incoming light ray L and the normal of the surface N.

• Since both the light L and the normal N vector are unit vectors, then the cosine of the angle between them can be represented by their dot product.

Phong and Blinn-Phong Light Intensity

• Diffuse Reflection value Ld derived from Lambert’s Law

– Ld = Kd * dot(N, L) * light source intensity

• Specular Reflection value Ls

– This value is dependent on the view and reflection vector, since the closer the two vectors are, the stronger the intensity will be.

– Must take into account smoothness of surface, since smoother surfaces will result in a tighter arc of reflected light. We can approximate this by use of an exponent in the intensity calculation.

– Phong equation Ls = Ks * exp(dot(R, V), a) * light source intensity

– Blinn-Phong equation Ls = Ks * exp(dot(N, H), a) * light source intensity

• Ambient light value La

– La = Ka * ambient light intensity

• Final light intensity equals the sum of La + Ls + Ld

HLSL Lighting Functions

HLSL Per-Vertex Blinn-Phong Shading

HLSL Per-Pixel Blinn-Phong Shading

HLSL Shader Results

Conclusion

• Real-time rendering is an extremely math-heavy area of computer programming and computer science.

• Knowledge in graphics may take years to attain and ultimately master.

• A pioneering and intellectually rewarding field of study, that is coming out with new developments every day – literally.

• For more information, please feel free to contact me a koray.hagen@gmail.com.

• My website: www.korayhagen.com