Graphic pipeline OpenGL, DirectX etc · 2 Computer Graphics Graphic pipeline Approach that is...

Computer Graphics

Graphic pipelineOpenGL, DirectX etc …

Computer Graphics

Graphic pipeline

Approach that is complementary to ray-tracing

Lots of different implementations Software implementation e.g. Pixar Renderman

Goal here : speed but also some realistic rendering (very flexible) Do a 2 hours movie in 1 year imposes leaves less than 3 minutes of CPU time

per image … (but it is easily run un parallel) Hardware implementations, e.g. PC graphic cards

Almost realtime : rendering of many millions of triangles per second

In what follows, we will restrict ourselves to an abstraction of an hardware implementation

Computer Graphics

Graphic pipeline

Goal here : operations are easily parallelized or vectorized Groups of dedicated GPUs that are able to execute dozens to thousands ot

operations in parallel

→ explains why GPUs are so powerful ( many times that of generic CPUs à 1/5 of the clock speed)

Dedicated high-bandwith memory

Computer Graphics

Graphic pipeline

Bus performance (dates back to 2010 but ideas are same today)

Computer Graphics

Graphic pipeline

GPU specific architecture

Computer Graphics

Graphic pipeline

Pipeline

Command flow

Transformed geometry

Fragments ( ~ pixels + interpolated data)

framebuffer

Application

vertex processing

Raster conversion

Operations on fragments (fragment processing)

Display

We are here

3D Transformations shading

Compositing, mixing, (shading)

What the user sees

« Langage » standards OpenGL, DirectX etc...

Conversion of primitives into

fragments

Computer Graphics

Graphic pipeline

Primitives Points (in space)

Line segments Polylines

Triangles Triangle fans / strips

That’s all ! Curves ? Transformed into polylines Polygones ? Decomposed into triangles Curved surfaces ? Approximated with triangles

The current « trend » is the restrict to a minimal number of primitives Simple, uniform, repetitive → good for vectorial processing

Computer Graphics

Graphic pipeline

Command flow : depends on the implementation

e.g. OpenGL 1, DirectX, others

Always somewhat similar See OpenGL labs

OpenGL advantages : Multiplatform, simple, high performance, still evolving, and not attached to a

specific architecture or company. It became a standard and is supported everywhere, even on smartphones

Computer Graphics

Graphic pipeline

Pipeline

Command flow

framebuffer

Application

vertex processing

Raster conversion

Display

We are here

What the user sees

fragments

Computer Graphics

Graphic pipeline

Pipeline of geometrical transformations

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Clipping Rasterization expects that primitives are visible on the

screen This is done in the 3D canonical space, after the application of the

perspective projection (see course 2 ), but before the perspective division.

All that is outside the volume limited by :

is discarded

Cut by 6 planes

−wxw−w yw−wzw

Computer Graphics

Graphic pipeline

Clipping Basic operation : cut a triangle into 3 by a plane

4 cases : All vertices inside – tr kept

All verticels exteriortriangle discarded

1 vertex in; 2 out → 1 triangle remains

2 vertices in; 1 out → 2 triangles remain

Computer Graphics

Graphic pipeline

Hidden face removal We have seen how to geometrically transform the primitives

to the screen

The perspective projection gives a strong hint on depth

Hidden face removal is another strong hint

Allows to draw exclusively what is seen (goal : performance)

Computer Graphics

Graphic pipeline

Hidden face removal Backface culling

For closed and opaque shapes, one does not see the inside !

Computer Graphics

Graphic pipeline

No need to draw the backward-facing faces

Computer Graphics

Graphic pipeline

No need to draw the backward-facing faces

v⋅n≤0

Computer Graphics

Graphic pipeline

Dépends ont the convention used to compute the normal to the shape

Usually, exterior pointing vector

Computation is easy if triangles are carefully oriented !

n=s1 s2×s1 s3

∥s1 s2×s1 s3∥

Computer Graphics

Graphic pipeline

Hidden face removal How to care for this case ?

Painter’s algorithm

Binary space partitionning

Z-buffer

Computer Graphics

Graphic pipeline

Pipeline

Command flow

framebuffer

Application

vertex processing

Raster conversion

Display

We are here

What the user sees

fragments

Computer Graphics

Graphic pipeline

Raster conversion First stage : enumerate pixels that are covered by a

« continuous » primitive

Second stage : Interpolate values that are known on the vertices of the primitive

Example : the colour known at the vertices of a triangle may be distributed on each pixel covered by the triangle

Other variables may also be interpolated. Normal vectors for instance...

Computer Graphics

Graphic pipeline

Raster conversion Transformation of continuous primitives into discrete pixels

Example : drawing of a line

Difficulty : aliasing

Computer Graphics

Graphic pipeline

Naïve algorithm Line = unit width rectangle

One specifies beginning and end vertices

Case here : black inside and white outside

Computer Graphics

Graphic pipeline

Naïve algorithm Line = unit width rectangle

One specifies beginning and end vertices

Case here : black inside and white outside

Computer Graphics

Graphic pipeline

Point sampling One approximates the rectangle by drawing every pixel

whose center is inside the rectangle

Computer Graphics

Graphic pipeline

Point sampling One approximates the rectangle by drawing every pixel

whose center is inside the rectangle

Problem : sometimes pixels with more that one adjacency are turned on

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Bresenham algorithm (midpoint alg.) We will define the thickness with respect to the y axis...

Computer Graphics

Graphic pipeline

One turns on only one pixel per column

45° slanted lineswill appearthinner.

Computer Graphics

Graphic pipeline

45° slanted lineswill appearthinner.

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Bresenham algorithm (midpoint alg.) Equation :

Evaluated for everu column

0≤m≤1for x = ceil(x0) to floor(x1) y=b+m*x plot(x,round(y))

y=0.49x−0.01

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Computer Graphics

Graphic pipeline

Optimization Multiply and rounding are rather slow operations (at least on

primitive CPUs)

For each pixel, the only options are E and NE One computes the error :

d > 0.5 decidesE or NE

d=m x1b− y

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

y=mxb mxb− y=0

Computer Graphics

Graphic pipeline

Optimization One must only update integer steps along x and y

Exclusive use of the addition (no mult or divide)

d > 0.5 decidesE or NE

d=m x1b− y

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

d=dmd =d −1

Computer Graphics

Graphic pipeline

Optimization

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

x=ceil(x0) // round to ceily=round(m*x+b) // round to nearestd=m*(x+1)+b-yWhile (x<floor(x1)) // round to floor{ If d>0.5 { y=y+1 d=d-1 } x=x+1 d=d+m plot(x,y)}

Computer Graphics

Graphic pipeline

Generally, endpoints are given

Implicit form :

y=dydx

⋅x+b x0

dx=x1−x0

dy= y1− y0

G x , y =dy⋅x−dx⋅ydx⋅b

G x , y0

G x , y 0

G x , y =0

≡F x , y=2 dy⋅x−2 dx⋅y2 dx⋅b

Computer Graphics

Graphic pipeline

What is the value of F at M ?

: M is under the linego to NE

: M is abovego to E

d i=F x i1, yi1/2=2 dy⋅x i1−2 dx⋅ y i1 /22 dx⋅b

d i≤0

x i , y i

Computer Graphics

Graphic pipeline

First point: what is the value of ?

As belongs to the line :

d 0=F x01, y01 /2=2 dy⋅ x01−2 dx⋅ y01/22 dx⋅b

=2 dy x0−2 dx⋅y02 dx⋅b2 dy−dx

=F x0, y02 dy−dx

x0, y0 F x0, y0=0

d 0=2dy−dx

Computer Graphics

Graphic pipeline

Recursion : what is the value of ?

If then go to E :

Otherwise go to NE :

d i≤0

x i1 , y i1=x i1 , y i

x i1 , y i1=x i1 , y i1

d i1=F x2, y1/2=d i2 dy

d i1=F x2, y3 /2=d i2dy−2 dx

Computer Graphics

Graphic pipeline

Algorithm is valid only for one octant

Bresenham( x1, y1, x2, y2 ){

dx = x2 – x1dy = y2 – y1d = 2*dy – dxplot( x1, y1 )while (x1 < x2 ){

if (d <= 0) // EASTd = d + 2*dy

else // NORTH-EAST{

d = d + 2*(dy-dx)y1 = y1 + 1

}x1 = x1 + 1plot( x1, y1 )

Computer Graphics

Graphic pipeline

What to do if the line is not in the right octant ? Exchange x and y

Exchange start and end points

Replace y by -y

Computer Graphics

Graphic pipeline

Same type of algorithm do exist for other geometric shapes : e.g. circles (see litterature

What about aliasing ? Algorithm derived from Bresenham that avoid aliasing do exist, see algorithm

of Xiaolin Wu

Oversampling → sampling on a finer grid then averaging on bigger pixels is also an option.

Bresenham, Jack E. "Algorithm for computer control of a digital plotter", IBM Systems Journal, Vol. 4, No.1, pp. 25–30, 1965

Wu, Xiaolin. "An efficient antialiasing technique". Computer Graphics 25 (4): 143–152, 1991.

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Interpolation Some variable is known at the vertices of the triangle (color,

normal vector, etc...)

One wish to get a representative value of the same variable along the line for every pixel that is turned on.

A progressive variation would be fine, thus a linear interpolation is the right tool here :

2D,3D : a is simply the distance ratio to the endpoints.

f x=1− f 0 f 1

=x−x0/ x1−x0

Computer Graphics

Graphic pipeline

Interpolation The pixels are not exactly on the line

One defines a projection on the line It is linear

One may useresults obtainedbefore to construct an interpolation

=v x / L=v y / L

=v⋅Q−P0/ L

L=v⋅P1−P0

Computer Graphics

Graphic pipeline

Interpolation The pixels are not exactly on the line

One defines a projection on the line It is linear

One may useresults obtainedbefore to construct an interpolation

=v x / L=v y / L

=v⋅Q−P0/ L

L=v⋅P1−P0

Computer Graphics

Graphic pipeline

Alternative meaning d and a are updated from pixel to the next pixel

d tells us at which distance we are to the line

a tells the position along the line

d and a are coordinates in the natural frame of the line

Computer Graphics

Graphic pipeline

Alternative meaning The loop means the pixels that we visit

Interpolation of d and a at each pixel

A fragment is emmitted if the pixel’s center is in the band

Interpolationbecomes themain operator

Computer Graphics

Graphic pipeline

Alternative meaning

x=ceil(x0)y=round(m*x+b)d=m*(x+1)+b-y etc...while (x<floor(x1)){

if (d>0.5){

y=y+1d=d-1 etc ...

} else

{x=x+1d=d+m etc...

}If (-0.5 < d <= 0.5)

plot(x,y, … )}

Computer Graphics

Graphic pipeline

Triangle raster conversion Very common case

With a good antialiasing, may be the only case ! Some systems represent lines with two very thin triangles

Triangle represented with 3 vertices

The algorithm has the same philosophy as for lines : One walks from pixel to pixel

Linear operators are evaluated for each step

Those operators allow us to know if a pixel is inside or outside

Computer Graphics

Graphic pipeline

Triangle raster conversion

Input :

Three 2D points

Variables to interpolate, at each vertex

Output : a list of fragments, with :

Integer coordinates of pixels

Interpolated variables

x0 , y0 ; x1 , y1; x2 , y2

q00 ,⋯, q0n ; q10 ,⋯, q1 n ; q20 ,⋯ , q2 n

q0 ,⋯ , qn

Computer Graphics

Graphic pipeline

Triangle raster conversion Incremental evaluation of linear functions on a grid of pixels

Functions are defined by the value at the vertices

Use of additionalfunctions to determinethe set of fragments to return back

x0 , y0

q00 ,⋯ , q0n

x1 , y1

q10 ,⋯ , q1n

x2 , y2

q20 ,⋯ , q2n

(x , y)

q0 ,⋯ , qn

Vertex

Fragment

Computer Graphics

Graphic pipeline

Incremental linear interpolation A linear affine function in the plane :

Evaluation on a grid is efficient :

q x , y =c x xc y yck

q x1, y=c x x1c y yck=q x , yc x

q x , y1=c x xc y y1ck=q x , y c y

Computer Graphics

Graphic pipeline

Interpolation of variables known at vertices Determine defining the unique linear function that

gives back the value at the vertices

3 parameters , 3 equations :

Singular system if points are collinear

cx , c y , ck

c x x0c y y0ck=q0

c x x1c y y1ck=q1

cx x2c y y2ck=q2x0 y0 1x1 y1 1x2 y2 1

Computer Graphics

Graphic pipeline

Interpolation of variables known at vertices Translation of the origin to

2X2 linear system

Solution using Cramer’s rule

x0 , y0

q x , y=c x x−x0c y y− y0q0

q x1, y1=c x x1−x0c y y1− y0q0=q1

q x2, y2=c x x2−x0c y y2− y0q0=q2

x1−x0 y1− y0

x2−x0 y2− y0cx

c y=q1−q0

q2−q0

c x= q1 y2− q2 y1/ x1 y2− x2 y1

c y= q2 x1− q1 x2/ x1 y2− x2 y1

Computer Graphics

Graphic pipeline

What are the fragments to consider ? Those for which the barycentric coordinates are positive …

Algebraically, on has

Inside iff

Pineda, Juan, " A parallel algorithm for polygon rasterization" Computer Graphics 22 (4): 17-20, 1988

, , p= a b c=1

0 ; 0 ; 0

Computer Graphics

Graphic pipeline

Barycentric coordinates are interpolated variables Each barycentric c. yields 1 on a specific vertex and 0 on all others.

They are an implicit representation of the sides of the triangle.

Computer Graphics

Graphic pipeline

Pixel-per-pixel raster conversion (Pineda’s algorithm - 1998)

One vists conservatively a superset of the pixels

Use of interpolation of linear functions

Use of barycentric coordinates todetermine whento emit a fragment

x0 , y0

q00 ,⋯ , q0n

x1 , y1

q10 ,⋯ , q1n

x2 , y2

q20 ,⋯, q2n

Computer Graphics

Graphic pipeline

Triangle raster conversion Beware of rounding and arbitrary decisions

One has to visit every pixel at least once... (otherwise there may be a hole !)

But not two times ! (Those pixels would take an arbitrary colour that depend in which order things are drawn)

Elegant solution : antialiasing...

Computer Graphics

Graphic pipeline

Pipeline

Command flow

framebuffer

Application

vertex processing

Raster conversion

Display

We are here

What the user sees

fragments

Computer Graphics

Graphic pipeline

Z-buffer (depth buffer) In many application, sorting with respect to depth is too

costly The order changes with the viewpoint

There exists the BSP tree Viewpoint independent but :

Heavy datastructures (difficult to implement « in silico ») Cutting of primitives Slow building Non-incremental (all data have to be known in advance)

Computer Graphics

Graphic pipeline

Solution that is usually favoured : draw in any order and keep track of the closest drawn pixel

Use an additional storage, that stores, for each pixel, the smallest depth to date

When one ought to draw a new pixel, the actual depth is compared to the stored depth and if the latter is greater, then the pixel is drawn and the depth is updated.

Computer Graphics

Graphic pipeline

Z-buffer

This is an axemple of a « brute-force » approach, it works because memory is cheap and fast.

Computer Graphics

Graphic pipeline

Z-buffer Evidently limited to bitmap images (not vectorial)

Somewhat more difficult to implement with transparency (alpha channel)

Computer Graphics

Graphic pipeline

Z-buffer and alpha channels One separates opaque objects and translucent ones

1) Opaque objects are drawn with an update of the z-buffer.

2) Then , use a BSP tree for partially transparent entities

3) Draw transparent entities using the BSP ordering, while takin the Z-buffer into account, but without updating it

Thus, transparent faces behind opaque ones are not drawn (they are not visible)

Computer Graphics

Graphic pipeline

Z-buffer : limited accuracy The supplementary channel is generally encoded as an

integer, as are the color channels

Cause : hardware implementation (simple and fast)

It is possible to have distinct objects seen at the same depth if the value that is stored in the Z-buffer is the same

The accuracy is spread between n (near plane) and f (far plane)

These were used to define the observable volume, see course 2.

0< z*< N −1

Computer Graphics

Graphic pipeline

z*=N −1

Plane n Plane f

Z-buffer

Computer Graphics

Graphic pipeline

Z-buffer Let us choose an integer with b bits (8 or 16...)

What is then the accuracy of the z-buffer?

If the z that is stored is proportional to the actual distance (case of orthogonal projections):

We have N=2b layers for a distance equal to f-n Therefore, the accuracy (independent to the distance!) is

: to maximize the accuracy, f-n must be kept small.

If a perspective projection is used, the z stored in the z-buffer is proportional to the result obtained after the perspective division.

Therfore, the size of the layers depends on depth By how much ?

Δ z=( f −n)

Computer Graphics

Graphic pipeline

z*=0 z*

=N −1

Plane n Plane f

Orthographic proj.

Perspective proj.

Z-buffer

Computer Graphics

Graphic pipeline

Z-buffer

In the second course, we have computed zc after the

perspective divide :

x , y , z ,1⋅2∣n∣

r−l0 0 0

02∣n∣

t−b0 0

rlr−l

tbt−b

∣ f ∣∣n∣

∣n∣−∣ f ∣−1

0 02∣ f ∣∣n∣∣n∣−∣ f ∣

M proj _ persp _ OpenGL

= X ,Y ,∣ f ∣∣n∣∣n∣−∣ f ∣

⋅z2∣ f ∣∣n∣∣n∣−∣ f ∣

,− z

z c=∣ f ∣∣n∣∣ f ∣−∣n∣

2∣ f ∣∣n∣

z ∣ f ∣−∣n∣

z c=f nf −n

z f −n

Computer Graphics

Graphic pipeline

Z-buffer

The interval for zc is 2 ( -1 to 1) and one still have N layers.

What is the size of the layers ? Need to solve for z (invert)

The largest layer is for z=f :

zc=f nf −n

z f −n zc≈

2 z f n

z2 f −n

f −n

N f n zmax=

f f −n

Computer Graphics

Graphic pipeline

Z-buffer Example: n = 1 m, f = 100 m, the z-buffer has 8 bits →

N=256, what is the actual size of the layers ?

With n=10 :

f −n

N f nΔ zmax=

f ( f −n)

100⋅99256⋅1

=39 m (0.15m with 16 bits)

zmin=n f −n

1⋅99256⋅100

=0.0039 m

Δ zmax=f ( f −n)

100⋅90256⋅10

=3.5 m (0.013 m with 16 bits)

zmin=n f −n

10⋅90256⋅100

=0.035 m

Computer Graphics

Graphic pipeline

Z-buffer For a good accuracy, it is best to increase n and decrease f.

Never ever set n=0 → cancels the z-buffer

Generally, the number of bits b is fixed by the hardware (usually 16, 24 or 32 bits)

The more the better but takes as much memory as the picture (usually 24 bits)

Computer Graphics

Graphic pipeline

Interpolation in perspective projection

projection plane

Projections of the endpoints

Computer Graphics

Graphic pipeline

projection plane

Projection of the middle of the line ≠ middle point of the projected lineLinear interpolation in screen coordinates ≠ interpolqtion in eye space

Computer Graphics

Graphic pipeline

projection plane

Projects in the middle ...

z1z2/2

Computer Graphics

Graphic pipeline

Projection plane

⏟ equidistant on z

Computer Graphics

Graphic pipeline

Projection plane

zc1 z c2

This projects in the middle

zc1z c2/2

Computer Graphics

Graphic pipeline

Projection plane

The depth variable that has to be interpolated (at the pixel level is zc (screen depth)

obtained after perspective divide, and not z, the real coordinate

⏟ equidistant on z c (screen depth)

Computer Graphics

Graphic pipeline

The perspective correction (use of zc instead of z ) aims to

avoid problems for e.g. textures applied on slanted surfaces

F u=1−u F 0u F 1

1−uF 0

1−u1zc0

Computer Graphics

Graphic pipeline

Minimal pipeline « Vertex » stage (input : 3D positions / vertices and colors / triangle)

Position transformation (object space → eye space) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)

Raster conversion Pixel list Transmission of the color

« Fragment » stage (output : color) Display pixels with right colors on the frambuffer

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Minimal pipeline with z-buffer « Vertex » stage (input : 3D positions / vertices and colors / triangle)

Position transformation (object space → eye space) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)

Raster conversion

Interpolation of zc (z in screen space)

Transmission of the color

« Fragment » stage (output : color, zc)

Display pixels with right colors on the frambuffer and update z-buffer only if zc <

current zc

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Flat shading Uses the « real » normal to the triangle

Faceted appearance Most realistic view

of the real geometry (as defined)

Foley et al.

Computer Graphics

Graphic pipeline

Minimal pipeline with z-buffer and flat shading « Vertex » stage (input : 3D positions / vertices and colors / triangle +

normal) Position transformation (object space → eye space) Color computation (flat shading) with the normal (one color / tri) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)

Raster conversion

Interpolation of zc (z in screen space)

Transmission of the color

current zc

Computer Graphics

Graphic pipeline

Flat shading

Computer Graphics

Graphic pipeline

Observer and illumination : close vs. far Phong illumination require some geometric info

Light vector (depends on position)

Obsever vector (depends on position)

Normal to the surface (computed beforehand)

Observer & light vectors are changing Must be computed & normalized for every facet

observer

Computer Graphics

Graphic pipeline

Observer and illumination : close vs. far Case where observer and source are far away

Almost parallel light rays

Almost orthographic projection

Light & observer vectors do not change much

A frequent optimization is to consider they do not change even though it is generally false

Computer Graphics

Graphic pipeline

Directional light (e.g. sun) Light vector is constant

In many cases, it increases dramatically the throughput of the pipeline by simplifying computations

observer

light[ x y z 0 ]

Computer Graphics

Graphic pipeline

Observer at the infinite Orthographic projection ?

Constant projection angle

One may also do that for the perspective projection, only for shading computations

The observer vector is considered constant (for instance, it is normal to the image plane)

Yield strange results if a wide angle view is used

Blinn-Phong shading: observer, light, and bisector vector all constant (see course 4)

Computer Graphics

Graphic pipeline

Directional light & far observer

Only the normal changes. Means the shading of every facet is the same if the orientation is the same.

observer

light[ x y z 0 ]

[xo yo zo 0]

Computer Graphics

Graphic pipeline

Gouraud interpolation One wants a smooth shading eventhough the geometry is

faceted Remember mapping : sensitivity to the normals much greater than the

sensitivity to positions

Idea : colour is computed at the facet’s vertices

Then, an interpolation is performed to get the color everywhere, at same time than the raster conversion.

Computer Graphics

Graphic pipeline

Gouraud interpolation

Foley et al.

Computer Graphics

Graphic pipeline

Pipeline with z-buffer and Gouraud interpolation « Vertex » stage (input : 3D positions / vertices and color per triangle +

normal per vertex) Position transformation (object space → eye space) Color computation made per vertex Position transformation (eye space → screen space)

Raster conversion

Interpolation of zc (z in screen space), rgb color

current zc

Computer Graphics

Graphic pipeline

Pipeline with z-buffer and Gouraud interpolation

Computer Graphics

Graphic pipeline

Gouraud interpolation Normals at vertices

Mathematically undefined

If the tessellation (triangulation) is obtained from a smooth surface (sphere, B-Spline, etc..), one may take the exact normal on that surface at the vertex

If not, just do as if by averaging neigboring triangle normals

∥∑i

N i∥N s

N 3N 4

Computer Graphics

Graphic pipeline

Gouraud interpolation May be applied to any shading model

Diffuse

Blinn-Phong

Etc ...

However, for specular shading, it does not work well There are strong variations of the shading with respect to the position, even

on a single triangle. A linear interpolation is just too basic to reproduce these variations.

Computer Graphics

Graphic pipeline

Blinn shading and with Gouraud interpolation

Foley et al.

Computer Graphics

Graphic pipeline

Phong interpolation Why not interpolate normals and compute shading at the

pixel (fragment) level ? As easy as interpolating colors

The shading will be computed are every pixel, from information that have been interpolated (colors, normals, etc...

In the Graphic pipeline, it means that we move the shading computation from the « Vertex » stage to the « fragment » stage.

Computer Graphics

Graphic pipeline

Pipeline

Command flow

framebuffer

Application

vertex processing

Raster conversion

Display

We are here

What the user sees

fragments

Computer Graphics

Graphic pipeline

Pipeline with z-buffer and Phong interpolation « Vertex » stage (input : 3D positions / vertices , triangle, + normal/color per

vertex) Position transformation (object space → eye space) Position transformation (eye space → screen space)

Raster conversion

Interpolation of zc (z in screen space), rgb color, xyz normal

Shading computation using interpolated data (including normal)

current zc

Computer Graphics

Graphic pipeline

Phong interpolation

Computer Graphics

Graphic pipeline

OpenGL Some shading models are part of the standard (typically :

Phong, Lambert, Gouraud, Blinn-Phong …), Nothing has to be done .

For specific shading computations, there is an API that allows defining exactly what has to be done at each stage :

Either at the « vertex shader » stage, vertex per vertex

Or, at the « fragment shader » stage, pixel per pixel

Recent graphic cards (and the software driver embedded in the operating system) allow to litterally program in pseudo C-language either the « vertex shader » or the « fragment shader » (see e.g. Nvidia CUDA , OpenCL)

Computer Graphics

Graphic pipeline

Programmable !

Not very versatile

Computer Graphics

GPGPU Use of computing power of GPUs to do other things that just

shading ...

Recent GPUs are somewhat versatile (but less than generic CPUs)

Branching became possible

Random memory access

But they are designed for vector computing (i.e. same operations applied to different data, SIMD)

There are standards to drive these GPUs : Open Computing Language (OpenCL, open)

Common Unified Device Architecture (CUDA, Nvidia)

Direct Compute (Microsoft)

Computer Graphics

OpenCL – pseudo-c language Host program may be in C++, and runs on the CPU

The OpenCL code is stored as a character string (char[] or std::string )

The OpenCL code is then compiled by the graphic card’s « driver » (the CPU does the job), result is uploaded on the GPU by dedicated system calls

To access to the machine code, system calls allow to pass parameters and memory chunks to the GPU

The computation is made on the GPU

Results are given back to the host program using memory transfer.

Computer Graphics

OpenCL sample code

__kernel void VectorAdd(__global float* c, __global float* a , __global float* b, __constant float *cst{ // Index of the elements to add unsigned int n = get_global_id(0); unsigned int nl = get_local_id(0); unsigned int gsz = get_global_size(0); unsigned int lsz = get_local_size(0); unsigned int gid = get_group_id(0); // do some math from vectors a and b and store in c __private float res; res=0.; int i; for (i=0;i<100;++i) // not a loop over elements of the arrays { res=a[n]+sin(a[n])+b[n]; if (res>5.5) res=a[n]* (*cst) ; } c[n] = res;}

Computer Graphics

In the host program … init

std::string Src ; // Source code (see preceding slide)

std::vector<cl::Platform> platforms;cl::Platform::get(&platforms);

cl_context_properties properties[] = { CL_CONTEXT_PLATFORM,(cl_context_properties)(platforms[0])(),

cl::Context context(CL_DEVICE_TYPE_ALL, properties);std::vector<cl::Device> devices =

context.getInfo<CL_CONTEXT_DEVICES>();cl::Program::Sources

source(1,std::make_pair(Src.c_str(),Src.size()));

cl::Program program = cl::Program(context, source);

program.build(devices); // compilation !

Computer Graphics

In the host program … system calls

cl::CommandQueue queue(context,devices[0],0,&err);cl::Buffer GPUOutVec(context, CL_MEM_WRITE_ONLY

,sizeof(float)*SIZE, NULL, &err);float cst=1;cl::Buffer GPUVec1(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR

,sizeof(float)*SIZE,HostVec1,&err);cl::Buffer GPUVec2(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR

,sizeof(float)*SIZE,HostVec2,&err);cl::Buffer GPUCst1(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR

,sizeof(float),&cst,&err);cl::Event event1;cl::Kernel kernel(program_, "VectorAdd", &err);kernel.setArg( 0, GPUOutVec);kernel.setArg( 1, GPUVec1);kernel.setArg( 2, GPUVec2);kernel.setArg( 3, GPUCst1);

queue.enqueueNDRangeKernel(kernel,cl::NullRange,cl::NDRange(SIZE_TEST),cl::NullRange,NULL,&event1);

event1.wait();// at this point, the computation has been made.

Computer Graphics

In the host program … collecting data back

Complete code sample on the website of the course

// at this point, the computation has been made.

cl::Event event2;queue.enqueueReadBuffer(GPUOutVec,CL_TRUE,0,

SIZE_TEST*sizeof(float),HostOutVec, NULL, &event2);event2.wait();

// HostOutVec contains the results

for (int Rows = 0; Rows < SIZE_TEST/32; Rows++){

for (int c = 0; c <32; c++) std::cout << HostOutVec[Rows * 32 + c] << " " ; std::cout << std::endl;}

Computer Graphics

OpenCL is a multiplateform paradigm The above code sample may be compiled on any computer (having a

GPGPU enabled driver)

It can even be executed on a ... CPU (when no GPU is available) There exist « dummy » drivers that will simply compile and execute the code on

the same CPU. Useful to debug and benchmark

Performance : factor 2 to 100 in favor of the GPU for vector operations (e.g. on big arrays)

Core i7 with 6 cores 3,33 Ghz vs. Nvidia Quadro FX 580 (not very powerful) : 110 s. vs 55 s.

On a specialized GPGPU graphic card : Nvidia Tesla C2075 : 11 s.

Computer Graphics

Graphic pipeline

Painter’s algorithm Idea : display every primitive in the right order

Everything « below » is overwritten, like when painting

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Amounts to define a topological sorting Find a path in an oriented graph

ABCDEFABDCFECAEBDF...

Computer Graphics

Graphic pipeline

Painter’s algorithm Impossible if cycles are present ...

ABC ???

Computer Graphics

Graphic pipeline

Painter’s algorithm Impossible if cycles are present …

Solution : cut all !

b,l cd,g

Computer Graphics

Graphic pipeline

Painter’s algorithm Useful when an order is easy to define

Works with vector graphics

May be very CPU intensive (cut;sort ..)

Computer Graphics

Graphic pipeline

Painter’s algorithm The ordering depends on the point of view

Sorting primitives is costly ( nlog(n) to the best ) and have to be done every time the observer moves

Primitives must be cut if forming cycles (how to detect ?)

A response to these drawbacks is the binary space partition tree (BSP Tree)

Computer Graphics

Graphic pipeline

BSP Tree Input data :

Segments in 2D

Triangles in 3D – same idea

Computer Graphics

Graphic pipeline

How to build the BSP Tree (1)

→ Take one of the segments and define a line cutting the plane in two separate half-planes→ Classify the other segments with respect to the boundary. If a segment is crossing, partition it and classify its parts→ On each subdomain, if there are more than one segment, repeat the procedure recursively

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

Computer Graphics

Graphic pipeline

How to choose the right segment at each step ? A random choice is not that bad...

Complete algorithm :

Let S be a set of line segments (or triangles in 3D)Build(S,BSP){

If (Card(S) <=1) BSP is a tree with only one node that contains the only segment of S or nothingElse{

Use a random segment s belonging to S as a cutting line and cut all the other segments S+ = segments belonging to H+ (« positive » halfspace) (without s)S- = segments belonging to H- (« negative » halfspace) (without s)Call recursively Build(S+,BSP+)Call recursively Build(S-,BSP-)Build a tree with BSP as a root, and BSP+ and BSP- as children. The root contains s.

Computer Graphics

Graphic pipeline

Use of the BSP Tree How to scan the tree to get the right display order ?

ObserverO

Let O c+ a point (the observer)→ it is clear that entities from c- must be displayed before entities from c, that must be displayed before those of c+.

Computer Graphics

Graphic pipeline

c- c c+The same remark arises for c- (then c+)O b.1- thus entities of b.1+ must be displayed before those on b.1, and before those in b.1-

ObserverO

Computer Graphics

Graphic pipeline

a b.1 (b.1-) c c+For c+ :O d+ thus the order is d- d d+

ObserverO

Computer Graphics

Graphic pipeline

a b.1 (b.1-) c d- d d+

ObserverO

Computer Graphics

Graphic pipeline

a b.1 (b.1-) c e.1 d d+For d+ :O e.2+ thus order is e.2- e.2 e.2+

10101112

ObserverO

Computer Graphics

Graphic pipeline

a b.1 (b.1-) c e.1 d b.2 e.2 (e.2+)

Final order : a b.1 c e.1 d b.2 e.2

10101112

ObserverO

Computer Graphics

Graphic pipeline

Recusive scanning algorithmDraw(BSP,ViewPoint){

If BSP is a leaf (no children)Draw the primitives contained in BSP

Let BSP+ and BSP- - children of BSPIf ViewPoint is in H- (« negative » halfspace){

Call Draw(BSP+,ViewPoint)Draw the primitives contained in BSPCall Draw(BSP-,ViewPoint)

}Else If ViewPoint is in H+ (« positive » halfspace){

Call Draw(BSP-,ViewPoint)Draw the primitives contained in BSPCall Draw(BSP+,ViewPoint)

}Else (we are exactly on the plane...){

(Draw the primitives contained in BSP) / but not necessaryCall Draw(BSP+,ViewPoint)Call Draw(BSP-,ViewPoint)

BSP-BSP+

Computer Graphics

Graphic pipeline

The BSP tree is generally not directly used in graphic cards Building a BSP tree is relatively slow (nlogn to the best), and the

datastructure is not well adapted to vector/parallel treatment.

→ Z-buffer : see further

It can however be used in certain cases in software to ease the work of graphic cards

e.g. « FPS » video games where the environnment is mostly fixed « Doom » is an old video game which used this principle.

Il can be used for ray-tracing...

Graphic pipeline OpenGL, DirectX etc · 2 Computer Graphics Graphic pipeline Approach that is...

Documents

Transcript of Graphic pipeline OpenGL, DirectX etc · 2 Computer Graphics Graphic pipeline Approach that is...

Abc of DirectX

Introduction to DirectX Implementation. Installing DirectX SDK To write and execute DirectX 9.0 programs, you need both : DirectX 9.0 runtime and the.

DirectX 10 Performance and DirectX 11 Preview

DirectX 10 Performance

DirectX Programming #1 - SNUcglab.snu.ac.kr/lectures/08-1/graphics/data/DX1.pdf · CGLab DirectX Layers ` Native DirectX ` COM-based API ` Supports C++ and Visual Basic ` Managed

Parallel Nsight for Accelerated DirectX 11 …on-demand.gputechconf.com/gtc/2010/presentations/S12212-Parallel...Parallel Nsight for Accelerated DirectX 11 ... HUD overlay GPU: DirectX

Prasentation Managed DirectX

Advanced Rendering with DirectX 12® - Nvidiadeveloper.download.nvidia.com/gameworks/events/GDC... · Advanced Rendering with DirectX 12® 2 gameworks.nvidia.com Agenda •DirectX

Window7 - DirectX 11 SDK - VisualStudio2013 Setting 1 ...cau.ac.kr/~bongbong/oop15/DirectX-VS2013Setting.pdfWindow7 - DirectX 11 SDK - VisualStudio2013 Setting 1. Download the sample

Open Standards for Today’s Gaming Industry€¦ · Accelerating OpenGL Innovation DirectX 10.1 OpenGL 2.0 OpenGL 2.1 OpenGL 3.0 OpenGL 3.1 DirectX 9.0c DirectX 10.0 DirectX 11 OpenGL

Far Cry and DirectX

Content Manager - DT Researchdtresearch.com.tw/landing/ds_download/WCM_Platinum...• Microsoft® DirectX 9.0c NVidia, ATI or Intel graphic card, supports 32bpp color depth • •Minimum

Progressive Mesh in DirectX

DirectX Shader Programming

WinRT Async Programming With PPL - jyurimaru.infojyurimaru.info/data/20120602wankuma_osaka49/20120602_sao.pdf · XAML DirectX XAML DirectX XAML

DirectX 11 Terrain Tessellation

C++ Graphics Directx

DirectX 10 - AMDdeveloper.amd.com/.../2013/02/GDC_2008_DirectX10.1.pdf · Introduction to DirectX 10.1 Super-set of DirectX 10 functionality Adds new features Adds new minimum requirements

DirectX 10 - AMDdeveloper.amd.com/wordpress/media/2012/10/DirectX10.1.pdf · Introduction to DirectX 10.1 Super-set of DirectX 10 functionality Adds new features Adds new minimum

GameFX C# / DirectX 2005