Graphic pipeline OpenGL, DirectX etc · 2 Computer Graphics Graphic pipeline Approach that is...
Transcript of Graphic pipeline OpenGL, DirectX etc · 2 Computer Graphics Graphic pipeline Approach that is...
1
Computer Graphics
Graphic pipelineOpenGL, DirectX etc …
2
Computer Graphics
Graphic pipeline
Approach that is complementary to ray-tracing
Lots of different implementations Software implementation e.g. Pixar Renderman
Goal here : speed but also some realistic rendering (very flexible) Do a 2 hours movie in 1 year imposes leaves less than 3 minutes of CPU time
per image … (but it is easily run un parallel) Hardware implementations, e.g. PC graphic cards
Almost realtime : rendering of many millions of triangles per second
In what follows, we will restrict ourselves to an abstraction of an hardware implementation
3
Computer Graphics
Graphic pipeline
Graphic pipeline
Goal here : operations are easily parallelized or vectorized Groups of dedicated GPUs that are able to execute dozens to thousands ot
operations in parallel
→ explains why GPUs are so powerful ( many times that of generic CPUs à 1/5 of the clock speed)
Dedicated high-bandwith memory
4
Computer Graphics
Graphic pipeline
Bus performance (dates back to 2010 but ideas are same today)
Nvi
dia
5
Computer Graphics
Graphic pipeline
GPU specific architecture
Nvi
dia
6
Computer Graphics
Graphic pipeline
Pipeline
Command flow
Transformed geometry
Fragments ( ~ pixels + interpolated data)
framebuffer
Application
vertex processing
Raster conversion
Operations on fragments (fragment processing)
Display
We are here
3D Transformations shading
Compositing, mixing, (shading)
What the user sees
« Langage » standards OpenGL, DirectX etc...
Conversion of primitives into
fragments
7
Computer Graphics
Graphic pipeline
Primitives Points (in space)
Line segments Polylines
Triangles Triangle fans / strips
That’s all ! Curves ? Transformed into polylines Polygones ? Decomposed into triangles Curved surfaces ? Approximated with triangles
The current « trend » is the restrict to a minimal number of primitives Simple, uniform, repetitive → good for vectorial processing
8
Computer Graphics
Graphic pipeline
Command flow : depends on the implementation
e.g. OpenGL 1, DirectX, others
Always somewhat similar See OpenGL labs
OpenGL advantages : Multiplatform, simple, high performance, still evolving, and not attached to a
specific architecture or company. It became a standard and is supported everywhere, even on smartphones
9
Computer Graphics
Graphic pipeline
Pipeline
Command flow
Transformed geometry
Fragments ( ~ pixels + interpolated data)
framebuffer
Application
vertex processing
Raster conversion
Operations on fragments (fragment processing)
Display
We are here
3D Transformations shading
Compositing, mixing, (shading)
What the user sees
« Langage » standards OpenGL, DirectX etc...
Conversion of primitives into
fragments
10
Computer Graphics
Graphic pipeline
Pipeline of geometrical transformations
11
Computer Graphics
Graphic pipeline
12
Computer Graphics
Graphic pipeline
Clipping Rasterization expects that primitives are visible on the
screen This is done in the 3D canonical space, after the application of the
perspective projection (see course 2 ), but before the perspective division.
All that is outside the volume limited by :
is discarded
Cut by 6 planes
−wxw−w yw−wzw
13
Computer Graphics
Graphic pipeline
Clipping Basic operation : cut a triangle into 3 by a plane
4 cases : All vertices inside – tr kept
All verticels exteriortriangle discarded
1 vertex in; 2 out → 1 triangle remains
2 vertices in; 1 out → 2 triangles remain
14
Computer Graphics
Graphic pipeline
Hidden face removal We have seen how to geometrically transform the primitives
to the screen
The perspective projection gives a strong hint on depth
Hidden face removal is another strong hint
Allows to draw exclusively what is seen (goal : performance)
15
Computer Graphics
Graphic pipeline
Hidden face removal Backface culling
For closed and opaque shapes, one does not see the inside !
16
Computer Graphics
Graphic pipeline
Hidden face removal Backface culling
No need to draw the backward-facing faces
vn
n
v
17
Computer Graphics
Graphic pipeline
Hidden face removal Backface culling
No need to draw the backward-facing faces
vn
n
v
v⋅n≤0
18
Computer Graphics
Graphic pipeline
Hidden face removal Backface culling
Dépends ont the convention used to compute the normal to the shape
Usually, exterior pointing vector
Computation is easy if triangles are carefully oriented !
s1
n=s1 s2×s1 s3
∥s1 s2×s1 s3∥
s2
s3
19
Computer Graphics
Graphic pipeline
Hidden face removal How to care for this case ?
Painter’s algorithm
Binary space partitionning
Z-buffer
20
Computer Graphics
Graphic pipeline
Pipeline
Command flow
Transformed geometry
Fragments ( ~ pixels + interpolated data)
framebuffer
Application
vertex processing
Raster conversion
Operations on fragments (fragment processing)
Display
We are here
3D Transformations shading
Compositing, mixing, (shading)
What the user sees
« Langage » standards OpenGL, DirectX etc...
Conversion of primitives into
fragments
21
Computer Graphics
Graphic pipeline
Raster conversion First stage : enumerate pixels that are covered by a
« continuous » primitive
Second stage : Interpolate values that are known on the vertices of the primitive
Example : the colour known at the vertices of a triangle may be distributed on each pixel covered by the triangle
Other variables may also be interpolated. Normal vectors for instance...
22
Computer Graphics
Graphic pipeline
Raster conversion Transformation of continuous primitives into discrete pixels
Example : drawing of a line
Difficulty : aliasing
23
Computer Graphics
Graphic pipeline
Naïve algorithm Line = unit width rectangle
One specifies beginning and end vertices
Case here : black inside and white outside
24
Computer Graphics
Graphic pipeline
Naïve algorithm Line = unit width rectangle
One specifies beginning and end vertices
Case here : black inside and white outside
25
Computer Graphics
Graphic pipeline
Point sampling One approximates the rectangle by drawing every pixel
whose center is inside the rectangle
26
Computer Graphics
Graphic pipeline
Point sampling One approximates the rectangle by drawing every pixel
whose center is inside the rectangle
Problem : sometimes pixels with more that one adjacency are turned on
27
Computer Graphics
Graphic pipeline
28
Computer Graphics
Graphic pipeline
Bresenham algorithm (midpoint alg.) We will define the thickness with respect to the y axis...
29
Computer Graphics
Graphic pipeline
Bresenham algorithm (midpoint alg.) We will define the thickness with respect to the y axis...
One turns on only one pixel per column
45° slanted lineswill appearthinner.
30
Computer Graphics
Graphic pipeline
Bresenham algorithm (midpoint alg.) We will define the thickness with respect to the y axis...
One turns on only one pixel per column
45° slanted lineswill appearthinner.
31
Computer Graphics
Graphic pipeline
32
Computer Graphics
Graphic pipeline
Bresenham algorithm (midpoint alg.) Equation :
Evaluated for everu column
One turns on only one pixel per column
y=mxb
x0x1
0≤m≤1for x = ceil(x0) to floor(x1) y=b+m*x plot(x,round(y))
0
1
2
4
5
6
7
3
y=0.49x−0.01
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
33
Computer Graphics
Graphic pipeline
Optimization Multiply and rounding are rather slow operations (at least on
primitive CPUs)
For each pixel, the only options are E and NE One computes the error :
d > 0.5 decidesE or NE
d=m x1b− y
0
1
2
4
5
6
7
3
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
x , y
NE
E
y=mxb mxb− y=0
34
Computer Graphics
Graphic pipeline
Optimization One must only update integer steps along x and y
Exclusive use of the addition (no mult or divide)
d > 0.5 decidesE or NE
d=m x1b− y
0
1
2
4
5
6
7
3
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
d=dmd =d −1
35
Computer Graphics
Graphic pipeline
Optimization
0
1
2
4
5
6
7
3
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
x=ceil(x0) // round to ceily=round(m*x+b) // round to nearestd=m*(x+1)+b-yWhile (x<floor(x1)) // round to floor{ If d>0.5 { y=y+1 d=d-1 } x=x+1 d=d+m plot(x,y)}
36
Computer Graphics
Graphic pipeline
Generally, endpoints are given
Implicit form :
y=dydx
⋅x+b x0
y0
x1
y1
dx=x1−x0
dy= y1− y0
G x , y =dy⋅x−dx⋅ydx⋅b
G x , y0
G x , y 0
G x , y =0
≡F x , y=2 dy⋅x−2 dx⋅y2 dx⋅b
37
Computer Graphics
Graphic pipeline
What is the value of F at M ?
: M is under the linego to NE
: M is abovego to E
d i=F x i1, yi1/2=2 dy⋅x i1−2 dx⋅ y i1 /22 dx⋅b
M
Q
NE?
E?
d i0
d i≤0
x i , y i
38
Computer Graphics
Graphic pipeline
First point: what is the value of ?
As belongs to the line :
d 0
d 0=F x01, y01 /2=2 dy⋅ x01−2 dx⋅ y01/22 dx⋅b
=2 dy x0−2 dx⋅y02 dx⋅b2 dy−dx
=F x0, y02 dy−dx
x0, y0 F x0, y0=0
d 0=2dy−dx
39
Computer Graphics
Graphic pipeline
Recursion : what is the value of ?
If then go to E :
Otherwise go to NE :
d i1
d i≤0
x i1 , y i1=x i1 , y i
x i1 , y i1=x i1 , y i1
d i1=F x2, y1/2=d i2 dy
d i1=F x2, y3 /2=d i2dy−2 dx
40
Computer Graphics
Graphic pipeline
Algorithm is valid only for one octant
Bresenham( x1, y1, x2, y2 ){
dx = x2 – x1dy = y2 – y1d = 2*dy – dxplot( x1, y1 )while (x1 < x2 ){
if (d <= 0) // EASTd = d + 2*dy
else // NORTH-EAST{
d = d + 2*(dy-dx)y1 = y1 + 1
}x1 = x1 + 1plot( x1, y1 )
}}
41
Computer Graphics
Graphic pipeline
What to do if the line is not in the right octant ? Exchange x and y
Exchange start and end points
Replace y by -y
42
Computer Graphics
Graphic pipeline
Same type of algorithm do exist for other geometric shapes : e.g. circles (see litterature
What about aliasing ? Algorithm derived from Bresenham that avoid aliasing do exist, see algorithm
of Xiaolin Wu
Oversampling → sampling on a finer grid then averaging on bigger pixels is also an option.
Bresenham, Jack E. "Algorithm for computer control of a digital plotter", IBM Systems Journal, Vol. 4, No.1, pp. 25–30, 1965
Wu, Xiaolin. "An efficient antialiasing technique". Computer Graphics 25 (4): 143–152, 1991.
43
Computer Graphics
Graphic pipeline
44
Computer Graphics
Graphic pipeline
Interpolation Some variable is known at the vertices of the triangle (color,
normal vector, etc...)
One wish to get a representative value of the same variable along the line for every pixel that is turned on.
A progressive variation would be fine, thus a linear interpolation is the right tool here :
1D :
2D,3D : a is simply the distance ratio to the endpoints.
f x=1− f 0 f 1
=x−x0/ x1−x0
45
Computer Graphics
Graphic pipeline
Interpolation The pixels are not exactly on the line
One defines a projection on the line It is linear
One may useresults obtainedbefore to construct an interpolation
P0
P1
v
=v x / L=v y / L
=v⋅Q−P0/ L
L=v⋅P1−P0
46
Computer Graphics
Graphic pipeline
Interpolation The pixels are not exactly on the line
One defines a projection on the line It is linear
One may useresults obtainedbefore to construct an interpolation
P0
P1
v
=v x / L=v y / L
=v⋅Q−P0/ L
L=v⋅P1−P0
47
Computer Graphics
Graphic pipeline
Alternative meaning d and a are updated from pixel to the next pixel
d tells us at which distance we are to the line
a tells the position along the line
d and a are coordinates in the natural frame of the line
48
Computer Graphics
Graphic pipeline
Alternative meaning The loop means the pixels that we visit
Interpolation of d and a at each pixel
A fragment is emmitted if the pixel’s center is in the band
Interpolationbecomes themain operator
P0
P1
v
u
49
Computer Graphics
Graphic pipeline
Alternative meaning
P0
P1
v
u
x=ceil(x0)y=round(m*x+b)d=m*(x+1)+b-y etc...while (x<floor(x1)){
if (d>0.5){
y=y+1d=d-1 etc ...
} else
{x=x+1d=d+m etc...
}If (-0.5 < d <= 0.5)
plot(x,y, … )}
50
Computer Graphics
Graphic pipeline
Triangle raster conversion Very common case
With a good antialiasing, may be the only case ! Some systems represent lines with two very thin triangles
Triangle represented with 3 vertices
The algorithm has the same philosophy as for lines : One walks from pixel to pixel
Linear operators are evaluated for each step
Those operators allow us to know if a pixel is inside or outside
51
Computer Graphics
Graphic pipeline
Triangle raster conversion
Input :
Three 2D points
Variables to interpolate, at each vertex
Output : a list of fragments, with :
Integer coordinates of pixels
Interpolated variables
x0 , y0 ; x1 , y1; x2 , y2
q00 ,⋯, q0n ; q10 ,⋯, q1 n ; q20 ,⋯ , q2 n
x , y
q0 ,⋯ , qn
52
Computer Graphics
Graphic pipeline
Triangle raster conversion Incremental evaluation of linear functions on a grid of pixels
Functions are defined by the value at the vertices
Use of additionalfunctions to determinethe set of fragments to return back
x0 , y0
q00 ,⋯ , q0n
x1 , y1
q10 ,⋯ , q1n
x2 , y2
q20 ,⋯ , q2n
(x , y)
q0 ,⋯ , qn
Vertex
Fragment
53
Computer Graphics
Graphic pipeline
Incremental linear interpolation A linear affine function in the plane :
Evaluation on a grid is efficient :
q x , y =c x xc y yck
q x1, y=c x x1c y yck=q x , yc x
q x , y1=c x xc y y1ck=q x , y c y
+cx+c
x+c
x
+cy
+cy
.....
.....
54
Computer Graphics
Graphic pipeline
Interpolation of variables known at vertices Determine defining the unique linear function that
gives back the value at the vertices
3 parameters , 3 equations :
Singular system if points are collinear
cx , c y , ck
c x x0c y y0ck=q0
c x x1c y y1ck=q1
cx x2c y y2ck=q2x0 y0 1x1 y1 1x2 y2 1
cx
c y
ck=
q0
q1
q2
55
Computer Graphics
Graphic pipeline
Interpolation of variables known at vertices Translation of the origin to
2X2 linear system
Solution using Cramer’s rule
x0 , y0
q x , y=c x x−x0c y y− y0q0
q x1, y1=c x x1−x0c y y1− y0q0=q1
q x2, y2=c x x2−x0c y y2− y0q0=q2
x1−x0 y1− y0
x2−x0 y2− y0cx
c y=q1−q0
q2−q0
c x= q1 y2− q2 y1/ x1 y2− x2 y1
c y= q2 x1− q1 x2/ x1 y2− x2 y1
56
Computer Graphics
Graphic pipeline
What are the fragments to consider ? Those for which the barycentric coordinates are positive …
Algebraically, on has
Inside iff
Pineda, Juan, " A parallel algorithm for polygon rasterization" Computer Graphics 22 (4): 17-20, 1988
a
c
b
, , p= a b c=1
0 ; 0 ; 0
57
Computer Graphics
Graphic pipeline
Barycentric coordinates are interpolated variables Each barycentric c. yields 1 on a specific vertex and 0 on all others.
They are an implicit representation of the sides of the triangle.
58
Computer Graphics
Graphic pipeline
Pixel-per-pixel raster conversion (Pineda’s algorithm - 1998)
One vists conservatively a superset of the pixels
Use of interpolation of linear functions
Use of barycentric coordinates todetermine whento emit a fragment
x0 , y0
q00 ,⋯ , q0n
x1 , y1
q10 ,⋯ , q1n
x2 , y2
q20 ,⋯, q2n
59
Computer Graphics
Graphic pipeline
Triangle raster conversion Beware of rounding and arbitrary decisions
One has to visit every pixel at least once... (otherwise there may be a hole !)
But not two times ! (Those pixels would take an arbitrary colour that depend in which order things are drawn)
Elegant solution : antialiasing...
60
Computer Graphics
Graphic pipeline
Pipeline
Command flow
Transformed geometry
Fragments ( ~ pixels + interpolated data)
framebuffer
Application
vertex processing
Raster conversion
Operations on fragments (fragment processing)
Display
We are here
3D Transformations shading
Compositing, mixing, (shading)
What the user sees
« Langage » standards OpenGL, DirectX etc...
Conversion of primitives into
fragments
61
Computer Graphics
Graphic pipeline
Z-buffer (depth buffer) In many application, sorting with respect to depth is too
costly The order changes with the viewpoint
There exists the BSP tree Viewpoint independent but :
Heavy datastructures (difficult to implement « in silico ») Cutting of primitives Slow building Non-incremental (all data have to be known in advance)
62
Computer Graphics
Graphic pipeline
Solution that is usually favoured : draw in any order and keep track of the closest drawn pixel
Use an additional storage, that stores, for each pixel, the smallest depth to date
When one ought to draw a new pixel, the actual depth is compared to the stored depth and if the latter is greater, then the pixel is drawn and the depth is updated.
63
Computer Graphics
Graphic pipeline
Z-buffer
This is an axemple of a « brute-force » approach, it works because memory is cheap and fast.
Fol
ey e
t al
.
64
Computer Graphics
Graphic pipeline
Z-buffer Evidently limited to bitmap images (not vectorial)
Somewhat more difficult to implement with transparency (alpha channel)
65
Computer Graphics
Graphic pipeline
Z-buffer and alpha channels One separates opaque objects and translucent ones
1) Opaque objects are drawn with an update of the z-buffer.
2) Then , use a BSP tree for partially transparent entities
3) Draw transparent entities using the BSP ordering, while takin the Z-buffer into account, but without updating it
Thus, transparent faces behind opaque ones are not drawn (they are not visible)
66
Computer Graphics
Graphic pipeline
Z-buffer : limited accuracy The supplementary channel is generally encoded as an
integer, as are the color channels
Cause : hardware implementation (simple and fast)
It is possible to have distinct objects seen at the same depth if the value that is stored in the Z-buffer is the same
The accuracy is spread between n (near plane) and f (far plane)
These were used to define the observable volume, see course 2.
0< z*< N −1
67
Computer Graphics
Graphic pipeline
z*=0
z*=N −1
Plane n Plane f
Z-buffer
68
Computer Graphics
Graphic pipeline
Z-buffer Let us choose an integer with b bits (8 or 16...)
What is then the accuracy of the z-buffer?
If the z that is stored is proportional to the actual distance (case of orthogonal projections):
We have N=2b layers for a distance equal to f-n Therefore, the accuracy (independent to the distance!) is
: to maximize the accuracy, f-n must be kept small.
If a perspective projection is used, the z stored in the z-buffer is proportional to the result obtained after the perspective division.
Therfore, the size of the layers depends on depth By how much ?
Δ z=( f −n)
2b
69
Computer Graphics
Graphic pipeline
z*=0 z*
=N −1
Plane n Plane f
Orthographic proj.
Perspective proj.
Z-buffer
70
Computer Graphics
Graphic pipeline
Z-buffer
In the second course, we have computed zc after the
perspective divide :
x , y , z ,1⋅2∣n∣
r−l0 0 0
02∣n∣
t−b0 0
rlr−l
tbt−b
∣ f ∣∣n∣
∣n∣−∣ f ∣−1
0 02∣ f ∣∣n∣∣n∣−∣ f ∣
0
M proj _ persp _ OpenGL
= X ,Y ,∣ f ∣∣n∣∣n∣−∣ f ∣
⋅z2∣ f ∣∣n∣∣n∣−∣ f ∣
,− z
z c=∣ f ∣∣n∣∣ f ∣−∣n∣
2∣ f ∣∣n∣
z ∣ f ∣−∣n∣
z c=f nf −n
2 f n
z f −n
71
Computer Graphics
Graphic pipeline
Z-buffer
The interval for zc is 2 ( -1 to 1) and one still have N layers.
What is the size of the layers ? Need to solve for z (invert)
The largest layer is for z=f :
zc=f nf −n
2 f n
z f −n zc≈
2 z f n
z2 f −n
=2N
z=z2
f −n
N f n zmax=
f f −n
N n
72
Computer Graphics
Graphic pipeline
Z-buffer Example: n = 1 m, f = 100 m, the z-buffer has 8 bits →
N=256, what is the actual size of the layers ?
With n=10 :
z=z2
f −n
N f nΔ zmax=
f ( f −n)
N n=
100⋅99256⋅1
=39 m (0.15m with 16 bits)
zmin=n f −n
N f=
1⋅99256⋅100
=0.0039 m
Δ zmax=f ( f −n)
N n=
100⋅90256⋅10
=3.5 m (0.013 m with 16 bits)
zmin=n f −n
N f=
10⋅90256⋅100
=0.035 m
73
Computer Graphics
Graphic pipeline
Z-buffer For a good accuracy, it is best to increase n and decrease f.
Never ever set n=0 → cancels the z-buffer
Generally, the number of bits b is fixed by the hardware (usually 16, 24 or 32 bits)
The more the better but takes as much memory as the picture (usually 24 bits)
74
Computer Graphics
Graphic pipeline
Interpolation in perspective projection
projection plane
eye
Projections of the endpoints
75
Computer Graphics
Graphic pipeline
Interpolation in perspective projection
projection plane
eye
Projection of the middle of the line ≠ middle point of the projected lineLinear interpolation in screen coordinates ≠ interpolqtion in eye space
p1
p2
p1 p2
2
76
Computer Graphics
Graphic pipeline
Interpolation in perspective projection
projection plane
eye
z1 z2
Projects in the middle ...
z1z2/2
Projection of the middle of the line ≠ middle point of the projected lineLinear interpolation in screen coordinates ≠ interpolqtion in eye space
77
Computer Graphics
Graphic pipeline
Interpolation in perspective projection
Projection plane
eye
⏟ equidistant on z
Projection of the middle of the line ≠ middle point of the projected lineLinear interpolation in screen coordinates ≠ interpolqtion in eye space
78
Computer Graphics
Graphic pipeline
Interpolation in perspective projection
Projection plane
eye
zc1 z c2
This projects in the middle
zc1z c2/2
Projection of the middle of the line ≠ middle point of the projected lineLinear interpolation in screen coordinates ≠ interpolqtion in eye space
79
Computer Graphics
Graphic pipeline
Interpolation in perspective projection
Projection plane
eye
The depth variable that has to be interpolated (at the pixel level is zc (screen depth)
obtained after perspective divide, and not z, the real coordinate
⏟ equidistant on z c (screen depth)
80
Computer Graphics
Graphic pipeline
The perspective correction (use of zc instead of z ) aims to
avoid problems for e.g. textures applied on slanted surfaces
F u=1−u F 0u F 1
F u=
1−uF 0
zc0
uF 1
zc1
1−u1zc0
u1zc1
wik
iped
ia
F0
F1
81
Computer Graphics
Graphic pipeline
Minimal pipeline « Vertex » stage (input : 3D positions / vertices and colors / triangle)
Position transformation (object space → eye space) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)
Raster conversion Pixel list Transmission of the color
« Fragment » stage (output : color) Display pixels with right colors on the frambuffer
82
Computer Graphics
Graphic pipeline
83
Computer Graphics
Graphic pipeline
Minimal pipeline with z-buffer « Vertex » stage (input : 3D positions / vertices and colors / triangle)
Position transformation (object space → eye space) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)
Raster conversion
Interpolation of zc (z in screen space)
Transmission of the color
« Fragment » stage (output : color, zc)
Display pixels with right colors on the frambuffer and update z-buffer only if zc <
current zc
84
Computer Graphics
Graphic pipeline
85
Computer Graphics
Graphic pipeline
Flat shading Uses the « real » normal to the triangle
Faceted appearance Most realistic view
of the real geometry (as defined)
Foley et al.
86
Computer Graphics
Graphic pipeline
Minimal pipeline with z-buffer and flat shading « Vertex » stage (input : 3D positions / vertices and colors / triangle +
normal) Position transformation (object space → eye space) Color computation (flat shading) with the normal (one color / tri) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)
Raster conversion
Interpolation of zc (z in screen space)
Transmission of the color
« Fragment » stage (output : color, zc)
Display pixels with right colors on the frambuffer and update z-buffer only if zc <
current zc
87
Computer Graphics
Graphic pipeline
Flat shading
88
Computer Graphics
Graphic pipeline
Observer and illumination : close vs. far Phong illumination require some geometric info
Light vector (depends on position)
Obsever vector (depends on position)
Normal to the surface (computed beforehand)
Observer & light vectors are changing Must be computed & normalized for every facet
observer
light
89
Computer Graphics
Graphic pipeline
Observer and illumination : close vs. far Case where observer and source are far away
Almost parallel light rays
Almost orthographic projection
Light & observer vectors do not change much
A frequent optimization is to consider they do not change even though it is generally false
90
Computer Graphics
Graphic pipeline
Directional light (e.g. sun) Light vector is constant
In many cases, it increases dramatically the throughput of the pipeline by simplifying computations
observer
light[ x y z 0 ]
91
Computer Graphics
Graphic pipeline
Observer at the infinite Orthographic projection ?
Constant projection angle
One may also do that for the perspective projection, only for shading computations
The observer vector is considered constant (for instance, it is normal to the image plane)
Yield strange results if a wide angle view is used
Blinn-Phong shading: observer, light, and bisector vector all constant (see course 4)
92
Computer Graphics
Graphic pipeline
Directional light & far observer
Only the normal changes. Means the shading of every facet is the same if the orientation is the same.
observer
light[ x y z 0 ]
[xo yo zo 0]
93
Computer Graphics
Graphic pipeline
Gouraud interpolation One wants a smooth shading eventhough the geometry is
faceted Remember mapping : sensitivity to the normals much greater than the
sensitivity to positions
Idea : colour is computed at the facet’s vertices
Then, an interpolation is performed to get the color everywhere, at same time than the raster conversion.
94
Computer Graphics
Graphic pipeline
Gouraud interpolation
Foley et al.
95
Computer Graphics
Graphic pipeline
Pipeline with z-buffer and Gouraud interpolation « Vertex » stage (input : 3D positions / vertices and color per triangle +
normal per vertex) Position transformation (object space → eye space) Color computation made per vertex Position transformation (eye space → screen space)
Raster conversion
Interpolation of zc (z in screen space), rgb color
« Fragment » stage (output : color, zc)
Display pixels with right colors on the frambuffer and update z-buffer only if zc <
current zc
96
Computer Graphics
Graphic pipeline
Pipeline with z-buffer and Gouraud interpolation
97
Computer Graphics
Graphic pipeline
Gouraud interpolation Normals at vertices
Mathematically undefined
If the tessellation (triangulation) is obtained from a smooth surface (sphere, B-Spline, etc..), one may take the exact normal on that surface at the vertex
If not, just do as if by averaging neigboring triangle normals
N s=
∑i
N i
∥∑i
N i∥N s
N 1
N 2
N 3N 4
N 5
98
Computer Graphics
Graphic pipeline
Gouraud interpolation May be applied to any shading model
Diffuse
Blinn-Phong
Etc ...
However, for specular shading, it does not work well There are strong variations of the shading with respect to the position, even
on a single triangle. A linear interpolation is just too basic to reproduce these variations.
99
Computer Graphics
Graphic pipeline
Blinn shading and with Gouraud interpolation
Foley et al.
100
Computer Graphics
Graphic pipeline
Phong interpolation Why not interpolate normals and compute shading at the
pixel (fragment) level ? As easy as interpolating colors
The shading will be computed are every pixel, from information that have been interpolated (colors, normals, etc...
In the Graphic pipeline, it means that we move the shading computation from the « Vertex » stage to the « fragment » stage.
101
Computer Graphics
Graphic pipeline
Pipeline
Command flow
Transformed geometry
Fragments ( ~ pixels + interpolated data)
framebuffer
Application
vertex processing
Raster conversion
Operations on fragments (fragment processing)
Display
We are here
3D Transformations shading
Compositing, mixing, (shading)
What the user sees
« Langage » standards OpenGL, DirectX etc...
Conversion of primitives into
fragments
102
Computer Graphics
Graphic pipeline
Pipeline with z-buffer and Phong interpolation « Vertex » stage (input : 3D positions / vertices , triangle, + normal/color per
vertex) Position transformation (object space → eye space) Position transformation (eye space → screen space)
Raster conversion
Interpolation of zc (z in screen space), rgb color, xyz normal
« Fragment » stage (output : color, zc)
Shading computation using interpolated data (including normal)
Display pixels with right colors on the frambuffer and update z-buffer only if zc <
current zc
103
Computer Graphics
Graphic pipeline
Phong interpolation
104
Computer Graphics
Graphic pipeline
OpenGL Some shading models are part of the standard (typically :
Phong, Lambert, Gouraud, Blinn-Phong …), Nothing has to be done .
For specific shading computations, there is an API that allows defining exactly what has to be done at each stage :
Either at the « vertex shader » stage, vertex per vertex
Or, at the « fragment shader » stage, pixel per pixel
Recent graphic cards (and the software driver embedded in the operating system) allow to litterally program in pseudo C-language either the « vertex shader » or the « fragment shader » (see e.g. Nvidia CUDA , OpenCL)
105
Computer Graphics
Graphic pipeline
Programmable !
Not very versatile
106
Computer Graphics
GPGPU
GPGPU Use of computing power of GPUs to do other things that just
shading ...
Recent GPUs are somewhat versatile (but less than generic CPUs)
Branching became possible
Random memory access
But they are designed for vector computing (i.e. same operations applied to different data, SIMD)
There are standards to drive these GPUs : Open Computing Language (OpenCL, open)
Common Unified Device Architecture (CUDA, Nvidia)
Direct Compute (Microsoft)
107
Computer Graphics
GPGPU
OpenCL – pseudo-c language Host program may be in C++, and runs on the CPU
The OpenCL code is stored as a character string (char[] or std::string )
The OpenCL code is then compiled by the graphic card’s « driver » (the CPU does the job), result is uploaded on the GPU by dedicated system calls
To access to the machine code, system calls allow to pass parameters and memory chunks to the GPU
The computation is made on the GPU
Results are given back to the host program using memory transfer.
108
Computer Graphics
GPGPU
OpenCL sample code
__kernel void VectorAdd(__global float* c, __global float* a , __global float* b, __constant float *cst{ // Index of the elements to add unsigned int n = get_global_id(0); unsigned int nl = get_local_id(0); unsigned int gsz = get_global_size(0); unsigned int lsz = get_local_size(0); unsigned int gid = get_group_id(0); // do some math from vectors a and b and store in c __private float res; res=0.; int i; for (i=0;i<100;++i) // not a loop over elements of the arrays { res=a[n]+sin(a[n])+b[n]; if (res>5.5) res=a[n]* (*cst) ; } c[n] = res;}
109
Computer Graphics
GPGPU
In the host program … init
std::string Src ; // Source code (see preceding slide)
std::vector<cl::Platform> platforms;cl::Platform::get(&platforms);
cl_context_properties properties[] = { CL_CONTEXT_PLATFORM,(cl_context_properties)(platforms[0])(),
0};
cl::Context context(CL_DEVICE_TYPE_ALL, properties);std::vector<cl::Device> devices =
context.getInfo<CL_CONTEXT_DEVICES>();cl::Program::Sources
source(1,std::make_pair(Src.c_str(),Src.size()));
cl::Program program = cl::Program(context, source);
program.build(devices); // compilation !
110
Computer Graphics
GPGPU
In the host program … system calls
cl::CommandQueue queue(context,devices[0],0,&err);cl::Buffer GPUOutVec(context, CL_MEM_WRITE_ONLY
,sizeof(float)*SIZE, NULL, &err);float cst=1;cl::Buffer GPUVec1(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR
,sizeof(float)*SIZE,HostVec1,&err);cl::Buffer GPUVec2(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR
,sizeof(float)*SIZE,HostVec2,&err);cl::Buffer GPUCst1(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR
,sizeof(float),&cst,&err);cl::Event event1;cl::Kernel kernel(program_, "VectorAdd", &err);kernel.setArg( 0, GPUOutVec);kernel.setArg( 1, GPUVec1);kernel.setArg( 2, GPUVec2);kernel.setArg( 3, GPUCst1);
queue.enqueueNDRangeKernel(kernel,cl::NullRange,cl::NDRange(SIZE_TEST),cl::NullRange,NULL,&event1);
event1.wait();// at this point, the computation has been made.
111
Computer Graphics
GPGPU
In the host program … collecting data back
Complete code sample on the website of the course
// at this point, the computation has been made.
cl::Event event2;queue.enqueueReadBuffer(GPUOutVec,CL_TRUE,0,
SIZE_TEST*sizeof(float),HostOutVec, NULL, &event2);event2.wait();
// HostOutVec contains the results
for (int Rows = 0; Rows < SIZE_TEST/32; Rows++){
for (int c = 0; c <32; c++) std::cout << HostOutVec[Rows * 32 + c] << " " ; std::cout << std::endl;}
112
Computer Graphics
GPGPU
OpenCL is a multiplateform paradigm The above code sample may be compiled on any computer (having a
GPGPU enabled driver)
It can even be executed on a ... CPU (when no GPU is available) There exist « dummy » drivers that will simply compile and execute the code on
the same CPU. Useful to debug and benchmark
Performance : factor 2 to 100 in favor of the GPU for vector operations (e.g. on big arrays)
Core i7 with 6 cores 3,33 Ghz vs. Nvidia Quadro FX 580 (not very powerful) : 110 s. vs 55 s.
On a specialized GPGPU graphic card : Nvidia Tesla C2075 : 11 s.
113
Computer Graphics
Graphic pipeline
Painter’s algorithm Idea : display every primitive in the right order
Everything « below » is overwritten, like when painting
114
Computer Graphics
Graphic pipeline
Painter’s algorithm Idea : display every primitive in the right order
Everything « below » is overwritten, like when painting
115
Computer Graphics
Graphic pipeline
Painter’s algorithm Idea : display every primitive in the right order
Everything « below » is overwritten, like when painting
Amounts to define a topological sorting Find a path in an oriented graph
AB
C
D
EF
A
B
C
D
E
F
ABCDEFABDCFECAEBDF...
116
Computer Graphics
Graphic pipeline
Painter’s algorithm Impossible if cycles are present ...
A
B
C
A
B
C
ABC ???
117
Computer Graphics
Graphic pipeline
Painter’s algorithm Impossible if cycles are present …
Solution : cut all !
a
f
o
b,l cd,g
m
k
ej
i,n
h a
o
l
cg
m
k
ei
h
db
n
118
Computer Graphics
Graphic pipeline
Painter’s algorithm Useful when an order is easy to define
Works with vector graphics
May be very CPU intensive (cut;sort ..)
Fol
ey e
t al
.
119
Computer Graphics
Graphic pipeline
Painter’s algorithm The ordering depends on the point of view
Sorting primitives is costly ( nlog(n) to the best ) and have to be done every time the observer moves
Primitives must be cut if forming cycles (how to detect ?)
A response to these drawbacks is the binary space partition tree (BSP Tree)
120
Computer Graphics
Graphic pipeline
BSP Tree Input data :
Segments in 2D
Triangles in 3D – same idea
a b
c
d
e
121
Computer Graphics
Graphic pipeline
How to build the BSP Tree (1)
a b.1
c
d
e
b.2
c+ -
+
-
→ Take one of the segments and define a line cutting the plane in two separate half-planes→ Classify the other segments with respect to the boundary. If a segment is crossing, partition it and classify its parts→ On each subdomain, if there are more than one segment, repeat the procedure recursively
122
Computer Graphics
Graphic pipeline
How to build the BSP Tree (2)
a b.1
c
d
e.1
b.2
c
e.2
d
+ -+
-
+- +-
e.1
→ Take one of the segments and define a line cutting the plane in two separate half-planes→ Classify the other segments with respect to the boundary. If a segment is crossing, partition it and classify its parts→ On each subdomain, if there are more than one segment, repeat the procedure recursively
123
Computer Graphics
Graphic pipeline
How to build the BSP Tree (3)
a b.1
c
d
e.1
b.2
e.2
c
d b.1
a
+ -+
-
+-
+-
e.1
→ Take one of the segments and define a line cutting the plane in two separate half-planes→ Classify the other segments with respect to the boundary. If a segment is crossing, partition it and classify its parts→ On each subdomain, if there are more than one segment, repeat the procedure recursively
124
Computer Graphics
Graphic pipeline
How to build the BSP Tree (4)
a b.1
c
d
e.1
b.2
e.2
c
d b.1
a
+ -+
-
+-
+-
e.1+
-
e.2
b.2
→ Take one of the segments and define a line cutting the plane in two separate half-planes→ Classify the other segments with respect to the boundary. If a segment is crossing, partition it and classify its parts→ On each subdomain, if there are more than one segment, repeat the procedure recursively
125
Computer Graphics
Graphic pipeline
How to choose the right segment at each step ? A random choice is not that bad...
Complete algorithm :
Let S be a set of line segments (or triangles in 3D)Build(S,BSP){
If (Card(S) <=1) BSP is a tree with only one node that contains the only segment of S or nothingElse{
Use a random segment s belonging to S as a cutting line and cut all the other segments S+ = segments belonging to H+ (« positive » halfspace) (without s)S- = segments belonging to H- (« negative » halfspace) (without s)Call recursively Build(S+,BSP+)Call recursively Build(S-,BSP-)Build a tree with BSP as a root, and BSP+ and BSP- as children. The root contains s.
}}
126
Computer Graphics
Graphic pipeline
Use of the BSP Tree How to scan the tree to get the right display order ?
a
+ -
e.1
b.2
ObserverO
Let O c+ a point (the observer)→ it is clear that entities from c- must be displayed before entities from c, that must be displayed before those of c+.
c
d b.1
e.2
a b.1
c
d
e.1
b.2
e.2
+
-
+-
+-
+
-
127
Computer Graphics
Graphic pipeline
Use of the BSP Tree How to scan the tree to get the right display order ?
a
+ -
e.1
b.2
c- c c+The same remark arises for c- (then c+)O b.1- thus entities of b.1+ must be displayed before those on b.1, and before those in b.1-
1
23
4
5
c
d b.1
e.2
a b.1
c
d
e.1
b.2
e.2
+
-
+-
+-
+
-
ObserverO
128
Computer Graphics
Graphic pipeline
Use of the BSP Tree How to scan the tree to get the right display order ?
a
+ -
e.1
b.2
a b.1 (b.1-) c c+For c+ :O d+ thus the order is d- d d+
1
23
4
5
6 c
d b.1
e.2
a b.1
c
d
e.1
b.2
e.2
+
-
+-
+-
+
-
ObserverO
129
Computer Graphics
Graphic pipeline
Use of the BSP Tree How to scan the tree to get the right display order ?
a
+ -
e.1
b.2
a b.1 (b.1-) c d- d d+
1
23
4
5
6
7
8
9
c
d b.1
e.2
a b.1
c
d
e.1
b.2
e.2
+
-
+-
+-
+
-
ObserverO
130
Computer Graphics
Graphic pipeline
Use of the BSP Tree How to scan the tree to get the right display order ?
a
+ -
e.1
b.2
a b.1 (b.1-) c e.1 d d+For d+ :O e.2+ thus order is e.2- e.2 e.2+
1
23
4
5
6
7
8
9
10101112
c
d b.1
e.2
a b.1
c
d
e.1
b.2
e.2
+
-
+-
+-
+
-
ObserverO
131
Computer Graphics
Graphic pipeline
Use of the BSP Tree How to scan the tree to get the right display order ?
a
+ -
e.1
b.2
a b.1 (b.1-) c e.1 d b.2 e.2 (e.2+)
Final order : a b.1 c e.1 d b.2 e.2
1
23
4
5
6
7
8
9
10101112
c
d b.1
e.2
a b.1
c
d
e.1
b.2
e.2
+
-
+-
+-
+
-
ObserverO
132
Computer Graphics
Graphic pipeline
Recusive scanning algorithmDraw(BSP,ViewPoint){
If BSP is a leaf (no children)Draw the primitives contained in BSP
Else{
Let BSP+ and BSP- - children of BSPIf ViewPoint is in H- (« negative » halfspace){
Call Draw(BSP+,ViewPoint)Draw the primitives contained in BSPCall Draw(BSP-,ViewPoint)
}Else If ViewPoint is in H+ (« positive » halfspace){
Call Draw(BSP-,ViewPoint)Draw the primitives contained in BSPCall Draw(BSP+,ViewPoint)
}Else (we are exactly on the plane...){
(Draw the primitives contained in BSP) / but not necessaryCall Draw(BSP+,ViewPoint)Call Draw(BSP-,ViewPoint)
}}
}
BSP
BSP-BSP+
133
Computer Graphics
Graphic pipeline
The BSP tree is generally not directly used in graphic cards Building a BSP tree is relatively slow (nlogn to the best), and the
datastructure is not well adapted to vector/parallel treatment.
→ Z-buffer : see further
It can however be used in certain cases in software to ease the work of graphic cards
e.g. « FPS » video games where the environnment is mostly fixed « Doom » is an old video game which used this principle.
Il can be used for ray-tracing...