Image Synthesis

computer graphics & visualization

Image Synthesis

Interactive Terrain Rendering


Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group

Motivation



Mobile Terrain Visualization• Mobile GPUs

– 250M pixels / second– 5M triangles / second– 6 Textures– OpenGL ES / Direct3Dm

• Challenges– 1,25 MB Video-RAM…– Fixpoint formats– Bandwidths– Compression, streaming etc.



Requirements• Games

– High, constant frame rates– Quality advantageous but not priorized

• Simulators– Constant frame rates (~30fps)– High degree of realism and detail, optionally stereo

• GIS– Interactive frame rates advantageous (15+ fps)– High resolution and precision



Challenges

Crater Lake, Oregon, US

Heightfield with bathimetry and texture

~ 500 MB

Oberbayern and Alps

Heightfield with texture

~ 350 MB

Puget Sound, WA, USA


~ 1.25 GB



ChallengesMars, MOLA mission


~ 4.5 GB

USA complete, USGS data

Heightfield

~ 40 GB

Mars, MarsExpress mission

Heightfield with texture(Video)

~ 175 GB



ChallengesData sets are:• Large to gigantic

– Data volume is increasing rapidly• Sometimes multi-modal

– Color, Albedo, Cloud coverage, Scattering properties, …• Sometimes annoted and commented

– Additional layers with meta data

Efficient Rendering ?



Terrain rendering• Terrain:

– A height field over a rectangular domain

– Given at discretesample points



Terrain renderingRendering lit and textured triangles

– Vertices, texture coords, colors, normals



Terrain rendering• Ray-Tracing

– Ray-tracing triangle meshes• Trace rays until a triangle is hit

• Implicit occlusion culling



Terrain rendering• Ray-Tracing - performance issues

– Mesh data structure has to be stored– Hierarchical representation necessary for improved

intersection testOctree or kD-tree

max/min(H)

max/min(H)

max/min(H)

• Store max/min heights for subregions• Skip regions below minimal height of ray• Employ ray coherences



Terrain rendering• Rendering polygonal models – analysis

– Aliasing due to undersampling of small features– No exploitation of the LOD-nature



Terrain rendering• Rendering techniques for terrains

– Render each triangle separately

– Draw: (v0,v1,vX), (v1,vX,vX+1), ... – Each vertex is rendered (transmitted, transformed) six times

V0 V2V1 V3

...

... VX-1

VX VX+2VX+1 VX+3



Terrain rendering• Rendering techniques for terrains

– Render triangle strips: v0,vX,v1,vX+1,v2,vX+1, ...

– Triangle defined by new point and previous two points– Each vertex is rendered only once

V0 V2V1 V3

...

... VX-1

VX VX+2VX+1 VX+3



Terrain rendering• Height field pre-processing

– Reduce number of primitives by mesh decimation– Mesh decimation based on any decimation ctriterion

• Curvature („flatness“)• Use fewer but larger triangles where the height field is flat



Terrain rendering• Even with mesh decimation there are still too

many triangles– Some features might not be visible because they

are too far from the viewer• Feature size is smaller than size of a pixel

– Level-of-detail should be adapted to the size of details when displayed rather than to their size in world space



Terrain rendering• Approach: adaptively decimate the mesh with

regard to the current view• General idea:

– Top-down approach– Start with coarse resolution (2 triangles) and

adaptively refine until desired level-of-detail is reached

– Consider world space and screen space deviation– Re-build mesh for every frame



Terrain rendering• Recursive split operation (as in ROAM)



Terrain rendering• Error criterion for split operation

– Deviation in world space• Difference between height value at center vertex and

average of left and right vertex– Deviation in screen space

• Determine upper bound for screen space length of world space difference vector (0,d,0)



Previous Work

– Process and render Batches of geometry: (P-)BDAM• Huge pre-processing requirements• Hardware requirements (SCSI Raid) • Average Quality

– Hierarchical, regular Grids: (Geometry Clipmaps)• Good compression, good framerates• Error control very hard, quality suffers• Losasso & Hoppe SIGGRAPH 2004

- Vast body of literature… see www.vterrain.org- Key Insight: Dynamic remeshing too expensive

http://www.vterrain.org/



Geometry clipmaps

Coarsest Level

Finest Level

- Terrain as mipmap pyramid- LOD using nested grids- LOD is shifted over domain, with varying height values



Geometry clipmaps• Advantages:

– Regular structure of nested grids enables fast rendering using triangle strips

– Only regular and continuous chunks of memory have to be read

• Disadvantages:– Does not allow to control local world/pixel error

• Renders more triangles than necessary• Introduces a significant higher pixel error in general



GPU-friendly terrain rendering• Tile-based, restricted Quadtree

• Lmax error metric allows for precise error control• Low pre-processing times (15M vertices/min)• Nested hierarchy

• Pre-filtered, tiled S3TC-Mipmaps• High-quality Lanczos pre-filtering• Reduces artifacts at tile borders• S3TC virtually for free

• Precise Level of Detail Oracle• Based on the Jacobian of the projection• Does not make linear approximations• Better anti-aliasing control



Tile-based…• Partition data into square tiles

– Can be pre-processed independently– Frustum culling per tile– Progressive data transfer per tile

• T-Vertices: Fix with zero-area triangles– Only dynamic triangulation during run-time.



Restricted Quadtree Hierarchy• Tiled meshes are decimated in a pre-process • Mesh decimation as in ROAM (Real-Time Optimally

Adapting Meshes - see www.vterrain.org)– Exploits a triangle bintree structure: split along the base

edge

http://www.vterrain.org/



Restricted Quadtree Hierarchy• The recursive split operation

– A triangle should be split but has a base neighbor from a coarser level

• Force split of base neighbor first• Recursively force further splits if necessary until diamond

is found• Diamond can be split without further splits



Restricted Quadtree Hierarchy• Recursive split operation



Restricted Quadtree Hierarchy• Vertices can be shared across levels

– „Coarse“ Vertices used in all finer levels– Progressive encoding, less memory consumption



Restricted Quadtree Hierarchy• By generating approximations of the height field with

decreasingly lower approximation error, a mesh hierarchy that represents the original terrain at ever finer scales is constructed

• To generate a discrete set of hierarchy levels, an error vector (e0; e1; … ;e(n-1)) of exponentially decreasing entries ei := 2(n-1-i) is specified

• Hierarchy levels that satisfy these errors are constructed via adaptive triangulation



Restricted Quadtree Hierarchy• Each hierarchy level can be render as „recursive triangle fans“

– Fans around „quadtree centers“– Exploit primitive restarts on nVidia architectures

• A flag in the primitive stream that indicates a new fan• Used to avoid frequent calls from application program



Restricted Quadtree Hierarchy• Nested Mesh Hierarchy:

• Quadtree refinement:– Automatically results in nested mesh– Each new triangle re-uses two old vertices

• Elegant geomorphing possible– Requires only one height value per vertex & level– Can be implemented in vertex shader

„The parameter domain of each triangle is a subset of the parameter domain of exactly one triangle at the coarser levels.“



Tiled, S3TC Mipmaps• Tiled Mipmaps:

– Start with large 2D texture– Generate large Clipmap using Lanczos filter– Tile after filtering artifacts at boundaries reduced

• S3 Texture Compression– Designed for Games Supported by virtually all GPUs– Fast encoding in driver– Decoding virtually for free– Block Truncation Code, 4x4 pixels per block– Lossy compression: 6:1 Quality ?– We are researching other GPU-friendly methods…



S3TC Quality ?Pros: „rough“ textures



S3TC QualitätAnd Cons… - „smooth“ textures



Texture compression• Vector quantization

Encoder

Decoder

Xn

in=E(Xn)

in

X‘n=C(in)

Input mapping

Output mapping

Codebook Cwith codewords



Vector Quantization• Find centroids (representative vectors) to which all

elements are assigned• Elements get assigned to „closest“ representative



Texture compressionVector quantization: the LBG-Algorithm

1. Inital codebook {Yi(0)}i=1..M; IRn: set of input vectors;

2. Initialization k=0, D(0) =0, set

3. Find quantization regions Vi (k) = {XI: d(X,Yi) < d(X,Yj) j≠i, j=1..M}

4. Compute distortion

5. Test: If (D(k)-D(k-1))/D(k) < ) stop else continue

6. Update: increment k, find new codebook {Yi(k)}i=1..M from regions

Vi (k-1); goto 3

M

i VX

ki

k

ki

YXdD1

)()(

)(

),(



Texture compressionTrue-Color (3*8 Bit) vs. 12 Bit Quantization + Codebook



Texture compression

8 Bit 6 Bit

4 Bit 2 Bit



Texture compressionRegion quantization

2 bpp 2 bpp, 2x2 pixel encoding



Confocal scan 10242x64 RGBA

32 bpp 1 bpp



CT scan 2563

Original Hierarchical encoding 21:1



Engine CT scan 2562x128

Original Hierarchical encoding 21:1



Visible Male RGB slice 2048x1216

Original 24 bpp Hierarchical encoding 0.3 bpp



Vortex simulation 100 x 1283

Original, 200 MB Hierarchical encoding 11 MB



Shockwave simulation 96 x 2563

Original, 1.4 GB Hierarchical encoding 70 MB



Texture compression• Vector quantization for GPUs

– Easy to decode in fragment shader• Only requires a dependent texture fetch (uses index to

lookup the entry in the codebook)– Does not allow for texture filtering (so far)

• Interpolation between indices gives errorenous results• Eventually larger regions can be decoded, rendered into

texture render target and then used to texture the object• Functionality is available but mipmapping might be a

problem



Scale of detail• Question:

– Now that we have a geometry and a texture MipMap, how is the resolution determined at which a particular tile has to be rendered

• Guess: – Depends on distance to viewer– Depends on projection used for rendering– Depends on orientation of tile



Scale of detail• Projection is non-linear (hom. division)• Pixel coords s(v), where v vertex in object space• Object space scale to pixel correspondence

– Similar to Mipmapping– Requires partial derivatives– Can be obtained by the inverse Jacobian of s(v):

• Jacobian s(v) = ∂si/∂vi

• Jacobian-1 s(v) = ∂vi/∂si



From scale to level of detail• Pre-process: levels L with errors 2L meters

LOD =log2() for bounding box corners of tiles

• Tri-linear interpolation of in box– LOD per vertex for geomorphs

• Find minimum of for each box– Finest detail determines used topology– Project vertices to appropriate level (geomorph)– Requires pre-computed height for each vertex & level

• But only for coarser levels• And can be done in reduced precision (i.e. 8bits)



Level of detail: benefits• Near-optimal geometry filtering

– Geometry properly anti-aliased– Geometry properly reduced– No over-estimation of error

• No linear approximations of the projection• … but so far no anisotropic geometry filter• Still works very well in practice

• LOD for Textures is easier:– Mipmaps– Anisotropic texture filtering– All supported by GPU without major performance penalty



Texture filtering - MipMap



Texture filtering - anisotropic



Demos

Puget Sound

Oberbayern und Alpen

Image Synthesis

Documents

Transcript of Image Synthesis