Company of Heroes 2 Rendering Tech: The cold facts of recreating the

hardest winter conditions of World War 2

Daniel Barrero PhDSenior Graphics ProgrammerRelic Entertainment / SEGA

Korea Game Conference – KGC 2013 – September 25 2013

Introduction

Presentation Index• Implementing winter tech, what was tried, what worked,

what didn't:– Rendering realistic and stylized snow and ice

• Procedurally generated snow • Automatic snow edge fading• Ice cracking

– Generating terrain snow tracks and procedural snow

– Solution for fast volumetric outdoor ambient occlusion

– Blizzard FX optimization

• Seamlessly upgrading the COH1 rendering engine from forward to deferred

COH2 Winter Tech Requirements• Procedural Snow on world objects• Fully dynamic terrain snow and persistent snow foot prints and

tracks• Blizzards• Fully dynamic Ice, can break and refreeze based on player

interaction• Improved lighting:

• Ambient occlusion is key for snowy enviroments.

• Possibility to add many dynamic lights to the world

Rendering Snow on World Objects• Mostly visual representation of snow accumulation on objects,

a.k.a snow caps:

Rendering Snow on World Objects• Possible solutions:– Snow prepainted on the texture by artists:

✔ Just an art change no change to the shaders or engine

✗ Every model has to have a new set of textures with snow painted on ( lots of work for the artists)

• Can't change the amount of snow dynamically in a realistic way

Procedural Snow on World ObjectsObservation: Snow accumulates on surfaces that tend towards

horizontal

It means we can just filter faces whose normals are closer than a given to the up world vector and apply snow to them!

✔ Simple shader change to check up for vertical normals

✔ Can use normal map information to get accurate snow

✔ Can control the snow amount by checking how far the pixel normal is from the vertical direction

✔ Blending object material with snow material based on normal up component is fast and looks good.

Procedural Snow on World ObjectsObservation:

• Snow accumulates on surfaces that tend towards horizontal

• It means we can just filter faces whose normals are closer than a given to the up world vector and apply snow to them!

Snow covered faces

Faces without snow

False positive for snow covered faces

Procedural Snow on World ObjectsObservation: Snow accumulates on surfaces that tend towards

horizontal

It means we can just filter faces whose normals are closer than a given to the up world vector and apply snow to them!

More Observations:

• Only the topmost surfaces should get snow

• Snow accumulating on top of objects changes the visible shape of the mesh.

• Snow is thinner and tends to melt towards the edges of surfaces first.

Getting rid of snow inside objectsProblem: Only the topmost surfaces should get snow

Solution: Generate a “snow shadow” map (a.k.a. Top down map)

1. Set a camera above the world it looking down

2. Render a depth map of closest surfaces with it

3. In the normal render check that world height against

the current pixel world height if is less then don't

Put snow on it

Top Down Map

Snow Surface Selection

Top Down CameraView

Top Down MapTop Down Map

Generating snow caps geometryProblem: Snow accumulating on top of objects changes the visible shape of the mesh.

Possible solution: Extrude the geometry of the horizontal faces upward.

ExtrudeDirection

Extruded Geometry

Original top faces

Generating snow caps: edge fadingProblem: Snow is thinner and tends to melt towards the edges of surfaces first

Possible solution: If we are extruding the mesh to generate geometry, then to make snow caps thinner just push the corners of the surface back in the opposite direction of the edge surface normal.

ExtrudeDirection

Push direction

Original top faces

Original non biased extruded geometry

Extruded Geometry

Average Edge Normal

Generating snow capsProblems with geometry extrusion:

✗ Need to know the edges of surfaces on a mesh and the normal on the edge. E.g. edges between wall and ceilings

✔Can be solved by offline tool easily by processing the mesh and looking for big changes in normal direction.

Surface 1

Surface 2

Surface 3

Surface Edge

Edge Normal

Triangle Normal

Generating snow capsProblems with geometry extrusion:

✗ Need to know the edges of surfaces on a mesh and the normal on the edge and pass it to the shaders. E.g. edges between wall and ceilings

✔Can be solved by offline tool easily by processing the mesh and looking for big changes in normal direction.

✗ Need to generate geometry in the GPU

✗ Performance issues on Dx10 hardware

✗How to pass the information to the shaders

•There are lots of special cases

Snow caps optimization: virtual extrusion

Observation: From the standard game camera view just “painting” over the snow and modifying the perpixel normals it looks almost the same as with snow caps since we can't see the extruded geometry close by.

Problem: Still need to fade snow towards the edge of surfaces.

• Need to pass edge information to the shaders

Solution: Do a “virtual extrusion”in the shaders:

Vertex Shader:1. Compute the vertex extruded point2. Project point into the triangle surface3. Compute distance to original vertex 4. Pass value into pixel shader

Pixel Shader:4. Use interpolated “extruded” Value as snow depth and fade Snow out.

Snow caps optimization: virtual extrusion

Observation: From the standard game camera view just “painting” over the snow and modifying the per pixel normal it looks almost the same as with snow caps since we can't see the extruded geometry close by.

Problem: Still need to fade snow towards the edge of surfaces.

• Need to pass edge information to the shaders

Solution: Do a “virtual extrusion” in the shaders

Improving Snow edge fadingProblem: Virtual extrusion produces some odd artifacts on many corner cases due to different mesh topologies.

Snow edge fading on damaged buildings

Problem: In COH it is possible to damage and destroy parts of a builings in many ways.

• Buildings are composed of panels

• Panels get damaged by punching “holes” in the texture and revealing underlying substance

• Shape of the hole is based on damaged neighbors

• Geometry is added on the edges of the damaged hole

Improving Snow edge fadingProblem: Virtual extrusion produces some odd artifacts on many corner cases due to different mesh topologies and use cases

Solution: Take advantage of the “snow shadow” map using image processing to improve snow edge fading, by finding the min/max

Top Down Map(world height)

Apply min heightFilter (erode)

Apply top height Filter (dilate)

Actual used top down map

RedChannel

GreenChannel

Final snow

Use the two channels toDetermine edge fading.Add noise to break the edge

Improving Snow edge fadingProblem: Virtual extrusion produces some odd artifacts on many corner cases due to different mesh topologies and use cases

Solution: Take advantage of the “snow shadow” map using image processing to improve snow edge fading, by finding the min/max

Sample of noise to break the edge

Improving Snow edge fadingProblem: Virtual extrusion produces some odd artifacts on many corner cases due to different mesh topologies and use cases

Solution: Take advantage of the “snow shadow” map using image processing to improve snow edge fading, by finding the min/max

Pros:✔Reuses “snow shadow” which is required anyway

✔ Solves the damaged buildings problem

✔ Mesh geometry doesn't matter

Cons:✗ Needs a high resolution texture to look good

✗ Snow map costly to generate every frame

After all that: final look

Terrain Snow• Artist driven

– Extra terrain layer

– Affects terrain height

Terrain Snow• Artist driven

– Extra terrain layer

– Affects terrain height

– Snow edges procedurally distorted to reproduce snow behavior

Terrain Snow• Artist driven

– Extra terrain layer

– Affects terrain height

– Snow edges procedurally distorted to reproduce snow behavior

– Use GPU tessellation in D3D11

Terrain Snow affecting Objects

Problem: consistency between terrain snow and object snow

Solution: Allow objects to “suck in” the terrain snow material.

✔ It allows the snow to look consistent between terrain and objects✔ Side effect that artists can tune edge fading more on objects

placed on a map by painting the snow under it✔ Side effect that damaging terrain removes snow also removes

parts of snow from objects improving the melting snow caused by damage effect.

Terrain Snow affecting Objects

Problem: consistency between terrain snow and object snow

Solution: Allow objects to “suck in” the terrain snow material.

Undamaged Terrain Snow Damaged Terrain Snow

Terrain Snow Tracks

Extra pass:– Render objects that moved into a texture with a top down camera view

and clipped at max snow height using an additive shader

Track MapTop Down CameraView

Clip Plane

Terrain Snow Tracks

●Extra pass:– Render objects that moved into a texture with a top down

camera view and clipped at max snow height using an additive shader

– Just visual, uses bump mapping in D3D10 HW and GPU tessellation on D3D11

Terrain Snow Tracks

Extra pass:– Render objects that moved into a texture with a top down camera view

and clipped at max snow height using an additive shader

– Pros:

✔ Easy way to get precise snow tracks and foot prints– Cons:

✗ Requires a high texel density per terrain area to get accurate foot prints.

✗ Since it persists over multiple frames it is not very SLI/Crossfire friendly.

✗ Requires high frame rate to get smooth continuous vehicle tracks.

Muck map

- Material based on snow track depth to make the tracks stand out more and reveal mud

Ice

Requirements:

- Fully dynamic

- Can break and refreeze

- Should look realistic

- Edge breaking should look good

- Easy to tune by artists

Ice

- World has an Ice cell map that controls ice health

- The sim controls breaking and refreezing looking at ice health of neighboring cells

Ice

- World has an Ice cell map that controls ice health

- The simulation controls breaking and refreezing looking at ice health of neighboring cells

- Ice Shader takes the Ice health map as a texture and uses the health information to determine how to blend between textures that represent each ice state:

- Solid

- Broken

- Slush

- Water

Ice

Problem: Grid is very visible on the edges of broken ice

Solution: Use an artist defined texture to offset the uv sampling to break the edges, use noise to break the edges even more:

Sample UV Offset TextureNormal UVUse UVTexture ValueTo sample Ice textures

Final look

Outdoor Volumetric Ambient Occlusion

- On snowy weather, ambient occlusion is what gives variety to the environment and makes snow variation stand out and be more visible.

- COH1 used fake splat based ambient occlusion.✗ Not good enough to depict light occlusion on snowy

environments.✗ Not accurate.✗ Looks like sets of blobs

Outdoor Volumetric Ambient Occlusion

Solution: Take advantage of the top down map used for the edge fading and “snow shadow” map to compute ambient occlusion:

- use the height information from the topdown map as a height map and do quick ray casting in the GPU to compute the ambient occlusion

TopDownMap Computed Ambient Occlusion

Outdoor Volumetric Ambient Occlusion

Solution: Take advantage of the top down map used for the edge fading and “snow shadow” map to compute ambient occlusion:

- use the height information from the TopDown map as a height map and do quick ray casting in the GPU to compute the ambient occlusion

Pros:✔ More accurateCons: ✗ Slow to compute

can be pre-computed for static objects

Outdoors Volumetric Ambient Occlusion

Without ambient occlusion

Outdoors Volumetric Ambient Occlusion

With ambient occlusion

Snow Blizzard FX Optimization

Problem:

- Uses lots of big particles for the snow blizzard meaning huge overdraw even using the special fast Weather FX path [KGC2012 Relic FX talk: http://www.slideshare.net/proyZ/relics-fx-system]

Solution:

- Make the weather FX deferred:– Render Weather FX in a 1/2 screen resolution texture

– Render that texture over the full screen as a post-process pass

Huge performance improvement, worst case went from a frame rate drop of 30% to below 4%

COH1 Forward Rendering Engine• Bleeding edge technology Circa

2006/2007:• Fully destructible environments • Dynamic Lighting & Shadows• Spherical harmonics• Normal mapping• HDR• Dynamic Weather• D3D9 ( SM 1.1, 2.0, 3.0)• D3D10 (SM 4.0)• Etc.

COH1 Forward Rendering Engine• Pros:

✔ Support for lots of old hardware by current standards.

✔ Proven technology.✔ Optimized for pre-DirectX11 hardware

• Cons:✗ Limited number of dynamic lights✗ Hugely complex ubershaders.✗ Static branching for performance: lots of auto

generated shader permutations.✗ Even with packed data used almost all available

pixel shader interpolators.✗ Literally no space in the shaders to add the new

features needed for COH2 (snow, etc.).

Ambient Occlusion

Near Shadow

Far Shadow

Water View

World View

Water View

Post Processing

Why Deferred Rendering for COH2

Cons:✔ Unable to handle transparent objects✔ Antialiasing is harder to do properly✔ Not very optimal for older generations of D3D9 HardwarePros:✔ Unlimited number of lights✔ Less shader permutations✔ Hugely simplified and faster shader code✔ Easier to add modern rendering techniques like (SSAO,etc)✔ Ability to add the needed features for COH2 to represent winter

and still lots of room to spare to add more features in the future.

COH2 Rendering

Top Down Map

Sun Near Shadow

Sun Far Shadow

Render Sun Light

Compute Next Light Shadow Map

Render Next Light

MoreVisibleLights?

Depth and Simplified Gbuffer Pass

Light Pass

Combiner Pass

Water View

Light Pass

No

World View

Post Processing

Yes

Track Map

COH2 Rendering

Sun Near Shadow

Sun Far Shadow

Post-Processing

Depth Prepass

Gbuffer Pass

SSAO / LAO Pass

Light Pass

Render Sun Light

MoreVisibleLights?

Combine/Resolve Pass

Forward Pass (Fx/Transparent Objects)

Depth and Simplified Gbuffer Pass

Light Pass

Combiner Pass

Water View

World View

Light Pass

No

Top Down Map

Track Map

Compute Next Light Shadow Map

Render Next Light

Yes

COH2 Rendering

Acknowledgements

• Thanks to Remy Saville, Ian Thompson, Bartek Nowakowski, Stuart Maxwell, Alun Bjorskten and James Harrison for their contributions to COH2 winter tech.

• Thanks to all the great Artists at Relic that keep pushing the limit of what the engine can do and keep us busy with more ideas of how to make it better.

Thank you

• Questions?

• Contact Information:

• daniel.barrero@gmail.com• http://danielbarrero.com