Per-Face Texture Mapping for Realtime Rendering
Transcript of Per-Face Texture Mapping for Realtime Rendering
![Page 1: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/1.jpg)
![Page 2: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/2.jpg)
Per-Face Texture Mapping for
Realtime Rendering ―Realtime Ptex‖
![Page 3: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/3.jpg)
Quick Note
• For CUDA/Compute folks:
– Ptex != PTX
![Page 4: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/4.jpg)
Goals
Render native Ptex datasets in real-time on commodity
hardware
Remove texture seams from textured models
Remove expensive, manual model unwrap step from art
pipeline
Support arbitrary resolutions on a per-face basis
![Page 5: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/5.jpg)
Video Time!
![Page 6: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/6.jpg)
Video Recap
Stored as 8Kx8K
Rendered as 2Kx2K
![Page 7: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/7.jpg)
Model Statistics
• ~800 FPS on GTX 460 with no optimization
• 278M of Color Texture Data
• 5812 Patches
![Page 8: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/8.jpg)
General Steps
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 9: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/9.jpg)
Load Model
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 10: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/10.jpg)
Load Model
• Vertex Data
– Any geometry arranged as a quad-based mesh
– Example: Wavefront OBJ
• Patch Texture
– Power-of-two texture images
• Adjacency Information
– 4 Neighbors of each quad patch
• Easily load with library available from http://ptex.us/
![Page 11: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/11.jpg)
Load Model (cont’d)
• Texel Data
– Per face, load the largest available mipmap level from the
source files
– In memory, place the loaded texel data into a memory
buffer that has a fixed-size texel border region
– The borders must be big enough for the largest filter kernel
you will support (Border size = ½ filter kernel size)
• Bilinear (2x2 kernel): 1 pixel border
• 16x Aniso (16x16 kernel): 8 pixel border
![Page 12: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/12.jpg)
Load Model
VB: …
IB:
![Page 13: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/13.jpg)
Load Model
VB: …
IB:
![Page 14: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/14.jpg)
Bucket and Sort
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 15: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/15.jpg)
Bucket and Sort
• Bucket ptex surfaces into groups by Aspect Ratio
– Each Aspect Ratio will be one bucket
– 1:1 bucket, 2:1 bucket, 4:1 bucket, etc
• Then, within each bucket, sort by decreasing surface
size and assign IDs.
– This allows us to densely pack texture arrays, avoiding ―empty‖
surfaces.
![Page 16: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/16.jpg)
Bucket and Sort
Bucket 4:1
Bucket 1:1
0
![Page 17: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/17.jpg)
Reorder Index Buffer
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 18: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/18.jpg)
Reorder Index Buffer
• Original • Post Sort, we have 2!
IB:
IB(1:1):
IB(4:1):
![Page 19: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/19.jpg)
Generate Mipmaps
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 20: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/20.jpg)
Generate Mipmaps
• Walk through surfaces, and generate a mipmap chain
from native size to (MinFilterSize x MinFilterSize) for
each surface.
– Bilinear stops at 2x2, 8xAniso stops at 8x8
• Ptex data files do not guarantee complete mipmap
chains—although the library can generate all levels for
you—with pre-multiplied alpha.
• Mipmap chains stop to allow for unique pinning values
in the corners
![Page 21: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/21.jpg)
Generate Mipmaps
• Done for every surface, but only inside the surface—the
border is not touched.
![Page 22: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/22.jpg)
Fill Borders
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 23: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/23.jpg)
Fill Borders
• Copy neighbor texels into border area of this surface’s
mip level
– Match source and destination number of pixels when
possible
• Bordered textures are the heart of the logical realtime
ptex solution
• Allows 1-2 texture lookups per ptex sample request
– 1 if not performing tween-mip-level interpolation, 2 otherwise
![Page 24: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/24.jpg)
Fill Borders
![Page 25: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/25.jpg)
Fill Borders
![Page 26: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/26.jpg)
Fill Borders
![Page 27: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/27.jpg)
Fill Borders
![Page 28: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/28.jpg)
Fill Borders
![Page 29: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/29.jpg)
Pack Texture Arrays
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 30: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/30.jpg)
Texture Arrays
• Like 3D / Volume Textures, except:
– No filtering between 2D slices
– Only X and Y decrease with mipmap level (Z doesn’t)
– Z indexed by integer index, not [0,1]
• E.g. (0.5, 0.5, 4) would be (0.5, 0.5) from the 5th slice
• API Support
– Direct3D 10+: Texture2DArray
– OpenGL 3.0+: GL_TEXTURE_2D_ARRAY
![Page 31: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/31.jpg)
Pack Texture Arrays
• Copy all generated data into Texture2DArray
• Each Texture2DArray represents a single mipmap level
– Texture2DArrays present a view of the data that is efficient
for GPU layout
– Logical Textures cut across the same page index of every
Texture2DArray
![Page 32: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/32.jpg)
Pack Texture Arrays
0 1 2
3
Logical Texture Layout
![Page 33: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/33.jpg)
Pack Texture Arrays
10x10x3 (1+8+1)x(1+8+1)x3
6x6x4 (1+4+1)x(1+4+1)x4
4x4x4 (1+2+1)x(1+2+1)x4
GPU Layout Texture2DArray
gColor[0]
gColor[1]
gColor[2]
![Page 34: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/34.jpg)
Pack Patch Constants
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 35: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/35.jpg)
Pack Patch Constants
• Each primitive has a ―PatchInfo‖ struct:
– TextureId – which array slice contains our data
– TopMipLevel – the index of the top-most mipmap level for this
texture
– FlipUVs – whether or not to flip UVs, allows 1:2 and 2:1 to be
grouped into same bucket
– MaxMipLevels – Maximum mipmap level for each edge
![Page 36: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/36.jpg)
Fill Borders
Load
Model
Render
Preprocess
Draw Time
Bucket
and
Sort
Generate
Mipmaps
Fill
Borders
Pack
Texture
Arrays
Reorder
Index
Buffer
Pack
Patch
Constants
Red: Vertex and Index data
Green: Patch Constant information
Blue: Texel data
Orange: Adjacency information
![Page 37: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/37.jpg)
Render
• Brief Recap of D3D11 Pipe stages
New Direct3D 11 Stages
IA VS HS TS DS GS RAS PS OM
Programmable (Shader)
Fixed Function
![Page 38: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/38.jpg)
Render
• In the Hull Shader
– Store pre-expansion PrimitiveID to output control points
– This is used everywhere to determine which set of Patch
Constants are owned by the currently running thread (in
Domain, Geometry or Pixel Shaders)
New Direct3D 11 Stages
IA VS HS TS DS GS RAS PS OM
![Page 39: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/39.jpg)
Render
• In the Domain Shader
– Vertices belonging to a quad meshes are evaluated with a
domain location, which is (0,0)-(1,1) for each patch
– Use this value to store our UV location
New Direct3D 11 Stages
IA VS HS TS DS GS RAS PS OM
![Page 40: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/40.jpg)
Render
• Texture lookups in Domain or Pixel Shader are replaced
with a ―ptex‖ sample function.
– Determines which logical texture to work from
– Compute mipmap level(s) to access
– Scale and bias computed (u,v) by mipmap size
– Lookup texels, return weighted average
![Page 41: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/41.jpg)
Texture Lookup Shader Code
• Traditional (D3D11)
return
gTxDiffuse.Sample(
gSampler,
I.fTextureUV );
• Ptex
return
ptex( gTxDiffuse,
gSampler,
I.uPrimitiveId,
I.fTextureUV );
Complex logic hidden
in single function call
![Page 42: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/42.jpg)
Questions?
• jmcdonald at nvidia dot com
• Brent dot Burley at disneyanimation dot com
• http://ptex.us/
• http://groups.google.com/group/ptex
• Or visit our studio session
– Monday, 8 August @ 4:30 PM – 5:00 PM
– The Studio / West Building, Ballroom A
![Page 43: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/43.jpg)
Additional Considerations
• Filtering across edges with differing numbers of pixels
• Filtering across corners
![Page 44: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/44.jpg)
Filtering across edges
![Page 45: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/45.jpg)
Filtering across edges
![Page 46: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/46.jpg)
Filtering across edges
![Page 47: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/47.jpg)
Filtering across corners
• Corners with valence > 4
cannot be exactly
matched with a
bilerp (4 samples)
![Page 48: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/48.jpg)
Filtering across corners
• The solution is a bit more involved.
– First, walk the mesh and determine which corners are shared
– For each shared group, determine the correct value for when
we’re exactly at that corner. E.g. Simple Average.
– Then, modify every mipmap level of every surface of that
group s.t. the shared corner has the same value
– When you’re in the corner, everyone will perform the same
lookup—regardless of mipmap level—and continuity prevails
![Page 49: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/49.jpg)
Filtering across corners
• For Realtime Ptex, we apply pinning to all corners,
regardless of valence.
![Page 50: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/50.jpg)
Pinned Corners
![Page 51: Per-Face Texture Mapping for Realtime Rendering](https://reader030.fdocuments.us/reader030/viewer/2022012520/61952a435edff12d594ecd67/html5/thumbnails/51.jpg)
Questions redux?
• jmcdonald at nvidia dot com
• Brent dot Burley at disneyanimation dot com
• http://ptex.us/
• http://groups.google.com/group/ptex
• Or visit our studio session
– Monday, 8 August @ 4:30 PM – 5:00 PM
– The Studio / West Building, Ballroom A