Kumar, RogerSepiashvili, DavidXie, Dan

18-796

Professor ChenApril 19, 1999

Progressive 3D Mesh Coding

• Synthetic/Natural Hybrid Coding (SNHC) in MPEG-4– A/V plane with 2D overlays, text, 3D objects, faces, etc.

• MPEG-4 System and Description Languages (MSDL)– MSDL will provide flexible run-time environment for invoking

decompression tools, algorithms, etc.

– MSDL will include class libraries for A/V object types along with their bitstream syntax.

– Client will download A/V object types for a specific session from the server at start up.

• MSDL A/V Objects…– VRML– ActiveX Animation– Java Media 3D– etc.

3D Models - MPEG-4 Overview

• MPEG-4 V2 will provide tools for coding 3-D Objects– Coding of generic 3-D Meshes

– Level of Detail (LOD) Scalability

• Decoder can render simplified version of the mesh

• Useful for viewing distant objects & for less-powerful rendering engines.

– Spatial Scalability

• Decoder can render the mesh at reduced spatial resolution

• Used in combination with LOD Scalability

– Progressive Transmission

• to receive progressively better approximations to the model

• possible approaches are:

– transmission of successive LOD approximations

– progressive mesh coding (discussed later)

3D Models - MPEG-4 Overview

• Texture Coding and Compression– Static textures can be compressed by JPEG– Moving textures can be compressed by MPEG-1 or H.263

• Geometry Coding and Compression– There is no standard for geometry coding and compression.– Most often geometry is represented by a triangular 3D mesh.– Algorithms that quantize vertex positions and normals, and then

apply Huffman or entropy encoding can achieve compression in the range of 15:1 to 65:1, depending on the nature of the model.

– Mechanical models are easier to compress than anatomical.Need for Geometry Compression:– Each vertex represented by three floating pt. numbers.– If each vertex shared by six polygons, and max number

of vertices per model is 2^20, then 76 bits/triangle needed.– Several hundred Kilobytes for

average model (just geometry)

Compression of 3D Models

triangle

bits

IDvertex

bits

triangle

IDsvertex

vertex

bits

triangle

vertices 7620*

332*

3*

6

1

Progressive Mesh (PM) addresses these issues:• Progressive Transmission

– show progressively better approximations to the model• Smooth Visual Transition (geomorphs)

– transit smoothly between different levels of approximation• Selective Refinement

– spatial refinement– level of detail refinement

• Mesh Optimization– constructing approximation

meshes from non-optimal scannedmeshes with smaller number of vertices

• Mesh Compression– mesh simplification algorithm allows very efficient coding

• Can Be Lossless (if no optimization was done)

Why Progressive Mesh Coding ?

Original mesh M^ is stored as a coarse mesh M0 and n detail records that indicate how to refine M0 into the original mesh M^.

• Mesh Optimization– reduction of number of faces

• Mesh Simplification– construction of a coarse mesh M0

• Mesh Compression– storage space reduction

Progressive Mesh (PM)

^10 110 ... MMMM nvsplitvsplitvsplit n

• Find mesh M that accurately fits the model and has a small number of vertices.

• Goal is to optimize for rendering efficiency• Often requires significant user intervention• Done by minimizing the energy function

Edist(M) = total squared distance of points of original model from the mesh

– measures accuracy

Erep(M) is proportional to the number of vertices in the mesh

– measures conciseness

Espring(M) is energy of a spring placed on each edge of a mesh

– penalizes vertices that are too far away from adjacent vertices

)()()()( MEMEMEME springrepdist

2

,

)( kj

kjspring vvkME

PM - Mesh Optimization

Original mesh M^ is stored as a coarse mesh M0 and n detail records that indicate how to refine M0 into the original mesh M^.

• Can simplify mesh using edge collapse, edge split, edge swap. Only edge collapse is needed.

• Edge collapse is fully invertible, so inverse transformation reconstructs the original mesh.

^10 110 ... MMMM nvsplitvsplitvsplit n

PM - Mesh Simplification

• Done by minimizing the energy function

Edist(M), Espring(M) are same as before

Escalar(M) measures accuracy of scalar attributes• are properties of corners (i.e. [vertex,face] tuples)

• examples are: diffuse color, texture, etc.

Edisc(M) measures geometric accuracy of discontinuity curves• preserving sharp edges is important

• Edisc(M) is equal to the total squared distance of a set of points sampled from sharp edges (discontinuity curves) of M^ to the discontinuity curve they belong.

• Algorithm:– mark all candidates for edge collapse in terms of priority, where priority of

each transformation is estimated by energy cost E.– starting with top at priority queue, perform edge collapse and recompute

priorities of edges in neighborhood of this edge collapse.– Keep doing until desired number of faces is met.

PM - Mesh Simplification

)()()()()( MEMEMEMEME discscalarspringdist

• Progressive Meshes (Microsoft Research)– http://www.research.microsoft.com/~hoppe

• Java 3D API - 3D Geometry Compression– http://www.javasoft.com/products/java-media/3D

/forDevelopers/j3dguide/AppendixCompress.doc.html

References

All movies are made by Hugues Hoppe

http://www.research.microsoft.com/~hoppe

• Did you mention that "Java3D also does compression"? How? – Our first slide stated that the MPEG-4 System and Description Languages (MSDL) will allow 3D geometry

to be compressed with different algorithms, one of which could be Java Media 3D (Java3D). MPEG-4 will also specify an built-in algorithm for geometry compression (most likely Progressive Mesh). If a better compression scheme is developed a few years later, MSDL would allow MPEG-4 to use this algorithm.

– Java3D specifies an algorithm for geometry compression. This algorithm does not allow for progressive transmission of meshes, as the Progressive Mesh (PM) algorithm does.

– In the Java3D algorithm, a 3D geometry is converted into a generalized triangular mesh. In Java3D, the vertex points are quantized, and then difference between two quantized vertex points is Huffman encoded. The vertex information is stored along with values for diffuse color and a normal vector.

– Color information is also quantized and Huffman encoded, whereas Normal information is encoded using a more complicated scheme (mapping normals to points on the unit sphere, please see http://www.javasoft.com/products/java-media/3D/forDevelopers/j3dguide/AppendixCompress.doc.html section B.8).

– If no color or normal information is sent for a vertex, the color or normal information for the previously-sent vertex is used.

Question #1

• What is the speed for Mesh Optimization? – In our presentation, we stated that Mesh Optimization is quite slow. Below are exact numbers and

examples.

Question #2

Mesh Optimization: 3152 faces to 334 faces17.0 minutes

Mesh Optimization: 18,274 faces to 1348 faces47.0 minutes

Mesh Optimization: 8073 faces to 515 faces44.5 minutes

Mesh Optimization: 3832 faces to 432 faces10.2 minutes

Timings preformed by Hugues Hoppe on a DEC Workstation in 1993.http://research.microsoft.com/~hoppe