Image Synthesis

computer graphics & visualization

Image Synthesis

What is left to do?„The Real-World Problems!!!“


Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group

So far …• Realtime

– OpenGL / DirectX basics– Programmable pipeline– Effects

• Non-Realtime– Raytracing– Radiosity– Photon Mapping



What‘s left?• „Optimizations“

• „Problem size“

TerabyteTo

ExabyteDatasets!


Image Synthesis

Scenegraph APIs



Scenegraph APIs

• Rather cumbersome to use for complex applications• Slow development cycle

• Non-reusable code• Non-structured data often difficult to optimize

• Expert knowledge often necessary• Low-level und hardware orientiert• Imperative instead of descriptive• No support for interactions• Minimal support for picking• No data formats• Immediate mode not always desired

Low-level APIs like GL/DX as well as self writen raytracing/radiosity implementations have several disadvantages:



Goals of a high-level API • Easy to use, easy to extend

• Multi processing• Build-in methods for efficient scene handling and rendering

• Plattform and Hardware Independent

• Management of complex scenes

• Hiding implementation details

• High performance - optimization

• Reliability

• Object oriented model

• Descriptive scene description

• Interaction and picking

• Data formats



What is a scenegraph?• An ordered data structure that stores a description of the scene to be

rendered

• The scenegraph comes with a traversal mechanism used to process the scene description in a pre-determined order

• It is accompanied with a data base that provides objects needed in graphics applications



API provides the interface to the user

• Library calls to

• Create the scenegraph• Access the data base• Use auxiliar objects and methods

• Usually, the API provides additional components

• Viewer necessary to view the scene• Editors• Manipulators



Scenegraph APIs:Rendering modes• Immediate mode (low-level)

• Render everything as soon as specified• No intermediate representation

• Retained mode (high-level)

• Store the scene it in an intermediate representation• Optimize (re-organize/simplify) this representation for rendering purposes

• Compile-immediate mode

• Caching strategy used to optimize the rendering of static parts of the scene (display lists)



Scenegraph APIsA historic overview

– 1992 OpenInventor (SGI) ? – 1992 Iris Performer (SGI) – 1995 Cosmo3D (SGI) – 1996 Optimizer (SGI) – 1996 DirectModel/Jupiter (HP) – 1997 OpenGL++ (ARB-Sun,SGI,Intel,...) – 1997 Java3D (Sun) – 1998 Fahrenheit (SGI,MS) – 2000 OpenSG (OpenSource - Germany) – 2000 OpenScenegraph – 2003 NVSG NVidia Scenegraph ?



Scenegraph APIsFeatures of a scenegraph API

– The scene is stored in a hierarchically organized tree-like data structure

Nodes are the objects describing the sceneEdges are references that define the relationship between objects

Group

Light

Shape

Appearance Geometr

y

Transform



Scenegraph APIsThe scenegraph is implemented as a directed acyclic graph

– It organizes and controls the rendering of the scene– It provides fixed or application dependent traversal

order (Depth-first, breath-first, arbitrary)– It determines the way in which attributes are

propagated to/associated with objects



Scenegraph APIsEach node belongs to a specific class

– Shape nodes -> geometry– Appearance nodes -> material properties– Transform nodes -> local object transforms– Environment nodes -> lights, space properties– Camera nodes -> viewing parameters– Etc.



OpenInventorFirst widely used scenegraph from SGI

– IRIS Inventor based on IrisGL– 1994: OpenInventor based on OpenGL– Basis for many products and projects

• Iris Explorer Visualization Toolkit• Amira

– Since 2000 original in OpenSource– Multipipe, Stereo,

3D Textures, Kollisionen,...– Multiplatforms:

Linux, Windows– http://oss.sgi.com/projects/inventor/

http://oss.sgi.com/projects/inventor/



OpenInventor - Concepts• Scenegraph (DAG) for 3D-Models with many node types (Shape, Camera, Light, ...)

• Traversal with many actions (render, pick, bounding box, callback, ...)

• Interaction with fast picking: Manipulators

• GUI-Elements: Components (z.B. Viewer)

• Animation: Engines

• 3D-data format

• Plattform independent via OpenGL

• User defined extensions (ToolMaker)

• C++-Class library



OpenInventor - Nodes• Geometry: Shape Nodes

• Cube, Cone, Cylinder, Sphere, 2D/3D-Text• NURBS, Triangle Strips, Quad Meshes, Face Set,...

• Properties: Property Nodes• Material, Drawstyle, Environment, Texture, Light Model,...• Transformationes

• Grouping: Group Nodes

• Group, Separator, Switch, ...

• Virtual camera: Camera Nodes

• Perspective, Orthographic

• Lighting: Light Nodes

• Point, Directional, Spot, ...



OpenInventor - Example



OpenInventor - ExampleTable-scene with path to one leg



Different nodes trigger different actions• Render nodes provide a renderAction that implements the algorithm used

to render the object

• The renderAction is called in the current traversal state whenever a render node is visited

• State nodes just change the current render state without rendering anything

• Traversing the graph means visiting each node



Scenegraph APIsScenegraph traversal

– OpenInventor• Depth-first traversal• Inheritence of states from left to right and top to bottom

– Performer• Depth-first traversal• Different traversal processes (app., cull, draw, etc.)• No inheritance from left to right



OpenInventor - Details• Traversal from top to bottom and left to right• Update of traversal state

– Current transformation, geometry, material, texture, ...– Separator: push/pop of traversal state

• Multiple references to nodes possible– Shared instancing– Explicit reference counting with automatic delete (important for memory

management under C++)• Data elements of node: Fields

– Basis types: Bool, Color, Vec3f, Matrix, List, Name, String, ...– Single Value Fields and Multiple Value Fields– setValue/getValue and defaults – Ignore/Overwrite Flags



OpenInventor - Interactions• Events

– Window independent (Event Translator)– Processed by application, viewer or nodes– HandleEventAction: propagates events through graph until

one node reacts (grab)• Picking

– Yields path through scenegraph• Manipulators

– Interaction nodes with own geometry (Pick, Feedback)– Supports 3D-Navigation with 2D-Mouse (Projector)– Consists of Draggers (one Transformation)



OpenInventor - ManipulatorsHandlebox



OpenInventor - Manipulatoren Transformer Spot Light



OpenInventor - Viewer & Editors



OpenInventor - Scenegraph-EditorDirect modifikation of nodes in graph



OpenInventor – Data format#Inventor V2.0 asciiSeparator {

PerspectiveCamera {position 0 0 4.18154nearDistance 2.44949farDistance 5.91359focalDistance 4.18154

}DirectionalLight {}Material {

diffuseColor [ 1 1 0.2 ]}Cone { }

}



OpenInventor – Data format• OpenInventor-2.0-Format

– Basis for VRML 1.0– Only slight differences, e.g. nodes for Web-Access

• VRML 2.0 and following (VRML 97, X3D)– 3D-Formats for Internet– Take some OpenInventor concepts– But: structural changes



Scenegraph APIsSG APIs provide additional features to achieve interactive frame rates for large scenes

– Handling and manipulation of huge data structures– Multiprocessor/Multipipe rendering– Build-in components for efficient rendering of large

geometries– Pre-defined components, e.g. viewer including

event handling



Scenegraph APIsBasic scenegraph optimization techniques can be split into different categories

– Storage – Generate Stage– Graph traversal – Traversal Stage– Geometry processing – Transform Stage– Rasterization



Generate stage optimization• Storage

– Links instead of instances– Hierarchical partitioning of the scenegraph

• On-demand rendering from disk• Paging of (less frequently used) objects

– Multiprocessor-control• Thread manager• Synchronization of multiprozessor-SG traversal

– Geometry compression• Quantization and topology encoding

– Geometry simplification and optimization• Mesh reduction• Efficient conversion in strips and fans



Generate stage optimizationGeometry simplification

– Objects are represented with less triangles– Relevant features and the original shape within a

certain error tolerance are retained– Automatic selection of triangles to be removed

based on local curvature or other criteria– Automatic triangle removal

• Collapse, split, swap



Generate stage optimizationMesh reduction criteria

Mean-curvature

Gauss-curvature 2-i

Difference between area before and after reduction



Mesh analysisCriteria to analyze mesh quality

– Distance to other mesh, e.g. mesh before modification • Hausdorff-distance (two-sided):

dHaus(A,B) = max{maxaA dist(a,B), maxbBdist(A,b)} dist(a,B) = minbB dist(a,b) (one-side Hausdorff) dist(A,b) = minaA dist(a,b) (one-side Hausdorff) • Maximum of all shortest vertex-to-mesh distances



Mesh analysisHausdorff-distance dhaus=d



Mesh analysisHausdorff-distance

– d1=maxqQ dist(P,q)– d2=maxpP dist(p,Q)



Mesh analysisHausdorff-distance dhaus=d1=d2



Generate stage optimizationTriangle removal by re-tesselation

– Use gluTesselate to tesselate arbitrary polys



Generate stage optimization• n practical applications

– User defines compression factor– Different Levels-of-Detail are precomputed– Manual or automatic LOD selection during

rendering– E.g, less details are shown during movement or if

the object is far away



Generate stage optimizationGeometry simplification (5,10,50,100 %)

progressive

nonprogressive



Generate stage optimization• Geometry simplification

#: 1.087716 286578 30392 3774




Original

Simplified



Generate stage optimization• Geometry optimization

– Large redundant structures are stored into vertex arrays• Sharing of multiple defined geometry and attributes

– Re-organization into strips and fans• Adjacency information is exploited for efficient coding

0

1

2

3

4

5(012)(123)(234)(345) ->(012345) one strip

1

0

2 34 (012)(023)(034) -> (01234) one

fan



Traversal stage optimization• Improved traversal by re-organization of the graph

structure– Spatialization

• Nodes are re-arranged with respect to their spatial position or other attributes

• Can lead to a more efficient traversal order• Accelerated culling and picking

– Flattening the graph structure results in less recursion stages• Better memory efficiency• Less CPU load

– Group geometry with equal material– Minimize GPU state changes



Traversal stage optimization• Spatialization

- Shared Instancing: group nodes

- Build parallel SG of inner nodes for efficient culling

- Remove unnecessary hierarchy stages

- Resolve shared instances

- Allows for adaptive BV-Hierarchies



Transform stage optimization• Objects rendered multiple times are stored into OpenGL

display lists• Renderable primitives that won´t be displayed are

automatically determined and culled– Culling: discard invisible objects before they are transfered to

the GPU• backface culling

discard backfacing facets• detail culling

discard „small“ objects• view frustrum culling

discard objects outside view frustum• occlusion culling

discard occluded objects



Transform stage optimization• Backface culling

– Triangles with a normal pointing away from the viewer are discarded

– Efficient computation by means of the area of the polygon in window coordinates

– Might lead tostrange results



Transform stage optimizationView frustum culling

– Parts of the scene outside the view frustum are discarded– Not supported by OpenGL; clipping is performed after

transformation to screen space coordinates– Efficient intersection test between objects and frustum

necessary



Transform stage optimizationOcclusion culling

– Parts of the scene that are entirely hidden by others are discarded

Occluded parts




– Sorting and front-to-back rendering necessary• Sorting key is distance to viewer

– Occlusions are determined in two stages:• Specify less complex enclosing hull (BoundingVolume)

and compute its projected screen rectangle• Grab the depth values within the rectangle and compare

them to the minimal depth between convex hull and view plane

– If at least one depth value is larger, the object has to be rendered




– Hardware support exists in OpenGL 1.5• A specific render mode is provided that allows to render

objects without affecting the color/depth buffer• An OcclusionQuery can be used (with BoundingVolume)

to determine the number of fragments that have been affected

– If no fragment has been affected, the object is invisible and can be culled, i.e. has not to be rendered



Transform stage optimization• Detail culling

– Objects that are small compared to the viewport resolution are discarded or represented at a coarse level

• E.g. if the entire object is projected into a few pixel



Transform stage optimizationSpace partitioning and portal culling

– Most frequently used in game engines– Partition the scene into disjoined parts and classify which

parts can´t see each other• Define portals through which one part can see another one

– While in one part, discard everything that can´t be seen and clip against the portal frustum



Transform stage optimizationFor efficient occlusion culling the API needs to compute ´good´ enclosing hulls

– Should be as close as possible to the surrounding geometry

– Should be as simple as possible for effcient computation and rendering

– Most commonly, bounding volumes are used

Axis aligned

Oriented



Transform stage optimizationBounding volumes

– Minimizing the number of computations necessary ... • ... to check wether an object might be hit or not• ... to check wether an object is contained in a cell or not

Bounding spheres axis-aligned BB oriented BB more complex intersection test



Transform stage optimization• Bounding volume hierarchy

– Adaptive space partitioning– Search complexity is O(logn)



Transform stage optimization• Bounding volume hierarchies

– Manually define BVs for sets of objects – Bottom-up approach

• Merge BVs of objects• Consider size of sectional areas• Consider number of objects

– Top-down approach• Recursively build smaller BVs• Subdivide underlying space

– Consider load needed to traverse each sub-space



Transform stage optimizationBounding volume hierarchies for triangle meshes

– Interpret mesh as point cloud– Successively generate smaller BVs

• Consider number of elements in each BV• Consider size of bounding volumes

OOBBs



Transform stage optimization• Normal-Maps for correct highlights• Decoupling of geometry and illumination



Transform stage optimization• For efficient occlusion culling the API needs to

compute ´good´ space partitions• Should allow for efficient visibility ordering of

objects• Most commonly tree structures are used

– Octrees– BSP-Trees– KD-Trees



Transform stage optimization• Space partitioning

– Minimizing the number of objects to be rendered by visibility ordering

– Grids partition the space into equidistant cells– Cells store links to all objects they touch– Grid cells are ordered with respect to current

viewpoint and viewing direction



Transform stage optimization• Grids

– Structure – Traversal

Ordering of cells easy to determine



Transform stage optimizationOctrees

– Hierarchical space partitioning• Start with one cell that covers everything• Recursively subdivide it into 2x2x2 äquidistant sub-cells• Each sub-cell is going to be subdivided further on if it

meats a certain criterion – E.g. it still contains too many objects



Transform stage optimizationOctrees

– Complex traversal algorithm • Varying distances between sub-cell borders at different levels• Recursive traversal and finding of neighbors in non-balanced trees



Transform stage optimization• Grids vs. Octrees

– Octrees:• Arbitrary but adaptive granularity• Non-simple traversal algorithms

– Grids:• Simple and efficient traversal• High memory requirement • Adaptivity by hierachical grids



Transform stage optimizationOctrees vs. Grids



Transform stage optimizationkD-Trees

– A tree where each node corresponds to an axis parallel split of the domain

• Objects are located in the leaves– Splitting is performed in alternating directions



Transform stage optimizationkD-Trees

– If a node is completely contained in a region, then all of its children are

-> output this subtree– Otherwise, recursively traverse the intersected subtree



Transform stage optimizationBsp-Trees

– Partition the space into two half-spaces at each level

– Continue with the same procedure for each of the half-spaces

– Choose arbitrary positions and orientation of the dividing planes

• More efficient scene partitioning than with octrees • Process half-space that contains viewpoint first



Transform stage optimizationOctrees vs. kD vs. Bsp

– Less objects have to be stored multiple times for kD/Bsp-trees

– More flexible/efficient scene partitioning than with octrees

– Traversing kD/Bsp-trees is more complex• Test with arbitrarily oriented dividing planes



Transform stage optimizationMultiprocessor rendering

– Parallelization of the rendering process difficult in general

• Explicit sorting necessary, i.e. transparent objects• Occlusion culling almost impossible• Good partitioning in order to achieve load balancing• High memory bandwidth necessary in order to merge

partial results rendered to multiple screens• Semaphor concepts needed to render from multiple

processors into one screen



Transform stage optimizationHow to parallelize ?

A)• Store the entire scene on each processor• Assign disjoint screen regions to each processor• Each unit renders the scene onto its local screen region• Append the results

B)• Partition the scene into disjoint, contiguous parts and distribute• Each unit renders it´s partition onto its local screen• Blend the results together

C)• Divide the rendering process into different tasks (Iris Performer)

that are assigned to processors• Pipeline the computation



Transform stage optimization• Performer style parallelization

– Rendering is split into• APP: the application dependent tasks• CULL: culling of non-visible geometry• DRAW: rendering the scene

– After the third rendering pass all three processes are busy– This strategy might lead to bad load balancing– Only three processors can be used in parallel

App 1 App 2 App 3Cull 1 Cull 2 Cull 3

Draw 1 Draw 2 Draw 3

CPU 1

CPU 2

CPU 3



Transform stage optimization• Synchronization

– Rendering output of multipe processors/graphics pipelines have to be synchronized

• Application in Tiled displays– Frame-to-frame synchronization

• Synchronize glSwapBuffer() using proc-to-proc communication, i.e. TCP/IP, MPI etc.

– Genlock-synchronization • Workstation’s graphics output is locked (LOCK) to an externally

generated (GEN) signal• Some graphics hardware ensures subscanline-level synchronization

to an external sync source• Synchronize with vertical refresh rate



Rasterization stage optimization• Build-in components for improved rasterization

– ???– Texture compression already available in low-level APIs

• Multiple texture entries are coded by their difference to an average value

• Less bits spent to adress the difference – Use textures instead of geometry

• Billboards (an image of a part of the scene) might be useful to display static parts of complex geometry

• Lightmaps, cubemaps etc. – Per-pixel deferred shading to render complex geometry

• Shift shading calulations to the fragment stage

Image Synthesis

Documents

Transcript of Image Synthesis