Scientific VisualizationFall 2015-11-10aht/Vis2015/RenderingTechs_SeSE_Vis2015.pdf · - Scientific...

Department of Information Technology

- Scientific Computing

Scientific VisualizationFall 2015-11-10

Stefan Seipel

Ch 7: Transparency

Texture mapping

Direct Volume Rendering (DVR)

Ray Casting

Splatting



Transparency in Visualization

Transparent visualizations used to show structures in spatial context rather than showing detail



Transparency and Alpha Values

Opaque objects reflect, scatter, or absorb light on the surface – no light is transmitted

The level of transparency/opacity of a surface is referred to as alpha in computer graphics

alpha = 0.5 -> object is semitransparent

alpha = 1.0 -> object is fully opaque

alpha = 0.0 -> object id fully transparent

On graphics cards the frame buffer can store the alpha value along with the RGB values



What is alpha compositing?

A method for combining colors of objects in a way that maintains the intervisibility of the scenes

“over” =

“under” =



Alpha compositing: Colors are blended, weighting their alpha values

The color of a pixel is represented by a quadruple (R,G,B,A)

𝑅 = 𝐴𝑆𝑅𝑆+(1-𝐴𝑆) 𝑅𝐵 𝐺 = 𝐴𝑆𝐺𝑆+(1-𝐴𝑆) 𝐺𝐵 𝐵 = 𝐴𝑆𝐵𝑆+(1-𝐴𝑆) 𝐵𝐵 𝐴 = 𝐴𝑆+(1-𝐴𝑆) 𝐴𝐵

0.0, 0.0, 1.0, 1.0 = 𝑜𝑜𝑜𝑜𝑜𝑜 𝑏𝑏𝑜𝑜 0.0, 1.0, 0.0, 0.5 = 𝑠𝑜𝑠𝑠𝑠𝑠𝑜𝑠𝑠𝑜𝑜𝑠𝑜𝑠𝑠 𝑔𝑠𝑜𝑜𝑠 1.0, 0.0, 0.0, 0.0 = 𝑓𝑜𝑏𝑏𝑓 𝑠𝑠𝑜𝑠𝑠𝑜𝑜𝑠𝑜𝑠𝑠 𝑠𝑜𝑟, 𝑠. 𝑜. 𝑠𝑠𝑖𝑠𝑠𝑠𝑏𝑏𝑜

Subscript S indicates current surface to render, and B indicates color behind the object (i.e. what is presently in the frame buffer).



Color of object 2: R = 100 G = 40 B = 200 A = 0.7

Example: Alpha blending Example: Object 1 behind Object 2

Color of object 1: R = 200 G = 100 B = 20 A = 1.0

Color of Background: R = 0 G = 0 B = 0 A = 1.0

1. Object 1 over background: Rout = 200*1.0 + (1.0-1.0)*0 = 200 Gout = 100*1.0 + (1.0-1.0)*0 = 100 Bout = 20*1.0 + (1.0-1.0)*0 = 20

2. Object 2 over background: Rout = 100*0.7 + (1.0-0.7)*0 = 70 Gout = 40*0.7 + (1.0-0.7)*0 = 28 Bout = 200*0.7 + (1.0-0.7)*0 = 140

3. Object 2 over Object 1: Rout = 100*0.7 + (1.0-0.7)*200 = 70+60=130 Gout = 40*0.7 + (1.0-0.7)*100 = 28+30=58 Bout = 200*0.7 + (1.0-0.7)*20 = 140+6=146

Background

Object 2

Object 1



A problem with transparency rendering

• Objects must be drawn in the correct order (back-to-front) for object order rendering algorithms like polygon rendering

• In VTK, this is solved by sorting all objects prior to rendering

• This makes transparency rendering quite slow

• In raycasting/raytracing, samples are always ordered



Texture mapping

2D discrete image 2D polygon (defined in 3D space)

+

Texture-mapped polygon

Applying 2D texture maps to the surface of an object is analogous to pasting a picture. The location of the texture map is specified via texture coordinates.



Texture mapping, cont´d

(0,0) (1,0)

(0,1) (1,1)

Each texel has 2D coordinates assigned to it

Assign texture coordinates to each polygon to establish the mapping

(0,0.5) (0.5,0.5)

(0,0) (0.5,0)



Texture mapping

Purposes: 1. Additional detail is introduced without extensive geometric modeling. 2. Visualization of sampled data

This vertex of the triangle has a coordinate (x,y,z) in a Cartesian coordinate system as well as an associated texture coordinate (u,v) in 2D texture image space.



Textures …

• can be images of the real world (discrete)

• can represent (multi)scalar data sampled on a regular grid

• are often (but do not have to be) 2D discrete images

• can be sampled data on 3D grids and are then referred to as volumetric textures

• can represent computed data on 2D or 3D grids, either with procedural algorithms or by evaluating continuous functions



Alpha value in texture mapping

• By adding alpha values to make an RGBA texture map, parts of the texture become transparent

• Trick in computer graphics

Example: Trees have a complex shape

Render a rectangular RGBA texture map per tree or per tree component leaf/trunks: α = 1 gaps and open space: α = 0

In scientific visualization, volume rendering can be accomplished by compositing a stack of partly transparent 2D textures.



Digital 3D volumetric images

Three-dimensional functions f(x, y, z)

Often sampled medical data from CT, MR, PET, SPECT, …

How can this kind of data be visualized?

N x 2D arrays = 3D array



Example: Visual exploration of Magnetic Resonance Angiography (MRA)



Slice by slice



Slice by slice

Issues:

• Spatial context

• Short term visual memory



Multi-Planar Reformatting (MPR)



(MPR) - Example



A direct ”3D” visualization (accomplished by transparent rendering of a stack of 2D slices)

3D imaging of the blood vessel system



3D Volume Rendering Categories of algorithms

3D Object representation Im

age

form

atio

n

Explicit Implicit

Object Order

e.g. Polygon Rasterizer

Volume Splatting

Volume Raytracing

or Raycasting

e.g. Polygon Raytracing

or Raycasting

Image Order



Rendering techniques

Image order rendering techniques: • Image is rendered for each image element

(pixels)

• Ray-casting and ray-tracing most common techniques

• Simulate the interaction of light with objects by following the path of each light ray

• Objects can be polygonal surfaces, parametric surfaces or discrete (3D) images



Ray-casting or ray-tracing (here: explicit geometric models)

Ray-casting Ray-tracing

• image sampling in screen space

• establish ray from eye through pixel

into the scene

• local lighting model assumed at surface

(shading and specular highlights)

• same as ray casting plus:

• iterative back-tracing of reflected and

transmitted rays

• global lighting effects

(shadows, reflections, transparency,

atmospheric effects)



Ray-casting or ray-tracing (here: implicit models – sampled 3D data)

• image sampling in screen space

• 3D data set is sampled along the trajectory of the viewing ray

• profiles of sampled data are recorded for every pixel/ray

• pixel color is determined by compositing function

• there exist many different compositing techniques depending on

the desired visual effect

• more details, see Chapter 7



Ray casting

For each display pixel, cast one or more rays Calculate contribution C for each voxel along the ray

Colour C(x,y,z) determined by gradient (approximate surface normal) lighting (independent of other voxels between the point

and the light)

Opacity α(x,y,z) determined by mapping density values to different types of tissue through transfer functions



Ray casting issues Uniform sampling

Voxel by voxel traversal

What connectivity? Face (6) connected

Edge (18) connceted

Point (26) connected



Various rendering modes

Maximum intensity projection (MIP) Integral projection Distance to a certain threshold value Depth shading Depth gradient shading Grey-level gradient shading Combined multi-modal rendering …



Ray traversal schemes

Depth

Intensity Max

Average

Accumulate First

Study the profile!



Ray traversal - First

Depth

Intensity

First

First: extracts iso-surfaces



Ray traversal - Average

Depth

Intensity

Average

Average: produces basically an X-ray picture, an integral projection

Distance along ray can also be taken into account and used



Ray traversal - MIP

Depth

Intensity Max

Max: Maximum Intensity Projection used commonly for MRA images



Ray traversal - Accumulate

Depth

Intensity

Accumulate

Accumulate opacity while compositing colors: make transparent layers visible



Ray traversal – Sampling frequency

Depth

Intensity

Depth

Intensity

High sample frequency (short sampling intervals) are needed to represent the sampled signal without loss of information.

(->Nyquist-Shannon sampling theorem)

Low sample frequency (long sampling intervals) reduce computational efforts->increase rendering speed.

But the sampled signal is represented with loss of information->aliasing artifacts!



Ray sampling frequency

0.1 unit step size 1.0 unit step size 2.0 unit step size

Smoothness vs. computational cost (linear in time)



Grey-level gradient shading

A threshold is given and the rays are cast into the volume until a voxel above the threshold is encountered

The gradient at the points are determined and taken as a normal vector of an iso-surface at this point

Normal vector together with virtual eye-position and virtual light source are used to render voxel

Cut planes can be used to remove parts of the volume



Gradient/Normal estimation

Estimate gradient vector G per voxel using central differences of the density data D Gx(i,j,k) = D(i+1, j, k) - D(i-1, j, k) Gy(i,j,k) = D(i, j+1, k) - D(i, j-1, k) Gz(i,j,k) = D(i, j, k+1) - D(i, j, k-1)

Gradient g ->derivative of f in



Grey-level gradient shading



Multi-modal rendering

Co-register two data volumes with each other Rays are cast into the first volume For points on cut planes, values from both volumes

are blended to produce a colour that enhances both modalities

MRI +

PET



Rendering techniques

Object order rendering techniques: • Rendering is processed in the order of

graphical objects for every object at a time (points, polygons, or voxels)

• Pixels in the final images are written to many times

• Visibility problem (which object is closest)

• Non-visible objects are processed

• Rasterization and splatting are two typical examples



Polygon rasterization (for explicit geometric models)

• Vertices of polygons are projected from 3D object space onto 2D screen space

• Pixels in the interior area of a polygon on screen is then filled

• Compare 3.7 and slide further down

vp

0 1

2 3

1

2 3

0

3 2

0 1

0

2 3

0 1

1

3

2 3

2

0 1

Screen Space

Object Space

Transform &

Projection



Voxel splatting (for eg. volumetric data sets)

• A discrete voxel space is traversed in voxel order

• Voxel is projected onto image plane (position [x,y])

• A 2D footprint (splat) of the voxel is determined (size and shape)

• The color of all pixels in the image covered by the splat are

updated using compositing functions

• Requires strictly ordered traversal

(back-to-front / front-to-back) to

maintain correct visual results



Splatting (feed-forward) “filling the gaps”

Discrete sampling intervals in object space

Sample displacement in screen-space is transform dependent



Fill the gaps

We need to fill the pixel values between the volume projection samples

To fit a continuous function through the discrete samples Convolution can be used



Convolution

g(x,y)

f(i,j)

The output g is a weighted average of inputs f

g(x,y) = Σ Σ f(x-i,y-j) h (i, j)

i= - j= -

sample value

weight



Convolution, cont´d

Another way of thinking convolution is to deposit each function value to its neighbor pixels

f(i,j)

This weighting function is called kernel



3D kernel for splatting

Need to know the 3D extent of each voxel Project the extent to the image plane

?

Splatting

x This is called a footprint



Splatting

Footprints can be precalculated and saved in a lookup table. Therafter, pasted as needed

Splatting is more difficult for perspective volume rendering; image space extent is not identical for all samples

Considerations: type of kernel radius of kernel resolution of footprint table



Efficient Volume Rendering

Consider 512 x 512 x 512 data

At least one byte per element

134 217 728 bytes = 128 MB

How to render such data interactively?



Big data problem

One way to handle big data problem is to use multiple-resolution data structures (hierarchical volumes)

8x4x4 4x4x2 2x2x2 …

…

Less amount of data required



Adaptive methods: Octrees for volume rendering

• Volume dataset is recursively subdivided on different levels

• Empty nodes on different levels are labeled • Empty nodes are not rendered, thus avoiding the

processing of ”empty space” • Recursion depth of the octree determines

precision with which empty space can be avoided • Trade-off: high resolution octree requires a lot of

memory in itself

The following illustrations are curtsey of Daniel Blomqvist, student at HiG, Spring 2010



Octrees for volume rendering



Surface versus Volume Rendering

• Surface rendering + Standard computer graphics software can

be used - simple + Supported by dedicated hardware - fast + A high data reduction can be achieved - Intensity information is lost - Cutting through the data is meaningless - Changes in surface definition criteria

means recalculation of the data



Surface versus Volume Rendering

• Volume rendering + Arbitrary cuts can be made allowing

the user to see inside the data + Allows for display of different aspects MIPs semi-transparent surfaces etc.

+ Rendering parameters can be changed interactively

- Slow



Intermixing Volumes and Geometry

Presentatör

Presentationsanteckningar

Mycket svårt att spela in videosekvenser med haptisk feedback eftersom inspelningsprogrammet stjäl så mycket datorkraft. Kom och hälsa på istället!

Scientific VisualizationFall 2015-11-10aht/Vis2015/RenderingTechs_SeSE_Vis2015.pdf · - Scientific...

Documents

Transcript of Scientific VisualizationFall 2015-11-10aht/Vis2015/RenderingTechs_SeSE_Vis2015.pdf · - Scientific...