Visualization on BioHPC

1 Updated for 2019-05-15

[web] portal.biohpc.swmed.edu

[email] biohpc-help@utsouthwestern.edu

Scientific Visualization : from multiple datasets to a single image/view

2

Red blood cell

https://www.alcf.anl.gov/user-guides/vis-paraview-red-blood-cell-tutorial

unstructured mesh

streamline (velocity)xy-plane slice (based on velocity)

diseased cell

healthy cellparticles

From image to understanding – workflow of whole brain image analysis

3

TissueVision TissueCyte 1000

7 by 14 mosaic~ 1.2 MB / tile

filtering stitching 3D renderingresampling

XY planeFactor = 4

~ 120 MB / section 150 sections

3D stack16GB -> 1GB

Segmen

tation

analysis

find out cells and fibers

* Images and workflow were provided by Denise Ramirez from WBMF

From image to understanding – workflow of visualizing the native ultrastructure of motile cilia

4

Tecnai G2 F30 Cryo-electron tomography 66 tile series

Image alignment and calculation of 3D tomogram

3D subtomogram averaging of axonemal repeats

* Images and workflow were provided by Kangkang Song from Nicastro LabNative 3D structure of axonemal repeat

From image to understanding – workflow of bleb detection and tracking

5

~ 18.2 KB / slice160 slice

Extract Surface3D rendering

prep

rocessin

g

MESPIM2-photon Bessel Beam Light Sheet Microscope

~ 13.8 MB

SegmentationLocating Protrusions

• Images were taken by Reto Fiolka, Kevin Dean, and Erik Welf from Danuser Lab• Bleb detection and tracking were processed by Meghan Driscoll from Danuser Lab

Hardware for Visualization on BioHPC

6

Not all visualization needs GPUs – 424 CPU nodes are at your service• Complex 2D visualization, simple / small 3D visualization• Final high quality rendering

When you do need 3D rendering we have nodes with various capability GPUs

Partition Name Number of Nodes GPU Type

GPU 8 Tesla K20/K40/K6000

GPUp4 16 Tesla P4

GPUp40 16 Tesla P40

GPUp100 12 2x Tesla P100

GPUv100 2 2x Tesla V100

GPUv100s 2 Tesla V100

Using dual GPU for visualization isn’t trivial – requires software to support this – please speak to BioHPC

How much GPU do I need?

7

The biggest factor for visualization is likely to be GPU vRAM vs data size.But some software can handle data >> GPU vRAM

Partition Architecture CUDA Cores vRAM

GPU Kepler 2880 >=5GB

GPUp4 Pascal 2560 8GB

GPUp40 Pascal 3840 24GB

GPUv100s Volta 5120 32GB

How Much GPU do I need?

8

GPU vRAM required doesn’t necessarily match your raw data size

Example: 3D image --> IsoVolume --> Extract Surface

Grid Type No. Points Memory

3D image Structured 7,362,900 14 MB

IsoVolume Unstructured 1,736,075 200 MB

Extracted Surface Polygonal Mesh 1,074,887 65 MB

But games look so cool without a $10k 32GB GPU card….

9

Scientific visualizations are generally rendering the true surface/shape of objects

Games use clever tricks:• Texture and bump/normal/displacement mapping give additional impression of

depth and texture on simple shapes. • It’s much simpler to lay textures over a simple shape, or use bump mapping to

alter lighting/shading than render the exact surface structure.

Image: https://en.wikipedia.org/wiki/Bump_mapping

Complexity of 3D volume rendering

10

Most objects in games are opaque, relatively simple shapes, modified with textures / surface maps.

Many scientific datasets are complex-surface 3D volumes, where the interior of the volume is also important to represent.

Image: https://en.wikipedia.org/wiki/Volume_rendering

Linux webGPU Sessions

11

BioHPC webGPU : launch a graphical environment on the Nucleus cluster

Launch sessions from the web: https://portal.biohpc.swmed.edu/terminal/webgui/

• WebGUIReserves a CPU node – useful for quick access for simple visualization tasks

• WebGPUChoose relevant GPU for your workAdd visualization software as a modulevglrun <software name>

Ensure the Best Visualization Experience

12

Use a wired network connection

Pause downloads, OneDrive etc.

Use the latest version of the TurboVNC client:

https://sourceforge.net/projects/turbovnc/files/2.2.1/

If on a Mac and having connection/Java errors, use the latest pre-release 2.2.2

https://s3.amazonaws.com/turbovnc-pr/master/osx/index.html

General 3D Visualization Software

13

Vaa3D

Domain Specific Software

14

PyMOL

IMOD

Windows Sessions

15

Windows sessions on the cluster will return in June

Windows 10

36 physical CPU cores256/384GB RAMP4/P40/V100 GPU

Run software not available on Linux

Let us know your needs!

Tackling Really Big Problems

16

How to interactively manage visually overwhelming amounts of data

Option A: Resampling on original dataset

Option B: Parallel visualizatione.g, paraview for Interactive Remote Parallel Visualization

- Allocate multiple nodes for visualization, each node/process will process one part of the image

Original Data Subsample Rate: 2 pixels Subsample Rate: 4 pixels Subsample Rate: 8 pixels

Client / Server Paraview

17

Client-Server mode of Paraview (remote visualization)

ClientData

ServerRenderServer

Minimize Communication

Render Remotely

Send Images

The ParaView Guide, p. 191

We can provide template batch scriptsPlease contact biohpc-help for assistance

Visualization Training at TACC

18

If you want to go in depth into Visualization…

Feedback / Help Session

19

What data are you visualizing / needing to visualize?

Is there visualization software not available on BioHPC that would help you?

Do you have issues with existing visualization services?