Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for...
Transcript of Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for...
![Page 1: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/1.jpg)
Recent Developments in
Ultrasound Visualization
Stefan Bruckner
Department of InformaticsUniversity of Bergen
![Page 2: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/2.jpg)
Basic Ultrasound Imaging
2
![Page 3: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/3.jpg)
Ultrasound Characteristics
• Non-invasive
• Cheap
• High resolution
– Spatially
– Temporally
• Noise
– Random
– Speckle
3
![Page 4: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/4.jpg)
Common Ultrasound Modes
• 2D Ultrasound– B-Mode
• 3D Ultrasound– Static 3D imaging
• 4D Ultrasound– Dynamic 3D imaging
• Doppler Ultrasound– Color Doppler: directional– Power Doppler: non-directional
• Contrast Ultrasound– Microbubbles-based contrast agents
4
![Page 5: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/5.jpg)
Outline
• Visualization of 3D/4D ultrasound data
• Recent advances in
– Filtering
– Classification
– Illumination
– Fusion and Guidance
5
![Page 6: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/6.jpg)
FILTERING
Recent Developments in Ultrasound Visualization
![Page 7: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/7.jpg)
Filtering
• Noisy character of ultrasound imaging makes filtering particularly important for 3D visualization
7
![Page 8: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/8.jpg)
Lowest Variance Filtering
• Remove speckle and random noise
• Structure-preserving filtering
– Determine local structure orientation
– Filter along direction of lowest variance
Solteszova el al. 2012: Lowest-Variance Streamlines for Filtering of 3D Ultrasound 8
![Page 9: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/9.jpg)
Local Structure Orientation
Solteszova el al. 2012: Lowest-Variance Streamlines for Filtering of 3D Ultrasound
• Sample local voxel neighborhood on on a sphere
9
![Page 10: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/10.jpg)
Directional Filtering
• Streamline integrationalong direction oflowest variance
FORWARD
BACKWARD
Solteszova el al. 2012: Lowest-Variance Streamlines for Filtering of 3D Ultrasound 10
![Page 11: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/11.jpg)
Results
Solteszova el al. 2012: Lowest-Variance Streamlines for Filtering of 3D Ultrasound 11
![Page 12: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/12.jpg)
4D Filtering (1)
• Acceptable complexity of filtering method is limited by the target frame rate
– Idea: only filter voxels that contribute to the final rendered image
– Problem: filtering changes data values and hence can affect visibility globally
– Solution: conservatively estimate a voxel’s visibility after filtering
12Solteszova el al. 2014: Visibility-Driven Processing of Streaming Volume Data
![Page 13: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/13.jpg)
4D Filtering (2)
• Only a fraction of voxels actually influence the final image due to transparency and occlusion
13Solteszova el al. 2014: Visibility-Driven Processing of Streaming Volume Data
![Page 14: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/14.jpg)
Visibility-Driven Filtering
14Solteszova el al. 2014: Visibility-Driven Processing of Streaming Volume Data
![Page 15: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/15.jpg)
Prediction of Filter Behavior
• Opacity of a filtered value of minimum and maximum of a neighborhood
• Possible for all convolution-based filters with normalized non-negative weights
• Lookup tables for conservative visibility mask calculation
Solteszova el al. 2014: Visibility-Driven Processing of Streaming Volume Data 15
![Page 16: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/16.jpg)
Results (1)
Solteszova el al. 2014: Visibility-Driven Processing of Streaming Volume Data
unfilteredregularfiltering
5 fps
visibilityoptimized
10 fps
=
16
![Page 17: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/17.jpg)
Results (2)
17Solteszova el al. 2014: Visibility-Driven Processing of Streaming Volume Data
![Page 18: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/18.jpg)
CLASSIFICATION
Recent Developments in Ultrasound Visualization
![Page 19: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/19.jpg)
Classification
• Mapping of data values to optical properties (usually color and opacity)
• Several challenges
– Low dynamic range
– Significant amount of noise and speckle
– Varying intensities for the same tissue
– Fuzzy boundaries
19
![Page 20: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/20.jpg)
Variational Classification
• Simultaneous denoising and opacity assignment
• Variational approach based on isovalue and gradient
Fattal and Lischinski 2001: Variational Classification for Visualization of 3D Ultrasound Data 20
![Page 21: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/21.jpg)
Scale Space Filtering
• Automatic adjustment of the global opacity transfer function based on scale-space filtering
21
Hönigmann et al. 2003: Adaptive Design of a Global Opacity Transfer Function for Direct Volume Rendering of Ultrasound Data
![Page 22: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/22.jpg)
Predicate-based Classification
• Problem: classification of 3D ultrasound data for volume visualization– Standard 1D transfer functions
don’t work well for ultrasound
– Additional attribute dimensions can help, but classification space becomes difficult to navigate
• Approach: define a set of point predicates which can be combined via logical operations
22Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 23: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/23.jpg)
Predicate Library
• Set of different local and non-local predicates 𝑃 = (𝑓𝑃: 𝑋 → 𝑡𝑟𝑢𝑒, 𝑓𝑎𝑙𝑠𝑒 , 𝜅𝑃, 𝛿𝑃)– 𝜅𝑃 is an importance factor
– 𝛿𝑃 is the color modulation
• Examples of possible predicates– Range-based predicates
– Direction-based predicates
– Signal-to-Noise ratio predicate
– Vesselness predicate
– Confidence predicate
– Label predicate
23Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 24: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/24.jpg)
Predicate Setup
• Simple widget to assign importances and colors
• Combination of predicates with Boolean operations (and, or, not)
24Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 25: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/25.jpg)
Visual Mapping
• Importance-driven layered compositing, cf. [Viola et al. 2004, Rautek et al. 2007]
• High-importance layers receive higher visibility (depth relationships can be overridden)
• Predicates only affect hue and opacity, luminance comes from data values
25Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 26: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/26.jpg)
Predicate Histogram
• Sketch-based interface for predicate setup
• User draws positiveand negative sketch
• Importance of each predicate is modulated accordingly
26Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 27: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/27.jpg)
Results (1)
• Shoulder dataset: combines visualization of bone and muscle tissue
27Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 28: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/28.jpg)
Results (2)
• Path of the carotid artery is shown in red
28Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 29: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/29.jpg)
Results (3)
• Achilles tendon is shown in red
29Schulte zu Berge et al. 2014: Predicate-based Focus-and-Context Visualization for 3D Ultrasound
![Page 30: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/30.jpg)
RENDERING
Recent Developments in Ultrasound Visualization
![Page 31: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/31.jpg)
Volume Rendering (1)
31
image plane
volume
eye
light source
![Page 32: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/32.jpg)
Volume Rendering (2)
32
in-scattering
absorption out-scattering
emission usuallyignored
![Page 33: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/33.jpg)
Local Volume Illumination
• Only a function of gradient direction and light source parameters
– Volumetric absorption between light source and sample point is ignored no shadows
– Multiple scattering is ignored no color bleeding effects
33
conventionalrendering
fetoscopicimage
![Page 34: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/34.jpg)
Light Propagation in Tissue
• Human skin (and tissue in general) is translucent
– Red penetrates deeper than blue and green light
– Light scatters predominantly in forward direction
– Light propagation tends to become isotropic after multiple scattering events
34
![Page 35: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/35.jpg)
Fetoscopic Illumination Model
35
volume data
indirect light
direct light
scattering
shadows
ambient
specular
tone mapping
final image
Varchola 2012: Live Fetoscopic Visualization of 4D Ultrasound Data
![Page 36: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/36.jpg)
Fetoscopic Illumination Model
36
volume data
indirect light
direct light
scattering
shadows
ambient
specular
tone mapping
final image
Varchola 2012: Live Fetoscopic Visualization of 4D Ultrasound Data
![Page 37: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/37.jpg)
Direct Lighting (1)
Light is attenuated along its way through the volume
37
![Page 38: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/38.jpg)
Direct Lighting (2)
38Kniss et al. 2003: A Model for Volume Lighting and Modeling
![Page 39: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/39.jpg)
Light Source Extent (1)
39hard shadows soft shadows
![Page 40: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/40.jpg)
Light Source Extent (2)
40
![Page 41: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/41.jpg)
Soft Shadows
41Patel et al. 2013: Instant Convolution Shadows for Volumetric Detail Mapping
![Page 42: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/42.jpg)
Kernel Size (1)
42
shadow softness - low shadow softness - medium shadow softness - high
![Page 43: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/43.jpg)
Kernel Size (2)
43
shadow softness - low shadow softness - medium shadow softness - high
![Page 44: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/44.jpg)
Fetoscopic Illumination Model
44
volume data
indirect light
direct light
scattering
shadows
ambient
specular
tone mapping
final image
Varchola 2012: Live Fetoscopic Visualization of 4D Ultrasound Data
![Page 45: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/45.jpg)
Indirect Lighting (1)
Light is scattered multiple times before it reaches the eye
45
![Page 46: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/46.jpg)
Indirect Lighting (2)
46Kniss et al. 2003: A Model for Volume Lighting and Modeling
![Page 47: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/47.jpg)
Chromatic Light Attenuation
47
color intensity (RGB)
position along diffusion profile
light orientation
R
G
B
![Page 48: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/48.jpg)
Forward Scattering (1)
48
rendering without scattering rendering with scattering
![Page 49: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/49.jpg)
Forward Scattering (2)
49
![Page 50: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/50.jpg)
Fetoscopic Illumination Model
50
volume data
indirect light
direct light
scattering
shadows
ambient
specular
tone mapping
final image
Varchola 2012: Live Fetoscopic Visualization of 4D Ultrasound Data
![Page 51: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/51.jpg)
Front and Back Lighting
51
Light positioned in front Light positioned behind the scene
![Page 52: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/52.jpg)
Local Ambient Occlusion (1)
• Evaluate the average visibility of each point
– Perform sampling in a small spherical neighborhood
– Modulate ambient illuminationintensity by the result
52
![Page 53: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/53.jpg)
Local Ambient Occlusion (2)
53
with ambient termwithout ambient term
![Page 54: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/54.jpg)
Fetoscopic Illumination Model
54
volume data
indirect light
direct light
scattering
shadows
ambient
specular
tone mapping
final image
Varchola 2012: Live Fetoscopic Visualization of 4D Ultrasound Data
![Page 55: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/55.jpg)
Specular Highlights
55
![Page 56: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/56.jpg)
Fetoscopic Illumination Model
56
volume data
indirect light
direct light
scattering
shadows
ambient
specular
tone mapping
final image
Varchola 2012: Live Fetoscopic Visualization of 4D Ultrasound Data
![Page 57: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/57.jpg)
Implementation
• GPU-based implementation using DirectX
– Available as HDlive in GE’s latest generation of ultrasound machines (Voluson E8 / Expert)
– Interactive performance of 15-20 fps limited by data acquisition
57
![Page 58: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/58.jpg)
Results (1)
58
conventional rendering fetoscopic rendering
![Page 59: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/59.jpg)
Results (2)
59
conventional rendering fetoscopic rendering
![Page 60: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/60.jpg)
Results (3)
60
conventional rendering fetoscopic rendering
![Page 61: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/61.jpg)
Results (4)
61
fetoscopic renderingconventional rendering
![Page 62: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/62.jpg)
Results (5)
62
photograph acquired with fetoscope[A Child is Born, Nilson and Hamberger]
fetoscopic rendering[Picture of the Month, Ultrasound in
Obstetrics & Gynecology 38(5)]
![Page 63: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/63.jpg)
Benefits
• Approximates realistic illumination in real-time
• Robust against noise and artifacts
• Better 3D perception may have diagnostic benefits
• Currently investigating other application scenarios (e.g., cardiac)
63
cleft lip: better visibility of border and separation
down syndrome: inclanation of palpepralfissures
![Page 64: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/64.jpg)
Cardiac Ultrasound
64
![Page 65: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/65.jpg)
Chromatic Shadows
• Comparison between black and illustration-inspired blue shadows
Solteszova el al. 2014: Chromatic Shadows for Improved Perception 65
![Page 66: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/66.jpg)
FUSION AND GUIDANCE
Recent Developments in Ultrasound Visualization
![Page 67: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/67.jpg)
Fusion and Guidance
• Fusion: combine multiple modalities to improve diagnostic value
– Registered CT/MRI scans, blood flow, etc.
• Guidance: augment images with additional information
– Orientation and navigation aids, etc.
67
![Page 68: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/68.jpg)
B-Mode/Doppler Fusion
• Integrated visualization of B-Mode and Doppler data
• Non-photorealistic silhouette rendering for reduced visual clutter
68Petersch et al. 2007: Blood flow in its context: Combining 3D B-Mode and Color doppler Ultrasonic Data
![Page 69: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/69.jpg)
Vector Flow Imaging Visualization
• Vector Flow Imaging provides 3D velocity information
– Pathlets-based visualization
– Pathline integration on the GPU
Angelelli et al. 2014: Live ultrasound-based particle visualization of blood flow in the heart 69
![Page 70: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/70.jpg)
Guidance in Liver Examinations
Jennifer N. Gentry
Viola et al. 2008: Illustrated Ultrasound for Multimodal Data Interpretation of Liver Examinations
• Couinaud segmentation: divides the liver into different sections dependent on the blood vessels
• Registration to a liver modelfor real-time Couinaudoverlays during the scan
70
![Page 71: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/71.jpg)
Cardiac Ultrasound Guidance
• Real-time augmentation of the ultrasound slice using an animated heart model
71Ford et al. 2012: HeartPad: Real-Time Visual Guidance for Cardiac Ultrasound
![Page 72: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/72.jpg)
CONCLUSIONS
Recent Developments in Ultrasound Visualization
![Page 73: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/73.jpg)
Conclusions
• Selection of recent approaches for improved visualization of ultrasound data
• Importance of 4D ultrasound as a cheap and effective imaging modality is ever-increasing
• Technological advances (e.g. beamforming) offer continuous improvements in frame rate and image resolution
• Live 4D data is still very challenging and many problems remain unsolved
73
![Page 74: Recent Developments in Ultrasound Visualizationisg · •Selection of recent approaches for improved visualization of ultrasound data •Importance of 4D ultrasound as a cheap and](https://reader033.fdocuments.us/reader033/viewer/2022043019/5f3b9f93bdc8927a0b287914/html5/thumbnails/74.jpg)
Thank you for your attention!
Acknowledgements
Veronika Solteszova, Åsmund Birkeland, Paolo Angelelli, Ivan Viola, Alexey Karimov, Andrej Varchola, M. Eduard Gröller,
Erik Steen, Gerald Schröcker, Daniel Buckton