Post on 14-Oct-2020
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved. Answers for life.
Real-Time Medical Imaging Using GPUs
with a Non-Real-Time Operating System
GPU Technology Conference 2016
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 2 Stefan Schneider / HC DI XP R&D IC IP
X-ray Modalities @ Siemens Healthcare
Radiography1
1Images by Ysio Max
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 3 Stefan Schneider / HC DI XP R&D IC IP
X-ray Modalities @ Siemens Healthcare
Mammography1
1Images by Mammomat Inspiration
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 4 Stefan Schneider / HC DI XP R&D IC IP
X-ray Modalities @ Siemens Healthcare
Surgery1
1Images by Cios Alpha
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 5 Stefan Schneider / HC DI XP R&D IC IP
X-ray Modalities @ Siemens Healthcare
Surgery1 (2)
1Images by Cios Alpha
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 6 Stefan Schneider / HC DI XP R&D IC IP
X-ray Modalities @ Siemens Healthcare
Surgery1 (3)
1Images by Cios Alpha
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 7 Stefan Schneider / HC DI XP R&D IC IP
Motivation for Harmonized Image Chain (harmonIC1)
Former imaging solutions were designed for
• Single modality
• Dedicated image processing hardware (FPGAs and DSPs)
• Software solutions were not suitable for real-time image processing
and not for
• Modularity and expandability
• Generality
to support these novel requirements.
Establish new software solution for medical imaging.
harmonIC 1Working title, i.e. no Siemens brand
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 8 Stefan Schneider / HC DI XP R&D IC IP
harmonIC – Profile
What is the “harmonized Image Chain”
• Software framework based on MS Windows
• Processes X-ray images from
• acquisition via
• image processing up to
• presentation
• One communal software for all platforms
• Interface provides easy and abstract access
to detector and image processing functionality
• Modular and object oriented approach
12 System Types
> 3 Modalities
17 Detectors and Cameras
> 400.000 Lines of Code
> 150 Algorithms
3 Image Systems
> 25 Contributors
Images by Cios Alpha and Ysio Max
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 9 Stefan Schneider / HC DI XP R&D IC IP
harmonIC - Overview
Core
• Contains all communal functions like
• Acquisition and post-processing workflows
• Resource management
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 10 Stefan Schneider / HC DI XP R&D IC IP
harmonIC - Overview
Core
• Contains all communal functions like
• Acquisition and post-processing workflows
• Resource management
Image Source Control
• Manages detectors and cameras in
• Frame grabbing
• Controlling
• Triggering
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 11 Stefan Schneider / HC DI XP R&D IC IP
harmonIC - Overview
Core
• Contains all communal functions like
• Acquisition and post-processing workflows
• Resource management
Image Source Control
• Manages detectors and cameras in
• Frame grabbing
• Controlling
• Triggering
Image Processing Control
• Handles all image processing pipelines
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 12 Stefan Schneider / HC DI XP R&D IC IP
harmonIC - Overview
Core
• Contains all communal functions like
• Acquisition and post-processing workflows
• Resource management
Image Source Control
• Manages detectors and cameras in
• Frame grabbing
• Controlling
• Triggering
Image Processing Control
• Handles all image processing pipelines which
• Access a CUDA based algorithm pool
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 13 Stefan Schneider / HC DI XP R&D IC IP
Modular Pipeline Concept
• Algorithm pool realized in software (CUDA)
→ High performance realization of image processing
→ Efficient debugging and bugfixing of image processing
• Platform specifics encapsulated in IP pipelines
→ Pipeline changes does not interfere with other platforms
→ Used for acquisition and postprocessing (!!!)
• Modular IP pipelines
→ Easy integration of new algorithms
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 14 Stefan Schneider / HC DI XP R&D IC IP
Modular Pipeline Concept
• Algorithm pool realized in software (CUDA)
→ High performance realization of image processing
→ Efficient debugging and bugfixing of image processing
• Platform specifics encapsulated in IP pipelines
→ Pipeline changes does not interfere with other platforms
→ Used for acquisition and postprocessing (!!!)
• Modular IP pipelines
→ Easy integration of new algorithms
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 15 Stefan Schneider / HC DI XP R&D IC IP
Modular Pipeline Concept
• Algorithm pool realized in software (CUDA)
→ High performance realization of image processing
→ Efficient debugging and bugfixing of image processing
• Platform specifics encapsulated in IP pipelines
→ Pipeline changes does not interfere with other platforms
→ Used for acquisition and postprocessing (!!!)
• Modular IP pipelines
→ Easy integration of new algorithms
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 16 Stefan Schneider / HC DI XP R&D IC IP
Modular Pipeline Concept
• Algorithm pool realized in software (CUDA)
→ High performance realization of image processing
→ Efficient debugging and bugfixing of image processing
• Platform specifics encapsulated in IP pipelines
→ Pipeline changes does not interfere with other platforms
→ Used for acquisition and postprocessing (!!!)
• Modular IP pipelines
→ Easy integration of new algorithms
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 17 Stefan Schneider / HC DI XP R&D IC IP
Modular Pipeline Concept
• Algorithm pool realized in software (CUDA)
→ High performance realization of image processing
→ Efficient debugging and bugfixing of image processing
• Platform specifics encapsulated in IP pipelines
→ Pipeline changes does not interfere with other platforms
→ Used for acquisition and postprocessing (!!!)
• Modular IP pipelines
→ Easy integration of new algorithms
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 18 Stefan Schneider / HC DI XP R&D IC IP
Modular Pipeline Concept
• Algorithm pool realized in software (CUDA)
→ High performance realization of image processing
→ Efficient debugging and bugfixing of image processing
• Platform specifics encapsulated in IP pipelines
→ Pipeline changes does not interfere with other platforms
→ Used for acquisition and postprocessing (!!!)
• Modular IP pipelines
→ Easy integration of new algorithms
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 19 Stefan Schneider / HC DI XP R&D IC IP
Real-Time Challenges I
Time Lag
Presenting the acquired X-ray (video) as fast as
possible is crucial.
Main time intervals:
• X-ray detector → framegrabber
• Framegrabber → Host memory (no GPUDirect )
• Host memory → Image Processing on GPU
• Image Processing → Display
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 20 Stefan Schneider / HC DI XP R&D IC IP
Real-Time Challenges I
Time Lag
Presenting the acquired X-ray (video) as fast as
possible is crucial.
Main time intervals:
• X-ray detector → framegrabber
• Framegrabber → Host memory (no GPUDirect )
• Host memory → Image Processing on GPU
• Image Processing → Display
Measure:
• Page-locked memory
• Render the processed image on the same GPU
without additional copy
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 21 Stefan Schneider / HC DI XP R&D IC IP
Real-Time Challenges II
Constant Framerate
Rendering the X-ray video jitter-free is essential.
Problems:
• Different clock-rates of involved components
(detector, monitor)
• Non-real-time “components”
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 22 Stefan Schneider / HC DI XP R&D IC IP
Real-Time Challenges II
Constant Framerate
Rendering the X-ray video jitter-free is essential.
Problems:
• Different clock-rates of involved components
(detector, monitor)
• Non-real-time “components”
Acquisition:
• Clock-pulse generator is the image source
Replay of X-ray video:
• Clock-pulse generator is the monitor
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 23 Stefan Schneider / HC DI XP R&D IC IP
Real-Time Challenges III
Stability
Presenting the acquired X-ray (video) as stable as
possible is fundamental.
Problems:
• OS-related unsteadiness
• Connected Image System software
• Hardware-interrupts (not solved yet )
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 24 Stefan Schneider / HC DI XP R&D IC IP
Real-Time Challenges III
Stability
Presenting the acquired X-ray (video) as stable as
possible is fundamental.
Problems:
• OS-related unsteadiness
• Connected Image System software
• Hardware-interrupts (not solved yet )
Two ring buffers:
• Acquisition-buffer for OS-related jitter
• Processed-image-buffer for Image System SW
Prevent radiation exposure without imaging:
• Two-stage escalation strategy
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 25 Stefan Schneider / HC DI XP R&D IC IP
Upcoming
• GPU sharing across components
• 2D image processing and visualization
• 3D reconstruction
• 3D volume visualization
• Parallelization of system workflows
• Acquisition and replay at once
• Critical vs. non-critical tasks
NVIDIA, help!
harmonIC
Unrestricted © Siemens Healthcare GmbH 2016 All rights reserved.
2016-04-07
Page 26 Stefan Schneider / HC DI XP R&D IC IP
Stefan Schneider
Siemens Healthcare GmbH
Diagnostic Imaging
X-Ray Products
Research & Development
HC DI XP R&D IC IP
Allee am Roethelheimpark 2
91052 Erlangen
Germany
Phone: +49 (9131) 84-3449
E-mail:
schneider.stefan@siemens.com
Questions?