Surgical Planning LaboratoryBrigham and Women’s HospitalBoston, Massachusetts USA

a teaching affiliate ofHarvard Medical School

3D Slicer And The NA-MIC Kit

Ron Kikinis, M.D.

Robert Greenes Distinguished Director of Biomedical Informatics Professor of Radiology, Harvard Medical School

Founding Director, Surgical Planning Laboratory, Brigham and Women’s HospitalPrincipal Investigator, National Alliance for Medical Image Computing (a National Center for Biomedical Computing), and Neuroimage Analysis Center (a NCRR National Resource Center)Research Director, National Center for Image Guided Therapy

©2011 Surgical Planning Laboratory, ARR Slide 2

Acknowledgments

• F. Jolesz, C. Tempany, P. Black, A. Golby, S. Wells, N. Hata, CF. Westin, M. Halle, S. Pieper, F. Talos, W. Schroeder, and many more….

V E R I TAS

©2011 Surgical Planning Laboratory, ARR Slide 3

Our Science

• Algorithm research

• Tool development

• Biomedical Research

Provided by Odonnell, et al.Provided by Odonnell, et al.

Provided by Kindlmann, et al.Provided by Kindlmann, et al.

Golby, Archip et al.

©2011 Surgical Planning Laboratory, ARR Slide 4

Medical Image Computing

• Focus is on group analysis in the brain– Where is a function typically located– What is the variability?

Fischl, Greve, et al - MGH

©2011 Surgical Planning Laboratory, ARR Slide 5

Personalized Medicine

• Subject specific analysis

– Research in a clinical setting

– Interactive Analysis

Golby, Pieper, Lemaire, BWH Neurosurgery

©2011 Surgical Planning Laboratory, ARR Slide 6

A Different Style Of Research

• MIC for personalized medicine– Software runs on your computer

(HIPAA)– Fast processing (clinical research)– Subject-specific analysis (pathology)

Picture courtesy Raul San Jose

©2011 Surgical Planning Laboratory, ARR Slide 7

Different Communities

• Neuroimaging – Willing to spend

time: calibration of algorithm, learn to use

– Automation is very important

– Large resources are available

• Subject specific analysis– Minutes to hours

per case– Automation is

mostly not available

– Limited computational resources

©2011 Surgical Planning Laboratory, ARR Slide 8

Example: Image Guided Therapy (IGT)

• Use diagnostic imaging to replace/augment direct inspection

• Requires subject specific analysis of imaging data

• A very different style of research– Patient specific analysis– Limited resources are available– False positives and false negatives cannot be

compensated by adding subjects (you have only one)

©2011 Surgical Planning Laboratory, ARR Slide 9

Augmented Reality

©2011 Surgical Planning Laboratory, ARR Slide 10

The NA-MIC Approach

• Open community process

• Modular and extensible software architecture

• Free open source software (BSD)

• Works on your computer

Picture courtesy Gabor Fichtinger

©2011 Surgical Planning Laboratory, ARR Slide 11

This is Where We are Today

Picture courtesy Kapur, Jakab, Kikinis

©2011 Surgical Planning Laboratory, ARR Slide 12

NA-MIC Kit Overview

©2011 Surgical Planning Laboratory, ARR Slide 13

Works on Your Computer

• Easy download and installation

• Runs natively on your computer on most common operating systems behind your institutional firewall

• Training concept:

• Self guided tutorials• Training events• Registration and segmentation support

Picture courtesy Dominik Meier

©2011 Surgical Planning Laboratory, ARR Slide 14

The 3D Slicer (slicer.org)

• 3D Slicer is our primary platform for delivering image computing technology for personalized medicine research– Basic and clinical

visualization– Longitudinal imaging– Registration– Segmentation

Picture courtesy Marianna Jakab

©2011 Surgical Planning Laboratory, ARR Slide 15

Slicer 3.6 Highlights

Picture courtesy S. Pieper

©2011 Surgical Planning Laboratory, ARR Slide 16

Slicer 3.6 Highlights (cont)

Picture courtesy S. Pieper

©2011 Surgical Planning Laboratory, ARR Slide 17

Slicer 3.6 Highlights (cont)

Picture courtesy S. Pieper

©2011 Surgical Planning Laboratory, ARR Slide 18

Slicer 3.6 Highlights (cont)

Picture courtesy S. Pieper

©2011 Surgical Planning Laboratory, ARR Slide 19

NA-MIC Image Informatics Today

• Limited: The tradition in MIC is to chop of the header!

• Non-standard data as output from research scans

• Limited Dicom support: ITK/GDCM– Communications layer missing– Many Dicom extensions missing

• Protoype interfaces to informatics frameworks: – FetchMI in Slicer

• XNAT

• Midas

©2011 Surgical Planning Laboratory, ARR Slide 20

XNAT is…

• The eXtensible Neuroimaging Archive Toolkit.• Extensible, free open source software• An imaging informatics platform for managing,

processing, and sharing imaging and related data.• Developed by the Neuroinformatics Research

Group at Washington University in St. Louis.

Picture courtesy D. Marcus

©2011 Surgical Planning Laboratory, ARR Slide 21

Modular Open Source Architecture

Picture courtesy D. Marcus

©2011 Surgical Planning Laboratory, ARR Slide 22

QUARANTINE LOCAL USE COLLABORATION PUBLIC ACCESS

IMAGING

NON-IMAGING

OTHER SOURCES

The XNAT Enterprise workflow

• Data archiving• Quality control• Data access• Security

• Visualization• Automation• Integration• Data sharing

Picture courtesy D. Marcus

©2011 Surgical Planning Laboratory, ARR Slide 23

XNAT Web services

• RESTful web services API• Access to metadata, images, • Upload, download• Comprehensive search• Multiple formats: HTML, XML, CSV, JSON• Example clients: XNAT Gateway, PyXNAT, 3D

Slicer, Grid Wizard Engine, LONI Pipeline

Picture courtesy D. Marcus

©2011 Surgical Planning Laboratory, ARR Slide 24

client applications XNAT instance

XNAT web services

Slicer as a XNAT Client

Picture courtesy D. Marcus

©2011 Surgical Planning Laboratory, ARR Slide 25

XNAT Data Sharing Challenges

• Standards are always behind• People don’t use the standards• Lots of moving targets • Many competing entities (BIRN/ caBIG, DICOM/NIFTI,

Java/Python, SOAP/REST/DICOM)• Integration with clinical systems is a driving requirement• It’s not all about the brain

Picture courtesy D. Marcus

©2011 Surgical Planning Laboratory, ARR Slide 26

NA-MIC Kit 2012

• ITK v4:– Dicom communication layer, DCMTK– Partial GPU acceleration

• Slicer 4: Focus on usability– QT, CTK widgets– Support for layouts, hanging

protocols are coming– Improved Dicom support– Sceneviews and annotations– Image informatics support

• XNAT, other

– Extended support for plug-ins

©2011 Surgical Planning Laboratory, ARR Slide 27

URL’s

National Alliance for Medical ImageComputingwww.na-mic.org

Neuroimage Analysis Center Nac.spl.harvard.edu

Surgical Planning Laboratory, Brigham and Women’s Hospitalspl.harvard.edu

National Center For Image Guided Therapywww.ncigt.org