Http://na-mic.org/Wiki/index.php/Projects:RegistrationDocumentation 1 Multimodal Registration Clinic...

http://na-mic.org/Wiki/index.php/Projects:RegistrationDocumentation

1Multimodal Registration Clinic

“All Things Registered”

I. Theory & Tool Overview

II. Live Demo of Registration in 3D Slicer

III. Open Discussion: what’s on your wishlist?

Dominik S. Meier, Ph.D.


2Multimodal Registration Clinic

“All Things Registered”

I. Background: Registration Theory

II. Image Registration Tools in 3DSlicer (v.3.5)

III. User-support , Training & Documentation


3

Part I : Registration Theory

Background


4Registration Concept

• Image registration seeks to bring two or more images into anatomical alignment.

• In mathematical terms: Image registration transforms multiple images into one coordinate system. This is necessary to compare or integrate/fuse the data obtained from different measurements.

• Purpose– change detection (small regional change, subtraction imaging)

– atlas building (normalize for individual anatomical idiosyncrasies)

– distortion or motion correction (different protocols or sensors)

– protocol matching (e.g. sagittal into axial)

– group analysis (anatomical reference space)

– Navigation, patient to reference, surgical planning

– atlas-based morphometry (estimates from atlases)


5Quality Assessment

speed

robustnessprecision

Cross-sectional:group analysis

•Large numbers of images processed in a fully automated fashion

•Limited review and editing

•Processing speed less relevant than reliable performance across all images in the study

speed

robustness

precision

speed

robustness

precision

IGT:

•Supervised, so robustness can be supplemented with user interaction

•Speed and precision are critical

•Precision/error estimates also critical

Longitudinal:Change Assessment

•Smaller study size compared to cross-sectional

•Some review and editing

•Precision determines detectable change, is the key criterion


6Registration ConceptImage Registration has 4 main components

– spatial transform: a model equation that describes how the two images should be aligned.

– similarity metric: a criterion that defines how well the images are aligned, i.e. what constitutes a “good match” (cost function).

– optimizer: an iterative exploring of the realm of possible solutions, looking to find the best one (search algorithm).

– interpolator: an algorithm to apply the transform and build the newly aligned images (resampling).

It is important to know what these 4 are and what they do to achieve the best possible result. We will briefly discuss each in turn.

metricoptimizer

interpolator

transform


7Transform

f(x,y,z)

•analytical model•deformation/vector field•invertable if linear and isomorphic

e.g. translation + scale :x’ = x + 10y’ = y - 3z’ = 1.05 z + 5

z

x

y

z

x

y

metric

optimizer

interpolator

transform


8

translation

rotation

scaling

shearing affine transform 12 DOF

3 x

3 x

3 x

3 x

similarity transform

9 DOF

rigid transform 6 DOF

Linear Transform: DOF

shift transform 3 DOF

+

+

+

metric

optimizer

interpolator

transform


9Non-linear Transform: DOF

rigid

affine

81375≈ 50 Million

distances and angles change proportionallyMotion guided by single equation.

many image control points move independently, i.e. no single equation

type/name DOF shape description caveat

intact

globally distorted

careful with volumetry

non-rigid locally distorted

•3-pt Bspline grid:•5-pt Bspline grid:•full image:

rigid body motion, distances and angles preserved. Motion guided by single equation.

distances and angles change dis-proportionally

will not match global scale distortions

DOF matching required: careful not to use excessive DOF and thus normalize/remove the differennces you want to measure.

12

6

metric

optimizer

interpolator

transform


10Coordinate Systems: the hidden Xform

Each image has a basic (linear) transform that connects the digital image grid to the physical world. This is usually part of the image header.

RAS : right - anterior - superior

R L

L

S

I

A P

vox2rasz

x

yImageto ImageTransform

z

x

y

vox2ras

R L

L

S

I

A P

metric

optimizer

interpolator

transform


11Registration Glossary

• DOF• fixed & moving image• linear vs. nonlinear• rigid vs. non-rigid• forward vs. backward/inverse mapping• multi-modality• registration parameters• affine, similarity, B-spline,warping• pivot point, image origin, coordinate system• pixel space vs. raster space• voxel size & anisotropy• intensity vs. feature-based registration


12Metric•defines how well the two images align•qualitative: you; visual assessment•quantitative:

•sum of intensity similarity•sum of topographical features

– Intensity Difference

– Intensity Ratio

– Cross-correlation

– Mutual Information

metrictransform optimizer

interpolator

speed

robustness

precision

– same subject, same contrast

– same subject, different contrast

– same subject, different modalities

– different subject, same contrast

– different subject, different contrast


13Similarity Metricmetric

optimizer

interpolator

transform

speed

robustnessprecision

same subject, same contrast

same subject, different contrast

same subject, different modalities

different subject, same contrast

different subject, different contrast

Difference Ratio Correlation Mutual-Info


14Interpolation

•Applies the transform to the image and generates a new volume.

•back from physical space into the image grid:

•the newly calculated position of an image voxel will not fall exactly onto a grid-point. Therefore its value is determined by the intensities of neighboring pixels. This process is known as interpolation.

T


interpolator


15Interpolation

•nearest neighbor: picks value of the voxel nearest the point coordinates

• + fast

• - coarse

• + MUST use for label-maps

•linear: picks a weighted mean of neighboring voxel intensities

• + stable, default

• - introduces blurring

•cubic, sinc: fits a non-linear model to estimate the intensity

• + sharper, less blurring

• - slower

• - may introduce spurious outliers near edges (e.g. negative intensities)

original nearest neighbor

linear cubic

Example of a T1-weighted brain MRI, rotated by 6 degrees. Showing magnified sagittal view of cerebellum and midbrain.

nearest neighbor: note the false contouring around the ponslinear: note the blurringcubic: less blurring than linear

metric

optimizer

transform

interpolator


16Optimizer

•manual: you

•automated: iterates between metric and transform

•exhaustive search

•gradient-based search

•annealing/stochastic schemes


interpolator


17Optimization

rotation x

rotation y

similarity

optimumlocal maxima (suboptimal solutions)

The algorithm moves/wiggles one of the images around trying to find the best match, according to the similarity metric. It does so in increments from its current position, evaluating if the new position is better than the old one. A “local maximum” is a position around which all nearby changes appear worse, but farther away there is a better solution available. Optimization algorithms often get “stuck” in such positions.Depending on the difference in contrast between the two images, the similarity metric employed, and the amount of initial misalignment, this is more or less likely to happen to an automated registration.

metricoptimizer

transform

interpolator


18

Part I : further topics

• the main components: transform, similarity metric, optimization, interpolation

• coordinate systems: physical vs. image space, RAS vs. LPI etc.

• relevant image meta-data: coordinate system, axis orientation, image/CS origin, voxel size

• overview of scenarios and their different challenges: image pairings, DOF, multi-modal, intra/inter-subject etc.

• how to evaluate a match: tools & concepts

• common mistakes to avoid: inappropriate DOF, overly flat similarity metric, CS inconsistencies, FOV discrepancies, wrong interpolation, insufficient search (sample points, multi-scale, DOF scale-space)

• Troubleshooting guide: insufficient match - what next? Parameter modification, DOF change, initial alignment assist, fiducial help, ROI masking (e.g. skull stripping)


19

Part II : 3DSlicer Registration

Tools


20Overview of Registration Toolsin 3D Slicer

Registration Main Modules

Registration Auxilary Modules

Driver:

•manual

•intensity

•surfaces

•fiducials

•segmentation

DOF:

6

7

9

12

27 ~ 103

Support for:•fiducials•ROI definition•mask building & editing•resampling•visualization/evaluation


21Registration Modules:

Manual - Interactive

• ideal for initial alignment

• immediate feedback in 3D

• fail-safe if automated registration fails or is too slow


22

speed & precision

mask

starting point

robustness

contrast & contentDOF

presets

Registration Modules:

Intensity Affine


23Registration Modules NEW:

Multi-resolution Affine

•Method of choice for robustness•Supports masking


24

presets

DOFimage contrast

constraints

speed & precision


Non-rigid BSpline


25

1. build 2 fiducial lists with 3 or more points each2. click: Apply3. supports translation to similarity (3-9 DOF)

very fast (< 1 sec)

Example: inter-subject knee registration


Fiducial Registration


26

• Input: 2 surface models• click: Apply• supports rigid to affine (6-12 DOF)• very fast (~ 1 sec)


Surface Registration


27

• Aligns brain image along midsagittal plane and places anterior-posterior commissures on a horizontal line

• Input: 2 fiducial pairs defining

• anterior & posterior commissure

• midsagittal plane


AC-PC alignment


28

• Non-rigid Cortical surface

alignment based on

WM/GM segmentation

• warps based on attributes

derived at gyrus crown,

sulcal root and ventricle

corners.


HAMMER Cortical Surface Matching


29

• Non-rigid registration based on optical flow principle

considered very robust


Demons - Warping


30

• define and edit in 3D• use for masking registration

Auxilary Tools:

ROI Module


31

• define and edit in 2D or 3D• use for masking registration

where masking is not (yet) explicitly supported

• increase speed and robustness

Auxilary Tools:

Extract Subvolume


32Auxilary Tools:

Visualization

Checkerboard Filter• to evaluate registration quality (particularly

for areas with high contrast/edges)• Subtraction Images to evaluate overall

alignment

Subtraction• Subtraction Images to evaluate regional

changes and alignment


33

Editor• create and manipulate binary

label maps from grayscale images

• fix labelmaps returned by other modules (skull stripping, Otsu’s etc.)

Auxilary Tools:

Editor


34Auxilary Tools:

Resampling

• Apply a transform to a scalar or vector (e.g.

DTI) volume

• select tailored interpolation scheme (nearest

neighbor, linear, sinc, b-spline)

• correctly reorients vector data


35

• define and edit in 3D• organize in fiducial lists• use ordered lists of

fiducial pairs for registration

Auxilary Tools: Fiducials


36

Part III : Registration

User Support Training

Documentation


37Registration Case LibraryBrain

Other

A growing collection of example registration problems, complete with image data, tutorial, solution, discussion and a parameter preset file that can be loaded into 3DSlicer.


38Registration Case Library


39Registration Case Library:Tutorials

Guided/narrated Video Tutorials

Step-by-step Powerpoint/PDF Tutorials


40Call For Datasets"if you have a registration problem that is not yet covered in our library, send us your case: we will post it along with our best registration solution/strategy. If you agree to the posting of the anonymized image data, you get a free registration, the user community gets a new example case. Everybody wins.”

What We Will Do•seek the best possible registration obtainable with the most recent version of 3DSlicer

•post the anonymized image as a new case in our Slicer Registration Case Library

•post the exact workflow used to obtain the shown solution registration will be posted alongside the data as a guided step-by-step tutorial

•the parameters for successful registration will also be posted as a loadable custom "Registration Preset" file that you can load directly into Slicer and apply on your data

•if you can provide us with fiducial pairs or other criteria that define a good registration, we will use them in optimization efforts.

•the registration objective & background, main challenges and strategy recommendations will be posted

•an acknowledgment of your lab as the data source is posted, if desired with a link to your institution and/or related research papers


41Registration Case Library:Tutorials

Slicer Training Compendium:Tutorials for all skill levels


42Acknowledgments

National Alliance for Medical ImageComputingNIH U54EB005149

Neuroimage Analysis Center NIH P41RR013218

Surgical Planning Laboratory, Brigham and Women’s Hospital

Http://na-mic.org/Wiki/index.php/Projects:RegistrationDocumentation 1 Multimodal Registration Clinic...

Documents

Transcript of Http://na-mic.org/Wiki/index.php/Projects:RegistrationDocumentation 1 Multimodal Registration Clinic...