Dagstuhl-Seminar, April 18-23, 2004

Some Developments in DT-MRI Registration and Visualization

James Gee, Hui Zhang, Jeffrey Duda, Paul Yushkevich, Brian Avants, Abraham DubbUniversity of Pennsylvania

Overview

Registration Tensor orientation pattern matching

– Non-rigid registration

Diffusion profile matching– Affine transformations

Visualization Anatomically labeled fiber tracts

– Partitioning the corpus callosum

Orientation Pattern Matching

Duda et al, SPIE Med Imag 2003

Definitions

• In this case the tensor may be represented by a symmetric 2x2 matrix, I

• For this formulation it is useful to represent the tensor by a column vector of the eigenvalues and an orientation angle

21

10

ii

iiI

2

1ˆ

I

11

22

x

y

Non-Rigid Registration of 2D Tensor Data

• Want to find a smooth continuous mapping, u(s), from I2 to I1

,12 sIsusI ,

y

xs

su

susu

y

x

• This is accomplished by minimizing a function consisting of three separate terms

Objective Function

• Eigenvalue (ellipsoid shape) difference

,ˆˆ 2

211

r Ns r

rusIsIuU

.ˆˆˆ 2

2211

r Ns

T

r

rusIsIsIuU

• Smoothness constraint

.22

r Ns r

suruuU

Objective Function

• Local neighborhood orientation-pattern difference

r Ns

T

r

rusrBsrAuU ,,cos3

srsrsrA IIII 2211,

rssrB II 22,

Local Neighborhood Orientation-Pattern Difference

srsrsrA IIII 2211,

r1

s1r1 s1

r2

s2

r2 s2

Local Neighborhood Orientation-Pattern Difference

0 0 45

0 0 45

0 0 450 0 45

0 0 45

0 0 45

0 0 0

0 0 0

0 0 0

Gradient of Objective Function

• The following function is minimized:

uUuUuUuU 332211

uUdu

udU 0

r Nr

TT

r

rusFsFsGsFuU ˆˆ11

r Nr r

suruuU 22

Gradient of Objective Function

• U’3(u) give a non-linear term and is approximated with the following term:

r Nr

T

PT

PT

T

r

rusrBsrArusrBsrA

rusrBsrAsrBuU ,,

,,

,,sin,33

2,0,, rusrBsrA T

• Where up is the value of u at the previous iteration and:

Preliminary Registration Example

A

B

B A

A B

Diffusion Profile Matching

Zhang, Yushkevich, Gee, ISBI 2004

A diffusion profile is a function that describes the rate of diffusion in any given direction at a point in space

Diffusion Profiles Diffusion Tensors

A diffusion tensor is a Cartesian tensor (s.p.d. matrix) that represents a Gaussian diffusion profile

Why Profiles?

Inclusion of non-Gaussian profilesAlexander AL et al. MRM 2001, Frank LR. MRM 2001, Tuch DS et al. MRM 2002

Well-defined physical interpretationAlexander DC. UCL Research Notes 2000

Generality

the distance metric is a function that measures their similarity

Diffusion Profile Metric

Given two diffusion profiles and :

Since diffusion profiles form a subset of Hilbert space, the functional distance metric is used:

with

Computing the Metric

Express diffusion profiles in terms of truncated spherical harmonic series:

Alexander DC et al. MRM 2002

inner product computed as:

where and are spherical harmonic coefficientsof and

Specializing to Diffusion Tensors

Coefficients from spherical harmonic expansion

Inner product and distance metric

where and are the Cartesian tensor inner product and distance metric respectively

Objective function

Template Subject

Affine Registration

Specialized to diffusion tensors

where

Finite Strain-based Reorientation

Polar Decomposition Theorem

is symmetric and positive-definite (pure deformation)

is an orthogonal matrix (pure rotation)

Finite Strain reorientationIgnore the reorientation effect of

Best orthogonal approximation to

Alexander DC et al. TMI 2001

Affine Transform Parametrization

Affine transformationwith

Parametrization of

Compared to the standard parametrization, this scheme allows us to explicitly express the finite strain-based reorientation operator in terms of and thus differentiate analytically Affine Parameters

Polar Decomposition-based:

with and

Preliminary Affine Registration Results

Experimental setup

A collection of 288 synthetic transformations

Relative error between the original and the recovered transformations is computed as the sum of the relative errors in each parameter

Data: 2D slices of human brain DT

Each transformation generates a synthetic subject imageThe original image is registered to the synthetic subject to recover the transformation

Relative error statisticsregistration with analytical derivatives using conjugate gradient optimizer

average relative error = 3%registration without derivatives using direction set optimizer

average relative error = 207%

with derivatives without derivatives

Relative Error Distribution

Piecewise Affine, Non-rigid Extension

1

2 3

1. The original B-Spline based displacement field

2. The piecewise affine displacement field recovered with the Cartesian distance metric3. The piecewise affine displacement field recovered with the diffusion profile metric

a. The trace and the anisotropy image of the original image

b. The trace and the anisotropy image of the image deformed by the displacement field (1)

Anatomically Labeled Fiber Tracts

Cortex-based Partitioning of the Corpus Callosum

Callosal MorphometryCallosal MorphometryWitelson PartitionWitelson Partition

Anterior (rostrum, genu) ↔ motor cortexAnterior (rostrum, genu) ↔ motor cortex Anterior half ↔ somatosensory cortexAnterior half ↔ somatosensory cortex Posterior two-thirds, dorsal splenium ↔ auditory regions, Posterior two-thirds, dorsal splenium ↔ auditory regions,

limbic cortexlimbic cortex Isthmus ↔ posterior parietal and superior temporal cortex, Isthmus ↔ posterior parietal and superior temporal cortex,

cortical regions related to functional asymmetrycortical regions related to functional asymmetry Splenium, body and anterior portion ↔ visual cortexSplenium, body and anterior portion ↔ visual cortex

““Modern” Callosal Modern” Callosal MorphometryMorphometry

Template DeformationsTemplate Deformations Correspondences Correspondences between callosa are between callosa are obtained by registering obtained by registering the the curve geometrycurve geometry at at different scalesdifferent scales

Boundary correspondence Boundary correspondence is interpolated in a is interpolated in a geometrically correctgeometrically correct way way throughout the interior throughout the interior (or even exterior) of the (or even exterior) of the curvecurve

Related workRelated work Pettey, D.J., Gee, J.C. Using a linear diagnostic function and Pettey, D.J., Gee, J.C. Using a linear diagnostic function and

non-rigid registration to search for morphological differences non-rigid registration to search for morphological differences between populations: An example involving the male and between populations: An example involving the male and female corpus callosum. female corpus callosum. Information Processing in Medical Information Processing in Medical ImagingImaging, Insana, M., Leahy, R., eds., Heidelberg:Springer-, Insana, M., Leahy, R., eds., Heidelberg:Springer-Verlag, LNCS 2082Verlag, LNCS 2082,, pp. 372-379, 2001. pp. 372-379, 2001.

Dubb, A., Avants, B., Gur, R., Gee, J.C. Shape characterization Dubb, A., Avants, B., Gur, R., Gee, J.C. Shape characterization of the corpus callosum in Schizophrenia using template of the corpus callosum in Schizophrenia using template deformation. deformation. Medical Image Computing and Computer-Medical Image Computing and Computer-Assisted InterventionAssisted Intervention, Kikinis, R., ed., Heidelberg:Springer-, Kikinis, R., ed., Heidelberg:Springer-Verlag, LNCS 2489, pp. 381-388, 2002.Verlag, LNCS 2489, pp. 381-388, 2002.

Pettey, D.J., Gee, J.C. Sexual dimorphism in the corpus Pettey, D.J., Gee, J.C. Sexual dimorphism in the corpus callosum: A characterization of local size variations and a callosum: A characterization of local size variations and a classification driven approach to morphometry. classification driven approach to morphometry. NeuroImageNeuroImage, , 17 (3), pp. 1504-1511, 2002.17 (3), pp. 1504-1511, 2002.

Dubb, A., Gur, R., Avants, B., Gee, J.C. Characterization of Dubb, A., Gur, R., Avants, B., Gee, J.C. Characterization of sexual dimorphism in the human corpus callosum using sexual dimorphism in the human corpus callosum using template deformation. template deformation. NeuroImageNeuroImage, 20(1), pp. 512-519, 2003., 20(1), pp. 512-519, 2003.

DTI-based Partitioning DTI-based Partitioning of Corpus Callosumof Corpus Callosum

Validate Witelson partitionValidate Witelson partition Enable more anatomically rigorous and Enable more anatomically rigorous and

detailed segmentation of corpus callosum detailed segmentation of corpus callosum and associated morphological studiesand associated morphological studies Behrens et al, Nature Neuroscience, 2003Behrens et al, Nature Neuroscience, 2003

Cortex Labeling: Coarse Cortex Labeling: Coarse StepStep

Combined GM and Combined GM and WM volume is divided WM volume is divided into three regions into three regions using graph using graph partitioning software partitioning software METISMETIS

Regions:Regions: Left cortexLeft cortex Right cortexRight cortex Cerebellum & brain Cerebellum & brain

stemstemCoarse level GM+WM partition

Cortex Labeling: Fine Cortex Labeling: Fine StepStep

Expert draws curves on Expert draws curves on the surface of a cortical the surface of a cortical region using Livewire-region using Livewire-style user interfacestyle user interface

These curves will be These curves will be used to partition the used to partition the surface into patchessurface into patches

Patch segmentation Patch segmentation will be propagated onto will be propagated onto the GM volume using the GM volume using level set/flow level set/flow techniques techniques

BrainTracer tool used to mark the cortical surface

Brain AtlasBrain Atlas Cortical structures:Cortical structures:

Segmented using surface-painting procedureSegmented using surface-painting procedure

Sub-cortical structuresSub-cortical structures Automatic, level set based segmentation for caudate and Automatic, level set based segmentation for caudate and

ventriclesventricles Manual SNAP segmentationManual SNAP segmentation

2D and 3D views of Left Cortical StructuresLevel set segmentation of the ventricles

Atlas-based Atlas-based SegmentationSegmentation

Variational Variational Diffeomorphic MatchingDiffeomorphic Matching

Chimp to Human

Atlas Image Subject

Anatomic Labels

Individualized Atlas

Atlas-based Localization

Spatial Transformation

REGISTER

OVERLAYWARP

Gee et al, JCAT, 1993; Avants and Gee, MedIA, NeuroImage, in press

Deterministic Deterministic Tractography Tractography

in the Presence of Noisein the Presence of Noise DT-MRI signal noisy DT-MRI signal noisy

near gray-white near gray-white interfaceinterface Early termination of Early termination of

reconstructed reconstructed pathwayspathwaysBehrens et al, 2003Behrens et al, 2003

Cortex segmentation Cortex segmentation is propagated is propagated medially using a medially using a Voronoi partitioning Voronoi partitioning schemescheme

AcknowledgementsAcknowledgements

This work was supported by the This work was supported by the USPHS under NIH grants MH62100, USPHS under NIH grants MH62100, NS044189, DA015886, NS045839, NS044189, DA015886, NS045839, and a Whitaker graduate fellowshipand a Whitaker graduate fellowship

Building a Brain Building a Brain Atlas Atlas

Using Graph Using Graph TheoryTheoryP. Yushkevich, A. Dubb and J. GeeP. Yushkevich, A. Dubb and J. Gee

University of PennsylvaniaUniversity of Pennsylvania

April, 2004April, 2004

Project AimsProject Aims

To Develop a Brain AtlasTo Develop a Brain Atlas An MRI template for registrationAn MRI template for registration A segmentation into anatomical A segmentation into anatomical

structuresstructures

Desired Properties of the Atlas:Desired Properties of the Atlas: High spatial resolutionHigh spatial resolution Large number of structuresLarge number of structures

Brodmann’s areasBrodmann’s areas Subcortical organsSubcortical organs

Atlas of Brodmann’s areas

http://ist-socrates.berkeley.edu/~jmp/LO2/6.html

Talairach AtlasTalairach Atlas

Frequently used to map Frequently used to map SPM locations to SPM locations to anatomical regionsanatomical regions

Contains many structures, Contains many structures, including Brodmann’s including Brodmann’s areasareas

Based on a post-mortem Based on a post-mortem study of an elderly subject study of an elderly subject so there is no matching so there is no matching MRIMRI

Low spatial resolutionLow spatial resolution

Talairach Brain Atlas

Courtesy of www.brainvoyager.com

Our ApproachOur Approach Build atlas for the Build atlas for the

widely used SPM ’96 widely used SPM ’96 templatetemplate Constructed by Constructed by

scanning a volunteer scanning a volunteer multiple times to multiple times to reduce noisereduce noise

Freely available from Freely available from the BrainWeb Projectthe BrainWeb Project

Segmentation into GM, Segmentation into GM, WM and CSF is also WM and CSF is also freely availablefreely available SPM’96 Template from BrainWeb

Initial ApproachInitial Approach Cortical structures:Cortical structures:

Segmented using surface-painting procedureSegmented using surface-painting procedure

Sub-cortical structuresSub-cortical structures Automatic, level set based segmentation for caudate and Automatic, level set based segmentation for caudate and

ventriclesventricles Manual SNAP segmentationManual SNAP segmentation

2D and 3D views of Left Cortical StructuresLevel set segmentation of the ventricles

Cortical SegmentationCortical Segmentation

Expert marks a dense set Expert marks a dense set of landmarks on the of landmarks on the surface of the cerebral surface of the cerebral cortexcortex

Landmarks are projected Landmarks are projected onto the white matter onto the white matter surface and connected, surface and connected, forming ribbonsforming ribbons

Ribbons are used as Ribbons are used as boundaries for level-set boundaries for level-set segmentationsegmentation

Grey matter with curves

Curves projected on the white matter

Initial Approach: Initial Approach: LimitationsLimitations

Curves projected on the Curves projected on the white matter surface have white matter surface have discontinuitiesdiscontinuities

Currently breaks in the Currently breaks in the projection are connected projection are connected by tracing the shortest by tracing the shortest path on the surfacepath on the surface

This results in ribbons that This results in ribbons that are unnatural and in are unnatural and in suboptimal segmentationsuboptimal segmentation

White matter projection discontinuity

Ribbons formed by projectionExample of segmentation error caused by a inadequate ribbon (the ribbon is pink)

Current Approach: Current Approach: Coarse StepCoarse Step

Combined GM and Combined GM and WM volume is divided WM volume is divided into three regions into three regions using graph using graph partitioning software partitioning software METISMETIS

Regions:Regions: Left cortexLeft cortex Right cortexRight cortex Cerebellum & brain Cerebellum & brain

stemstemCoarse level GM+WM partition

Current Approach: Fine Current Approach: Fine StepStep

Expert draws curves on Expert draws curves on the surface of a cortical the surface of a cortical region using Livewire-region using Livewire-style user interfacestyle user interface

These curves will be These curves will be used to partition the used to partition the surface into patchessurface into patches

Patch segmentation Patch segmentation will be propagated onto will be propagated onto the GM volume using the GM volume using level set/flow level set/flow techniques techniques

BrainTracer tool used to mark the cortical surface

Summary

Diffusion profile metric

Diffusion profile reorientationAnalytical derivative-based affine registrationPiecewise-affine registration

Thank You!

Towards Diffusion Profile Image Registration

Hui Zhang, Paul A. Yushkevich and James C. Gee

University of PennsylvaniaISBI’04

Funded in part by NIH grants MH62100, NS044189, DA015886, NS045839

Extra Slides

Motivation

Diffusion WeightedImaging

Diffusion ProfileImages

Spatial Normalizationamong Populations

New Insight intoWhite Matter

Structure

Affine Registration

Closed-form Objective function

Template Subject

Applied to diffusion tensors

Analytical Derivatives

Some notations:

Derivatives of rotation matrix

Derivatives of deformation matrix

Derivatives of translations

Template Subject

Non-rigid Extension

Piecewise affine registration

Multi-resolution hierarchyNumber of regions are

Displacement fields recovered at each level aresmoothed with B-Spline

0 1

2 3

Our Contributions

•Methodology for diffusion profile registration

•metric

•reorientation

•affine parametrization scheme

•affine registration objective function with analytical derivatives

•Results applied to diffusion tensors

•affine registration

•piecewise affine registration

Diffusion Profile Reorientation

The reorientation effect of rigid transformation

Reorientation OperatorA function of transformation’s Jacobian

matrixRigid transformation

Applied to diffusion tensors

Affine Reorientation

One possible solution

like “Preservation of principle directions”Non-linear and not shape preserving

Affine transformation is not an isometry and

Acknowledgements

This work was supported by the USPHS under NIH grants MH62100, NS044189, DA015886, NS045839, and a Whitaker graduate fellowship

Atlas Image Subject

Anatomic Labels

Individualized Atlas

Atlas-based Localization

Spatial Transformation

REGISTER

OVERLAYWARP

Anatomic Atlas

Witelson partitioning Witelson partitioning approach to approach to morphometrymorphometry

Deformation based Deformation based morphometrymorphometry Cite studiesCite studies

New, anatomy-based New, anatomy-based partitioning approachpartitioning approach

HowHow AtlasAtlas MatchingMatching Propagating labels---see Propagating labels---see

Behrens?Behrens?