Registration of MR Images Master Thesis By Naga Padma Krishnam Raju Dandu Supervisor: Ole Fogh...

Registration of MR Images Registration of MR Images

Master ThesisMaster Thesis

ByByNaga Padma Krishnam Raju DanduNaga Padma Krishnam Raju Dandu

Supervisor:Supervisor:Ole Fogh Olsen, associate professor, IT University of CopenhagenOle Fogh Olsen, associate professor, IT University of Copenhagen

Table of ContentsTable of Contents

1. Introduction2. Objective3. Registration4. Rigid Registration5. Results on Rigid Registration6. Non Rigid Registration7. Results on Non Rigid Registration8. Conclusion9. Future Improvements10.References

IntroductionIntroduction

Motivation Motivation to increase the accuracy, while fusing the useful information from to increase the accuracy, while fusing the useful information from

differnt MR images, inorder to quatify the articular cartilage of differnt MR images, inorder to quatify the articular cartilage of knee during the osteoartherities studyknee during the osteoartherities study

OsteoarthritisOsteoarthritis

Figure 1: (a) Normal knee joint (side view) (b) A knee joint with osteoarthritis [1]

Goals Goals

QuantificationQuantification FusionFusion RegistrationRegistration

ObjectiveObjective

Registration of MR ImageRegistration of MR Image Why MRI only ?Why MRI only ?

DefinitionDefinition

RegistrationRegistration Types of RegistrationTypes of Registration

Type of ImageType of Image 2D-2D2D-2D 3D-3D 3D-3D 2D-3D2D-3D

Subject of imageSubject of image IntrapersonalIntrapersonal InterpersonalInterpersonal

Modality of Image Modality of Image Mono ModelMono Model MultimodalMultimodal

TransformationTransformation RigidRigid Non RigidNon Rigid

Search ModeSearch Mode LandmarkLandmark Voxel IntensityVoxel Intensity

Processes in RegistrationProcesses in Registration

Transformation Transformation Interpolation Interpolation Similarity Similarity MeasureMeasure

Initialize one image as It Template Image and other as Reference image Ir

Is similarity optimal

Transform template image ItT

Use interpolation to find voxel intensity of It

T

Compute similarity measure between It

T and Ir

Register It with Ir

Yes

No

Figure : General work flow of registration

OptimizationOptimization

++

Rigid RegistrationRigid Registration

Rigid Transformation Rigid Transformation Translation Translation RotationRotation

Interpolation Interpolation Nearest Neighbour Nearest Neighbour LinearLinear

Required Voxel

(a) (b)

Figure : (a) pixel in Template Image (2D) (b) Transformed pixel in template image

A(i+1,j,k+1)

A(i+1,j+1,k+1)

A(i+1,j,k)A(i,j,k)

A(i,j,k+1)

A(i+1,j+1,k)A(i,j+1,k)

A(i´,j´,k´)

Figure 12: Finding intensity from boarders using interpolation

Similarity measures-1Similarity measures-1

SSD (Sum of the Squared Differences)SSD (Sum of the Squared Differences)If is the template image to register and is the reference image If is the template image to register and is the reference image thenthen

Where : voxel, : transformation and : total number of voxels.Where : voxel, : transformation and : total number of voxels. NCC (Normalized Cross Correlation)NCC (Normalized Cross Correlation)

Here : mean intensity in reference image Here : mean intensity in reference image

: mean intensity in template image: mean intensity in template image

2)))(()((

1 vTIvIN

SSD tr

tI rI

v T N

N

i

N

ititrir

N

ititrir

vTIvI

vTIvINCC

1 1

22

1

))))((()())(((

)))(()()((

rrI

ttI

Similarity measures-2Similarity measures-2

RIU (Ratio Image Uniformity)RIU (Ratio Image Uniformity)

Here : standard deviation of ratios Here : standard deviation of ratios : mean value of ratios: mean value of ratios

NMI (Normalized Mutual Information)NMI (Normalized Mutual Information)

Here : Marginal Entropy of Image Here : Marginal Entropy of Image

: Joint Entropy of image and : Joint Entropy of image and

r

rRIU

r rr r

)(

)()(

it

iri vI

vIvr

),()()(),( trjtmrmtr IIHIHIHIIMI )( rm IH

rI

),( trj IIH rI tI

),(

)()(),(

tr

trtr IIH

IHIHIINMI

Implementation DetailsImplementation Details

Implementing TransformationsImplementing Transformations

Required Voxel

(a) (b)

Required Voxel Intensity

(c) (d)

Transformation

Reverse Transformation

Finding intensity from boarders using Interpolation

A(i+1,j,k+1)

A(i+1,j+1,k+1)

A(i+1,j,k)A(i,j,k)

A(i,j,k+1)

A(i+1,j+1,k)A(i,j+1,k)

A(i´,j´,k´)

TIRI tT

t *

)(*1 TIRI Ttt

Figure : (a) pixel in Template Image (2D) (b) Transformed pixel in template image

(c) Pixel in transformed template image (d) Reverse transformed pixel co-ordinates in original template image


Handling Resolution Differences during Similarity MeasureHandling Resolution Differences during Similarity Measure

Implementation DetailsImplementation Details

(a) (b)

X in mm

Y in

mm

X in mm

Y in

mm

Figure 2: (a) voxel in transformed template image in x y and z-directions (b) voxel in reference image in x, y and z-directions


Implementation DetailsImplementation Details Hierarchal OptimizationHierarchal Optimization

Course level

- - - - - - 0.1*Lx0.1*Mx0.1*N

- - - - - -

0.2*Lx0.2*Mx0.2*N

Resolution increases by 2 times

-------------LxMxN

Level 1

Level 2

Finer Level

Parameter step size 1 in X, Y and Z directopns

Parameter step size decresed by half in X, Y and Z directopns

Parameter step size 0.5 in X, Y and Z directopns

Hirarichal Optimization


Initialize parameter region [P1:step:P2], depth levels D and course level sampling rate S for

resolution

Is itr > D

Find template image and corresponding reference imege

Find optimal parameters optP using registration algorithm( iteartively transform the template and find

maximum similarity transformation with respect to reference image)

Register the template image with final optimal parameter combination

Yes

No

Initialse itr = 1

Is optimal parameters fall in boundaryYes

No

P1=optP-step:P2=optP+step

Itr = itr +1S = S*2

step=step/2;

Re initilize parameter space[P1:step:P2]

Hirarichal Optimization

Limitations of Rigid Registration AlgorithmLimitations of Rigid Registration Algorithm

Strictly for rigid featuresStrictly for rigid features Not guaranteed under Noisy environmentNot guaranteed under Noisy environment Local Minimization/MaximizationLocal Minimization/Maximization

Good GuessBad Guess

Loacal Minimum Problem

Cost Function


ResultsResults

Tested on MR Images of Knee and Phantom[7]Tested on MR Images of Knee and Phantom[7]


Turbo 3D T1Turbo 3D T1(256x256x120)(256x256x120)High Res. GE 16(512x512x26)High Res. GE 16(512x512x26)

Gradient Echo T2(256x256x16)Gradient Echo T2(256x256x16) Gradient Echo STIR(256x256x26) Gradient Echo STIR(256x256x26)

Patient 00xx - Turbo 3D T1, Pilot Patient 00xx - High Res. GE 16, Pilot

Patient 00xx - Gradient Echo T2, Pilot Patient 00xx - Gradient Echo STIR, Pilot

MR Images of PhantomMR Images of Phantom

ResultsResults

Tested on MR Images of Knee and Phantom[7]Tested on MR Images of Knee and Phantom[7]


Turbo 3D T1(256x256x104) Turbo 3D T1(256x256x104) High Res. GE 16(512x512x20)High Res. GE 16(512x512x20)

Gradient Echo T2(256x256x20)Gradient Echo T2(256x256x20) Gradient Echo STIR(256x256x20) Gradient Echo STIR(256x256x20)

Patient 0006 - Turbo 3D T1, Pilot Patient 0006 - High Res. GE 16, Pilot

Patient 0006 - Gradient Echo T2, PilotPatient 0006 - Gradient Echo STIR, Pilot

MR Images of KneeMR Images of Knee

ResultsResultsPhantom MRI

Template Image Reference Image

name ‘00xx-High Res. GE 16, Pilot’ ‘00xx-Turbo 3D T1, Pilot’

Image modality

MRI MRI

Original resolution

512x512x26 256x256x120

Voxel sizes along [X, Y,

Z] in mm[0.3516, 0.3516, 4.4000] [0.7031, 0.7031, 0.8594]

Original image

Lookup resolution

512x512x26 (always template image)

Lookup reference

image data

Template Image00xx-High Res. GE 16, Pilot

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

500

Reference Image00xx-Turbo 3D T1, Pilot

50 100 150 200 250

50

100

150

200

250

Reference Image00xx-Turbo 3D T1, Pilot

50 100 150 200 250

50

100

150

200

250



S. No Initialparameter rangein all directions

Step size

initial sampling

iterations interpolation SimilarityMeasure

Registered image with parameter(in voxels)

Time taken(hours)

1 [-2 2] 2 .0250 2 trilinear RIU [3 -2 0 0 0 1] 16.6740

2 [-2 2] 2 .0250 2 trilinear CC [1 2 -1 0 0 0] 44.5191

3 [-2 2] 2 .0250 2 trilinear SSD [-5 -6 10 3 0 1] 35.2291

4 [-2 2] 2 .0250 2 trilinear NMI [-7 -5 1 0 0 0] 86.4965



combined image before registration

combined image after registration using RIU combined image after registration using CC

combined image after registration using SSD combined image after registration using NMI


ResultsResultsKnee MRI

S. No Intial parameter range in all directions

Step size

InitialSampling quatity

iterations

interpolation Similarity measure

Registered image with parameter(in voxels)

Time taken(hours)

1 [-2 2] 2 0.1 2 trilinear RIU [0 -1 2 0 0 0] 1.3817

2 [-2 2] 2 0.1 2 trilinear CC [0 0 -1 0 0 -1] 1.4163

3 [-2 2] 2 0.1 2 trilinear NMI [11 7 0 0 0 -8] 10.5982

4 [-2 2] 2 0.05 2 trilinear SSD [-11 4 11 -4 -2 -2] 8.0679

5 [-2 2] 2 0.05 2 trilinear RIU [-3 -2 -3 0 0 0] 15.6590

6 [-2 2] 2 0.05 2 trilinear CC [1 0 -1 0 0 -1] 29.5741

7 [-2 2] 2 0.05 2 trilinear NMI [18 30 0 0 0 26] 70.1021

8 [-2 2] 2 0.1 3 trilinear RIU [-0.5 -1 2 0 0 0] 14.5860

9 [-2 2] 2 0.1 3 trilinear CC [0.5 -0.5 -1.5 0 0 0]

31.7189

10 [-2 2] 2 0.1 3 trilinear NMI [24 21 0.5 0 0 -25.5]

338.1166




combined image after registration using CC

combined image after registration using RIU

combined image after registration using NMI

Registered Knee Images with depth level 2 and Sampling rate 0.1 of S.No 1 , 2 and 3 from above table




Registered Knee Images with depth level 2 and Sampling rate 0.05 of S.No 4 , 5, 6 and 7 from above table

combined image after registration using SSD combined image after registration using RIU

combined image after registration using CC combined image after registration using NMI




Registered Knee Images with depth level 3 and Sampling rate 0.1 of S.No 8 , 9 and 10 from above table

combined image after registration using RIU

combined image after registration using CC combined image after registration using NMI


DiscussionDiscussion

Which Similarity Measure (such as SSD, NCC, RIU and NMI) Which Similarity Measure (such as SSD, NCC, RIU and NMI) gave better results?gave better results?

Why?Why?


FlawsFlaws

Good guess of initial sampling rate is neededGood guess of initial sampling rate is needed Execution is too slow. Better optimizationExecution is too slow. Better optimization Programs are hard coded in parameter re-initializationPrograms are hard coded in parameter re-initialization Noise tolarability Noise tolarability


Non Rigid RegistrationNon Rigid RegistrationNon Rigid RegistrationNon Rigid Registration

Why Non Rigid Registration ?Why Non Rigid Registration ? What is Non Rigid Registration ?What is Non Rigid Registration ?

Non Rigid Transformations 1Non Rigid Transformations 1Non Rigid RegistrationNon Rigid Registration

Scale TransformationsScale Transformations

Affine TransformationsAffine Transformations

z

y

x

S

S

S

S

00

00

00

TIRSI tT

t **

333231

232221

131211

aaa

aaa

aaa

A TIAI tT

At *


Curve TransformationsCurve Transformations Cubic SplinesCubic Splines

Thin plate splinesThin plate splines

Figure : Grid of Knots for Cubic Splines

Image

Control Voxel Grid


Two level Transformation ModelTwo level Transformation Model

Regularization term*Regularization term*

Cost Function ( similarity measure)Cost Function ( similarity measure)

LocalGlobal TTT CSplinesLocal TT

ScaleGlobal TT

dxdydzxz

T

yz

T

xy

T

z

T

y

T

x

TC

X Y Z

smooth

0 0 0

2222222

2

22

2

22

2

2

222V

1

smoothT

trsimilarity CIICC ),(

*This penalty term was earlier used by [33] D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes for their application to Brest MR Images. The same penalty term is adapted to the current Knee MR Images.

Search CriteriaSearch CriteriaNon Rigid RegistrationNon Rigid Registration

Search CriteriaSearch Criteria

Figure Figure : search criteria in non-rigid registration: search criteria in non-rigid registration

Image

Control Voxel Grid

Control voxel grid has to move 1 in each direction to

cover the entire search region

Search Criteria

AlgorithmAlgorithmNon Rigid RegistrationNon Rigid Registration

AlgorithmAlgorithm

1.1. Do the rigid registration and find out optimal translational and rotational parameters Do the rigid registration and find out optimal translational and rotational parameters

2.2. Initialize search region for scale transformations and transform the image and find the optimal scale transformation Initialize search region for scale transformations and transform the image and find the optimal scale transformation

3.3. Initialize grid step size & depth of hierarchy levelInitialize grid step size & depth of hierarchy level

4.4. Repeat until required depth is achievedRepeat until required depth is achieved For each possible position of control voxels gridFor each possible position of control voxels grid

• Initialize control voxel gridInitialize control voxel grid• Perform spline interpolation Perform spline interpolation

• Calculate the extra penalty term called regularizar Calculate the extra penalty term called regularizar • Calculate the total costCalculate the total cost

Update depth & grid step sizeUpdate depth & grid step size

5.5. Find the minimum of all costs, optimal depth and display corresponding image volume as registered image volume. Find the minimum of all costs, optimal depth and display corresponding image volume as registered image volume.


Registered Knee Images with lamda 0.2 and Depth level 2, 3, 4 and 5

Non Rigid RegistrationNon Rigid Registration

diff b/w rigid registered and reference

diff b/w template and reference non rigid registered(depth=2, lamda=0.1)

non rigid registered(depth=3, lamda=0.1)




Registered Knee Images with lamda 0.2 and Depth level 2, 3, 4 and 5


diff b/w rigid registered and reference

diff b/w template and reference non rigid registered(depth=2, lamda=0.2)



DiscussionDiscussion

Is the results are improved from rigid registration?Is the results are improved from rigid registration? Why?Why? Any flaws?Any flaws? Optimal weighting factor?Optimal weighting factor? Is penalty term good enough?Is penalty term good enough?


Conclusion 1Conclusion 1


Successfully doneSuccessfully done 3D-3D Intrapersonel Multi model registration3D-3D Intrapersonel Multi model registration Handling of resolution differences between imagesHandling of resolution differences between images Rigid transformations and interpolation using linear interpolation methodRigid transformations and interpolation using linear interpolation method Investigation of suitable similarity measure among Sum of the Squared Diffrences, Investigation of suitable similarity measure among Sum of the Squared Diffrences,

Normalized Cross Correlation, Normalized Cross Correlation, Ratios Image Uniformity, Normalized Mutual Ratios Image Uniformity, Normalized Mutual Information.Information.

Hirarichal OptimizationHirarichal Optimization



Future ImprovementsFuture Improvements Better OptimizationBetter Optimization

• Better parameter re-initialization Better parameter re-initialization • Should handle local minimum/maximum problemShould handle local minimum/maximum problem• Should include more percentage of voxels during similarity measureShould include more percentage of voxels during similarity measure• Redundancy should be reduced in iterationsRedundancy should be reduced in iterations

Instead of going from coarse level to finer level during hierarchal optimization, It has to check from Instead of going from coarse level to finer level during hierarchal optimization, It has to check from small sub image from the center of image to full image verification.small sub image from the center of image to full image verification.

Noise tolerability has to be tested.Noise tolerability has to be tested. Better implementation & executions using Visual c++ ITK tools instead of slow matlab routinesBetter implementation & executions using Visual c++ ITK tools instead of slow matlab routines



Successfully doneSuccessfully done Two level transformation approchTwo level transformation approch Global transformations using Scale transformatinsGlobal transformations using Scale transformatins Local transformations using Cubic SplinesLocal transformations using Cubic Splines Regulizer termRegulizer term Similarity mesure using NCCSimilarity mesure using NCC Improved resultsImproved results

Need to be takes care ofNeed to be takes care of Cut off weighing factor LamdaCut off weighing factor Lamda More experiments to find optimal depthMore experiments to find optimal depth



Future ImprovementsFuture Improvements Instead of checking from the more number of knots to less number of Instead of checking from the more number of knots to less number of

knots in depth levels, it has to verify from less number of knots to more knots in depth levels, it has to verify from less number of knots to more number of knots during depth levelsnumber of knots during depth levels

Noise tolerability has to be tested. Noise tolerability has to be tested.

1.1. Derek L G Hill, Philipp G Batchelor, Mark Holden and David J Hawkes, 12 June 2000, Topical review, Medical Image RegistrationDerek L G Hill, Philipp G Batchelor, Mark Holden and David J Hawkes, 12 June 2000, Topical review, Medical Image Registration

2.2. J.B. Antoine Maintz, Max A.Viergever, Medical Image Analysis (1998) volume 2, number 1, pp 1-36, Oxford University Press, A J.B. Antoine Maintz, Max A.Viergever, Medical Image Analysis (1998) volume 2, number 1, pp 1-36, Oxford University Press, A Survey of Medical Image Registration.Survey of Medical Image Registration.

3.3. Rasmus Larsen, DTU, Teaching material for medical image analysis ‘Image registration pixel/voxel based’Rasmus Larsen, DTU, Teaching material for medical image analysis ‘Image registration pixel/voxel based’

4.4. J.Michal Fitzpatrick, Derek L.G.Hill, Calvin R. Maurer. Jr, chapter 8 ’ Image registration’J.Michal Fitzpatrick, Derek L.G.Hill, Calvin R. Maurer. Jr, chapter 8 ’ Image registration’

5.5. A.Ardeshir Goshtasby ‘2-D and 3-D Image registration for Medical, Remote Sensing and Industrial applications’A.Ardeshir Goshtasby ‘2-D and 3-D Image registration for Medical, Remote Sensing and Industrial applications’

6.6. John Ashburner & Karl J.Friston, chapter2, ‘Rigid body Registration’John Ashburner & Karl J.Friston, chapter2, ‘Rigid body Registration’

7.7. Phantom and Knee MRI test images from CCBR research institute through Eric DamPhantom and Knee MRI test images from CCBR research institute through Eric Dam

8.8. Hongliang Yu, may 2005, Dissertation on ‘automatic Rigid and Deformable Medical image Registration’Hongliang Yu, may 2005, Dissertation on ‘automatic Rigid and Deformable Medical image Registration’

9.9. Ramsay & Silverman (1997) “Functional Data Analysis”Ramsay & Silverman (1997) “Functional Data Analysis”

10.10. Sky McKinley & M Levine “Cubic Spline Interpolation” Sky McKinley & M Levine “Cubic Spline Interpolation”

11.11. Gerardo I. Sánchez-Ortiz, Daniel Rueckert and Peter Burger “Motion and Deformation Analysis of the Heart using Thin-Plate Splines Gerardo I. Sánchez-Ortiz, Daniel Rueckert and Peter Burger “Motion and Deformation Analysis of the Heart using Thin-Plate Splines and Density and Velocity Encoded MR Images” http://wwwhomes.doc.ic.ac.uk/~giso/pubs/leedsok/leedsok.htmland Density and Velocity Encoded MR Images” http://wwwhomes.doc.ic.ac.uk/~giso/pubs/leedsok/leedsok.html

12.12. D. Rueckert,* L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes “Nonrigid Registration Using Free-Form D. Rueckert,* L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes “Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images” IEEE Transactions on medical imaging, Vol. 18, August 1999Deformations: Application to Breast MR Images” IEEE Transactions on medical imaging, Vol. 18, August 1999

13.13. The Arthritis Research Campaign (arc), http://www.arc.org.uk/about_arth/booklets/6027/6027.htmThe Arthritis Research Campaign (arc), http://www.arc.org.uk/about_arth/booklets/6027/6027.htm

THANKSTHANKS

ReferencesReferences

Registration of MR Images Master Thesis By Naga Padma Krishnam Raju Dandu Supervisor: Ole Fogh...

Documents

Transcript of Registration of MR Images Master Thesis By Naga Padma Krishnam Raju Dandu Supervisor: Ole Fogh...