MultiModality Registration MultiModality Registration Using Hilbert-Schmidt EstimatorsUsing Hilbert-Schmidt Estimators

By: Srinivas Peddi

Computer Integrated Surgery II

April 27th, 2001

Final PresentationFinal Presentation

OutlineOutline

• Brief refresher of my project

• Things accomplished in the project

• Things left to improve upon

• Future Directions

• Conclusions

The ProjectThe Project

T1 PDT2

I want to be able to register different modalities of MR images accurately. This means coming up with a new registration algorithm and getting around the intensity difference problem.

Original Maximal DeliverablesOriginal Maximal Deliverables

• To register the three different modalities accurately

by first using Bayesian Segmentation to circumvent

the intensity-difference problem.

• To compare this approach with other multimodality

registration algorithms such as the Maximization of

Mutual Information algorithm.

• To examine the feasibility of using the Hilbert-

Schmidt algorithm in real-life applications.

What Has Been Achieved?What Has Been Achieved?

T1 PDT2

Original images

PD SegmentationPD Segmentation T1 SegmentationT1 Segmentation T2 SegmentationT2 Segmentation

Original images Bayesian SegmentationOriginal images Bayesian Segmentation Switching of Intensities

PD Switched PD Switched SegmentationSegmentation

T1 Switched T1 Switched SegmentationSegmentation

T2 Switched T2 Switched SegmentationSegmentation

• Once the segmentation process is done, and the

intensities have been switched, we can actually do

the registration.

• We apply the Hilbert-Schmidt algorithm which uses

a minimum mean-squared (MMSE) estimator.

• Registration is achieved by finding the element of

the special Euclidean group (SEn) that minimizes

the error.

RegistrationRegistration

Registering different modalitiesRegistering different modalities

• The maximum aim of the project has been

achieved.

• At this point, we are able to register PD with T1

(which you saw in the checkpoint presentation)

but we can now also register PD with T2 and T1

with T2.

• These registrations are possible at different noise

levels as long as the segmentation is reasonable.

How accurate is this method?How accurate is this method?

• To examine how accurate something is, we must

first define an error measure. The one that I will be

using is called the Hilbert-Schmidt bound.

• A second thing that can help in getting an intuitive

feel about the accuracy of the algorithm, is having

another algorithm to compare it to. In this

presentation, I will be using the Maximization of

Mutual Information Algorithm.

What is the Hilbert-Schmidt bound?What is the Hilbert-Schmidt bound?

• The Hilbert-Schmidt norm is defined as the norm of a matrix. Example:

A = [2 -1 -4 -2]

||A|| = [ 22 + (-1)2 + (-4)2

+ (-2)2 ]1/2 = 5

• The Hilbert-Schmidt bound (HSB) is defined as the

matrix norm of the difference between the true matrix

transformation and the calculated matrix transformation.

Maximization of Mutual InformationMaximization of Mutual Information

• This algorithm was implemented by Wells et al.

at the SPL in 1996. Since then, it has become

the registration tool of choice when doing

multimodality registration.

• The algorithm attempts to find the registration by

maximizing the information that one volumetric

image provides about the other.

Comparison of the two algorithmsComparison of the two algorithms

As one can see, the Hilbert-Schmidt algorithm outperforms the mutual information algorithm at high noise but at low noise, they both register the images accurately.

HSB as a function of Signal Strength for registering T1 vs. T2 images

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Signal Strength

HS

B Mutual Information

Hilbert-Schmidt

Pros and ConsPros and Cons

• Accuracy: As mentioned, it seems that the Hilbert-Schmidt

algorithm outperforms the mutual information algorithm in this

category, especially at high noise levels.

• Speed: The Mutual Information algorithm runs much faster,

at least for now, especially because it does not do much

preprocessing.

• Generality: The Mutual Information algorithm assumes no a

priori relationship between the two modalities and hence all

modalities can be registered using the same algorithm. The

Hilbert-Schmidt algorithm is striving to do the same.

Pros and Cons Cont’dPros and Cons Cont’d

• Ease of use: Since the Mutual Information algorithm has

less steps or at least is better integrated, it is easier to

use. The hope is that later, the segmentation and the

registration can be coupled in one program in which

case, the Hilbert-Schmidt would also become easier to

use.

• Robustness: Since the Mutual Information algorithm is

essentially a simpler algorithm with less steps, it is very

robust. With time, I hope to make the Hilbert-Schmidt

algorithm as robust.

Improvements to be madeImprovements to be made

• Gradient descent algorithm has been

implemented but can be improved upon especially

by using a ‘blurring’ algorithm and also by

selecting random points more wisely.

• The algorithm needs to be extended so that one

can register 3D volumes rather than just 2D

images which is what we have now.

Gradient Descent AlgorithmGradient Descent AlgorithmProbability Density Function

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0 30 60 90 120 150 180 210 240 270 300 330 360

Angle of Rotation

Pro

ba

bil

ity

What I presently do is pick a series of random points from 0 to What I presently do is pick a series of random points from 0 to 360, and then march in the direction of increasing probability. It 360, and then march in the direction of increasing probability. It would be nice to add two layers of random points so that there would be nice to add two layers of random points so that there would be more density of random points near the maximum. would be more density of random points near the maximum. Being able to blur the image without shifting the maximum Being able to blur the image without shifting the maximum would also be helpful.would also be helpful.

Future DirectionsFuture Directions

• From the analysis so far it seems that the Hilbert-

Schmidt algorithm outperforms other algorithms at

higher noise levels. Therefore, it is possible that

the HS algorithm might be very useful in functional

modalities such as fMRI and PET where the SNRs

are considerably higher than MR or CT.

• The segmentation algorithm can be improved to

incorporate things such as windowing so as to

improve the accuracy of the algorithm.

Feasibility of HS algorithm in Feasibility of HS algorithm in real-life circumstancesreal-life circumstances

• The algorithm developed in this project definitely has

potential for real-life applications.

• However, some key improvements have to be made first

which integrate the various steps of the protocol better

and the algorithm needs to extended to 3D volumes.

• As far as speed goes, computers are becoming more

powerful as we speak, and it does not seem that this will

be a limiting factor in the application of this algorithm.

What have I learned?What have I learned?

• Projects are a lot of work, and many times, things

do not go according to plan. Always have

contingencies.

• Presentation of ideas well is very important. Almost

as important as the project itself. Presentation is

also a lot more work than I thought.

• There are many tools out there that can make one’s

life easier. It is much better to use these tools than

trying to reconstruct the wheel.

What would I do differently?What would I do differently?

• Have a more detailed plan at the beginning of the

project. There were points when I wasn’t sure if I was

ahead or behind schedule.

• Have a thorough understanding of the theory of the

topic before starting to code. It can save a lot of time

and effort!

• Efficiency and ease-of-use are the two very important

features of an algorithm. I wish I had started working

earlier on those aspects of the algorithm.

ConclusionsConclusions

• I have a learned a lot from this class and there

are some things that I would do differently if I

have were to start from the beginning again.

• However, all in all, I think that the project went

relatively smoothly and that the Hilbert-Schmidt

algorithm developed here has potential for

success in the future.

The most important conclusion to draw from this presentation though is that …

I’m DONE!!!I’m DONE!!!