A New Method of Probability Density Estimation for Mutual Information Based

Image Registration

Ajit Rajwade, Ajit Rajwade,

Arunava Banerjee,Arunava Banerjee,

Anand Rangarajan.Anand Rangarajan.

Dept. of Computer and Information Sciences & Engineering,

University of Florida.

Image Registration: problem definition

• Given two images of an object, to find the geometric transformation that “best” aligns one with the other, w.r.t. some image similarity measure.

Mutual Information for Image Registration

• Mutual Information (MI) is a well known image similarity measure ([Viola95], [Maes97]).

• Insensitive to illumination changes; useful in multimodality image registration.

)|()(),( 21121 IIHIHIIMI

)|( 21 IIH

),()()(),( 212121 IIHIHIHIIMI

)|( 12 IIH),IH(I 21

),(MI 21 II)( 1IH )( 2IH

)( 1IH ),( 21 IIH

Marginal entropy Joint entropy

Mathematical Definition for MI

)|( 21 IIH

Conditional entropy

Calculation of MI

• Entropies calculated as follows:

)(p 2112 ,)(

)(

22

11

p

pJoint Probability

Marginal Probabilities

),(log),(),(

)(log)()(

)(log)()(

2112211221

22222

11111

1 2

2

1

ppIIH

ppIH

ppIH

Joint Probability

j)(ip ,12

1I 2I

),( 21 IIH ),(MI 21 II

Functions of Geometric Transformation

Estimating probability distributions

Histograms

How do we selectbin width?

Too large bin width:Over-smooth distribution

Too small bin width:Sparse, noisy distribution

Estimating probability distributions

Parzen Windows

Choice of kernel

Choice of kernel width

Too large:Over-smoothing

Too small:Noisy, spiky

Estimating probability distributions

Mixture Models[Leventon98]

How many components?

Difficult optimization in every step of registration.

Local optima

Direct (Renyi) entropy estimation

Minimal SpanningTrees, Entropic kNN Graphs

[Ma00, Costa03]

Requires creation of MSTfrom complete graph of all samples

Cumulative Distributions

Entropy definedon cumulatives

[Wang03]

Extremely Robust,Differentiable

A New Method

What’s common to allprevious approaches?

Take samplesObtain approximation

to the density

More samples More accurateapproximation

A New Method

Assume uniform distribution on location

TransformationLocation

Intensity

Distribution on intensity

Uncountable infinityof samples taken

Each point in thecontinuum contributes

to intensitydistribution

Image-Based

Other Previous Work

• A similar approach presented in [Kadir05].

• Does not detail the case of joint density of multiple images.

• Does not detail the case of singularities in density estimates.

• Applied to segmentation and not registration.

A New Method

Continuous image representation (use some interpolationscheme) No pixels!

Trace out iso-intensity level curves of the imageat several intensity values.

Intensity at Curves Level andIntensity at Curves Level regionbrown of area )( IP

Analytical Formulation: Marginal Density

• Marginal density expression for image I(x,y) of area A:

• Relation between density and local image gradient (u is the direction tangent to the level curve):

I

dxdy

Ap ] 0lim[

1)(

I yxI

du

Ap

|),(|

1)(

Joint Probability

2211 Iin and Iin Intensity at Curves Level 2222

1111

Iin ),( and

Iin ) ,(Intensity at Curves Level

regionblack of area

),( 22221111 IIP

Joint Probability

Analytical Formulation: Joint Density

• The joint density of images and with area of

overlap A is related to the area of intersection of the

regions between level curves at and of

, and at and of as

.

• Relation to local image gradients and the angle

between them ( and are the level curve tangent vectors in the two images):

1 11

2 22 0,0 21

2211 , 21

2121 |sin),(),(|

1),(

IIyxIyxI

dudu

Ap

),(1 yxI ),(2 yxI

1I

2I

1u 2u

Practical Issues

• Marginal density diverges to infinity, in areas of zero gradient (level curve does not exist!).

I yxI

du

Ap

|),(|

1)(

2211 , 21

2121 |sin),(),(|

1),(

IIyxIyxI

dudu

Ap

• Joint density diverges in areas of zero gradient of either or both image(s). in areas where gradient vectors of the two images are parallel.

Work-around

• Switch from densities (infinitesimal bin width) to distributions (finite bin width).

• That is, switch from an analytical to a computational procedure.

Binning without the binning problem!More bins = more (and closer) level curves.

Choose as many bins as desired.

Standard histograms Our Method32 bins64 bins128 bins256 bins512 bins1024 bins

Pathological Case: regions in 2D space whereboth images have constant intensity

Pathological Case: regions in 2D space whereonly one image has constant intensity

Pathological Case: regions in 2D space where gradients from the

two images run locally parallel

Registration Experiments: Single Rotation

• Registration between a face image and its 15 degree rotated version with noise of variance 0.1 (on a scale of 0 to 1).

• Optimal transformation obtained by a brute-force search for the maximum of MI.

• Tried on a varied number of histogram bins.

MI Trajectory versus rotation: noise variance 0.1

Standard Histograms Our Method

16 bins32 bins64 bins128 bins

MI Trajectory versus rotation: noise variance 0.8

Standard Histograms Our Method

16 bins32 bins64 bins128 bins

PD slice T2 slice

Affine Image Registration

BRAINWEB

Warped T2 sliceWarped and Noisy T2 slice

Brute force search for themaximum of MI

Affine Image RegistrationMI with standard

histograms

MI with our method

Directions for Future Work

• Our distribution estimates are not differentiable as we use a computational (not analytical) procedure.

• Differentiability required for non-rigid registration of images.

Directions for Future Work

• Simultaneous registration of multiple images: efficient high dimensional density estimation and entropy calculation.

• 3D Datasets.

References

• [Viola95] “Alignment by maximization of mutual information”, P. Viola and W. M. Wells III, IJCV 1997.

• [Maes97] “Multimodality image registration by maximization of mutual information”, F. Maes, A. Collignon et al, IEEE TMI, 1997.

• [Wang03] “A new & robust information theoretic measure and its application to image alignment”, F. Wang, B. Vemuri, M. Rao & Y. Chen, IPMI 2003.

• [BRAINWEB] http://www.bic.mni.mcgill.ca/brainweb/

References

• [Ma00] “Image registration with minimum spanning tree algorithm”, B. Ma, A. Hero et al, ICIP 2000.

• [Costa03] “Entropic graphs for manifold learning”, J. Costa & A. Hero, IEEE Asilomar Conference on Signals, Systems and Computers 2003.

• [Leventon98] “Multi-modal volume registration using joint intensity distributions”, M. Leventon & E. Grimson, MICCAI 98.

• [Kadir05] “Estimating statistics in arbitrary regions of interest”, T. Kadir & M. Brady, BMVC 2005.

Acknowledgements

• NSF IIS 0307712

• NIH 2 R01 NS046812-04A2.

Questions??