A minimax entropy registration framework for patient setup verification in radiotherapy

Biomedical Paper

A Minimax Entropy Registration Framework forPatient Setup Verification in Radiotherapy

Ravi Bansal, M.Phil., Lawrence Staib, Ph.D., Zhe Chen, Ph.D., Anand Rangarajan, Ph.D.,Jonathan Knisely, M.D., Ravinder Nath, Ph.D., and James Duncan, Ph.D.

Department of Electrical Engineering and Department of Diagnostic Radiology (R.B., L.S., A.R., J.D.)and Department of Therapeutic Radiology (Z.C., J.K., R.N.), Yale University, New Haven, Connecticut

ABSTRACT In external beam radiotherapy (EBRT), patient setup verification over the entirecourse of fractionated treatment is necessary for accurate delivery of a specified dose to the tumor.We are working on the development of a minimax entropy registration framework for patient setupverification using dual portal images and the treatment planning 3D CT dataset. In this paper, wepresent an overview of our registration framework, where an iteratively and automatically estimatedsegmentation of the portal image is utilized to more accurately and robustly register the portal imageto the 3D treatment-planning CT data. In addition, we describe initial testing of this approach. Wenote that, due to low resolution and low contrast of the portal images, this registration presents adifficult problem. We also note that the registration of the images in our proposed method is guidedby the bony structure visible in the portal and the 3D CT images. However, since the prostate canmove with respect to the pelvic bone, we propose using ultrasound images to quantify this move-ment. Comp Aid Surg 4:287–304 (1999). ©1999 Wiley-Liss, Inc.

Key words: registration, segmentation, portal images, simulator images, radiotherapy, computedtomography

INTRODUCTIONThe effectiveness of external beam radiotherapy(EBRT) for prostate cancer treatment is decreaseddue to a variety of uncertainties in the treatmentsetup, including the physical characteristics of thetreatment beam, patient positioning issues, patientorgan motion, and operator non-reproducibility.The development and administration of a treatmentplan using image-guided techniques to account forsome of these uncertainties could improve the ef-fectiveness of EBRT.

In this work, we focus primarily on reducingerrors and uncertainties due to patient positioning.First, we note that uncertainties due to patient setup

errors can be reduced by registering the high-con-trast simulator images obtained at diagnostic ener-gies (40–100 KeV) to the low-resolution, low-contrast two-dimensional (2D) portal images,which are obtained using the treatment energy X-rays (4–20 MeVs). However, 2D analysis of thepatient setup verification using single portal andsimulator images is restricted to the verification ofonly in-plane rotations and translations; out-of-plane rotations and translations of the patient candegrade the accuracy of the image registration.14

To account for out-of-plane rotations and transla-tions, a pair of simulator and portal images of the

Received February 22, 1999; accepted September 7, 1999.

Address correspondence/reprint requests to: R. Bansal, Department of Diagnostic Radiology, Yale University, BML 332, 310Cedar Street, New Haven, CT 06520-8042. E-mail: [email protected].

Computer Aided Surgery 4:287–304 (1999)

©1999 Wiley-Liss, Inc.

same patient, obtained from different views, can beemployed. Such a three-dimensional (3D) analysisof the patient setup can lead to inconsistencies inthe determination of the transformation parame-ters.14 Many treatment centers are moving towardoffering full 3D conformal treatments that are ini-tially planned from 3D CT datasets. Thus, for con-sistent and accurate 3D analysis of the patientsetup, it is necessary to register the 3D CT datasetsto multiple 2D portal images. Unfortunately, due tothe poor quality of the portal images, automatedregistration of the portal images to the CT datasethas remained a difficult task.

Related WorkA number of studies have been proposed for both2D and 3D analyses of the patient setup. Algo-rithms for 2D analysis include gray-level intensity-based image alignment algorithms,3,15 visual in-spection by the physician,24 and anatomicallandmark-based approaches.5,14,21–23 Approachesthat register 3D treatment-planning CT data to 2Dportal images include interactive determination ofpatient setup,12 silhouette-based techniques,19 gray-scale correlation-based methods,7,20 a pattern-in-tensity-based method,30 a method based on medi-alness orcores,10 and a ridge-based method.11 Aninteresting optical flow-based registration methodis proposed by Vemuri et al.,28 while an ICP-basedregistration algorithm for registering 3D MR an-giography and 2D X-ray angiograms has been pro-posed by Feldmar et al.9 Gueziec et al.13 proposedan anatomy-based registration method for register-ing CT scans to intraoperative X-ray images. Theaccuracy of the estimated transformation parame-ters in the feature-based registration algorithms isrestricted by the accuracy of the detection andlocalization of the features in the portal image. Theprocess of feature extraction is sensitive to thenoise in the image. Since the portal images are ofpoor quality, this often requires human interventionfor correct detection and localization of the fea-tures. On the other hand, the gray-scale-based reg-istration methods, though robust to noise, tend to becomputationally expensive.

Our ApproachIn this work we focus on registering bony structureto align 3D CT and portal films as a first steptoward reducing patient setup errors. If the 2Dportal and 3D CT images are properly aligned, thenthe information from the high-resolution 3D CTdata can be used to segment the portal image. Onthe other hand, if we have an accurate segmentation

of the portal image, an accurate registration can beobtained. Thus, this becomes a problem of decidingwhether to try to locate a structure in the portalimage for registration purposes or to first perform acrude registration that might help find a structurethat could be used for better registration. In thispaper we propose that there is no clear answer, andthe registration and segmentation should be carriedout simultaneously, each helping the other. How-ever, note that the primary goal of the proposedframework is to robustly and accurately estimatethe registration parameters.

Thus, we propose an iterative framework inwhich the segmentation and registration of twoimages are estimated in a two-stage algorithm us-ing two different entropies, termedminimax en-tropy. In the entropy maximization step, the seg-mentation of the portal image is estimated using thecurrent (or initialized) estimates of the registrationparameters. In the entropy minimization step, theregistration parameters are estimated based on thecurrent estimates of the segmentation. The algo-rithm can start at either step, with some appropriateinitialization on the other.

In our proposed image registration frame-work, we seek to segment portal images into tworegions—the bone and the background—and theimage registration is therefore guided by the bonyanatomy visible in the images. However, the pros-tate can move with respect to the pelvic bone,depending on whether the bladder is full or empty.Also, over the course of 5–6 weeks of irradiation,the prostate can shrink. To provide full conformalradiotherapy, therefore, it is necessary to accuratelylocalize the prostate. While we do not address thisissue directly in the current paper, in our futurework we plan to use our approach to aid in regis-tering ultrasound images with 3D CT images inorder to finally localize the prostate.

PROBLEM DEFINITION

We propose that patient setup verification is a clas-sical pose estimation problem in computer vision.In the pose estimation problem, the pose of a 3Dmodel is to be determined from a set of 2D imagessuch that the model is aligned with the images. Weassume that a rigid transformation of the 3D CTdataset will bring it into alignment with the portalimage, and hence there are only six transformationparameters to be considered. Since our algorithmworks directly on the pixel intensity values, weclassify it as an automated, intrinsic, global, dense-field-based (although features in the form of labels

288 Bansal et al.: A Minimax Entropy Registration Framework

are also used) algorithm, based on the classificationof Van den Elsen and Viergever.27

In the next subsection, we first formulate thepose estimation as a maximum-likelihood estima-tion (MLE) problem. This formulation requiresknowledge of the joint density functionp(xi, yi (T))between the portal image intensities and the inten-sities of a projection through the 3D CT datasetknown as a digitally reconstructed radiograph(DRR). As noted above, we want to estimate asegmentation of the portal image in order to help inthe registration process. The problem of segment-ing the portal image is formulated as a labelingproblem in which each pixel is labeled either asbone or background. To incorporate the segmenta-tion information, the joint density function is writ-ten as a mixture density where the labels on theportal image pixels are not known. Hence, thesegmentation labels are treated as the missing in-formation, and the parameters to be determined arethe pose parameters. We formulate our problem asa maximum-a-posteriori(MAP) estimate of thetransformation parameters. However, this problemis reduced to a maximum-likelihood estimation(MLE) problem by assuming uniform priors on thetransformation parameters. One approach for com-puting the MLE from incomplete data is presentedby Dempster et al.,6 and is termed the expectation-maximization (EM) algorithm. Our first thoughtswere to select the EM algorithm to compute theestimates, due to its proven monotonic convergenceproperties, ease of programming, and—unlikeother optimization techniques—the lack of need toset the step-size parameter. Thus, the EM algorithmcould have been used to estimate both the poseparameters and the segmentation labels. However,for our problem, the EM approach has severalrestrictions, as noted below. These led us to pro-pose our new minimax entropy strategy, which isdescribed later.

Mathematical SetupLet X 5 { x(i)} for i 5 1, . . . ,N2 denote theN 3N random field from which the portal images aresampled, wherex(i) is the random variable denot-ing the pixel intensity at theith pixel in the portalimage. LetG 5 { g(i)} for i 5 1, . . . ,N3 denote therandom field from which 3D CT images are sam-pled. LetY(T) 5 { y(i,T)} for i 5 1, . . . ,N2 denotetheN 3 N random field from which the projectionsfrom the 3D CT dataset are sampled, at a given setof transformation parametersT 5 T. The projected3D CT images are also called the digitally recon-structed radiographs (DRRs). Note that here we

index pixels in the images using a single index,even though the images are 2D (or 3D) images. Forsimplicity in the formulations below, we have as-sumed that the portal and projected 3D CT imagesare of the same dimensions. However, this need notbe true. We will assume that the pixels, for all therandom fields, are independently distributed. Thus,the probability density function of the random fieldX can be written in the factored form aspX~X!5 PipX i~xi!. Each pixel in the portal image isclassified as belonging to one of two classes—boneor background—and the pixels in each class areassumed to be identically distributed according tothe probability density function of that class. Wecan determine the random field from which theportal images are sampled. Thus, we formulate thepose estimation problem as follows:

T 5 arg maxT

ln p~TuX, G!

5 arg maxT

ln Sp~X, GuT! p~T!

p~X, G! D5 arg max

T

ln p~X, GuT!

5 arg maxT

ln ~ p~XuG, T! p~GuT!!

5 arg maxT

ln p~XuY~T!!

5 arg maxT

@ln p~X, Y~T!! 2 ln p~Y~T!!#

5 arg maxT

Oi

@ln p~ xi, yi~T!! 2 ln p~ yi~T!!#

(1)

where we assume a uniform prior on the transfor-mation parameters and ignore the termp~GuT!,since the 3D CT dataset,G, is independent of thetransformation parameters,T. In Equation (1) weassume that the image pixels are independent. Thelogarithm of the likelihood function is taken tosimplify the mathematical formulation. Note that,for notational simplicity, we shall now writex(i) asxi andy(i,T) asyi, where the current transformationparameters for which the DRR is obtained shouldbe clear from the context; otherwise it will be statedexplicitly.

Segmentation information is incorporatedinto the problem by considering the joint densityfunction p( xi, yi) as a mixture density. To set upthe joint density function in the mixture density

Bansal et al.: A Minimax Entropy Registration Framework 289

form, letA 5 {1, 2} be the set of classes (i.e., bone,background). Since X-rays pass through both boneand tissue before hitting the portal image, clarifi-cation of the classesbone, backgroundis neces-sary. In our classification of the portal image pixels,we seek to classify asbone those pixels in theportal image for which the X-rays passed throughat least some bone tissue. Other pixels we seek tolabel asbackground. Let M 5 {m(i)} for i 5 1, . . . ,N2 denote theN 3 N random field on the segmen-tation of the portal images. Using a mixture densitymodel, we can now write the joint mixture densitymodel, from Equation (1), for the portal image andDRR as, at pixeli,

p~ xi, yi! 5 Oa[A

P~m i 5 a! p~ xi, yium i 5 a!

5 Oa[A

P~m i 5 a! pa~ xi, yi!

5 Oa[A

Pi~a! pa~ xi, yi!

wheremi is the random variable denoting label atthe ith pixel, and pa(xi, yi) is the joint densityfunction of the pixel intensities, given that theithportal image pixel is labeleda. To simplifythe notational complexity, we will abbreviatepxiyi~xi,yiumi 5 a! to pa(xi, yi), and similarly forother probability density functions (PDFs), wherethe random variables in the abbreviated formshould be clear from the context.

As noted above, the problem of maximum-likelihood estimation of the transformation param-eters, as formulated, is ideally suited for the EMframework. However, for our application, the EMframework has two major restrictions. First, in theEM framework, the prior probabilities on eachpixel, Pi(a), are required to be known. If theseprobabilities are not known, then they can also beestimated within the EM framework, assuming thatthe segmentation labels on each pixel are indepen-dently and identically distributed. This is definitelynot the case for our problem. Second, the EMalgorithm requires that the form of the componentdensities,pa(xi, yi), are known (i.e., one shouldknow whether they are Gaussian, Rayleigh, expo-nential, etc.). For multi-modal image registration itis difficult, if not impossible, to knowa priori thejoint density function between the pixel intensitiesin the two images. The proposed minimax entropyframework tries to address these issues by evaluat-ing appropriate entropies to estimate the segmen-

tation of the portal image and the registration pa-rameters.

MINIMAX ENTROPY APPROACHThe proposed minimax algorithm for solving thebasic problem posed by Equation (1), in a compu-tational form similar to EM, has two steps: themaxstep and themin step. These are evaluated itera-tively to determine the registration parameters andthe probability distribution of the portal image seg-mentation. The details of the development of theproposed algorithm are discussed in Appendix A.Figure 1 summarizes the algorithm, highlightingthe key points and the iterative nature of the algo-rithm. Note that for multiple portal image registra-tion to a single 3D CT dataset, themaxstep must beperformed for each portal image. The derivation ofthemaxstep in Appendix A1 bridges the notationsin this and the previous section.

The max step is formulated as follows (Ap-pendix A1 sketches the outline of formulation):

Max Step:

Pk~M! 5 arg maxP~M!

F2OM

P~M! ln P~M!

1 OM

P~M! ln P~MuX, Y~T~k21!!!G (2)

under the constraintSM P(M 5 M) 5 1, whereMis the random variable whose domain is the set ofpossible segmentations of the portal image, whereeach pixel can be labeled from the set of labelsA.We assume that pixel labels,mi, are statisticallyindependent, i.e.,P(M 5 M) 5 Pi, P(mi 5 a) 5 Pi

Pi(a). As formulated above, themax step simplystates that the maximum entropy estimate of theprobability P(M 5 M) is the posterior probabilityon the segmentation of the portal image, i.e.,P(MuX, Y; T(k21)), given the current estimate of thetransformation parameters,T(k21), the portal image,X, and the DRR,Y, by minimizing the Kullback-Leibler (KL) divergence between the two probabil-ity distributions.4 This simple formulation of theestimated probability of a segmentation of the por-tal image allows us to systematically put con-straints on the segmentation probability function, asshown below. Thus, the analytical solution toEquation (2) estimates the probability of a segmen-tation label at theith pixel to be:

Pik~a! 5 S Pi

k21~a! pak21~ xi, yi!

¥b[A Pik21~b! pb

k21~ xi, yi!D


where the component density functions,pak21~xi,yi!,

are estimated in the next step.Note that thePi

k~a!’s, in thekth iteration, formthe weighing terms in the Parzen window esti-mates, in Equation (5) below, of the componentdensity functions,pa(x, y). The component densityfunctions, in turn, are used to estimate the jointentropies,Ha~x,y! 5 2**pa~x,y! ln pa~x,y! dx dy,which are minimized in theminstep to estimate theregistration parameters.

In order to better incorporate subjective in-formation into the problem, an annealing sched-ule18 is imposed on the estimated probability of asegmentation of the portal image pixel. The modi-fied maxstep, Equation (2), can thus be written as:

Modified Max Step:

Pk~M! 5 arg maxP~M!

F21

bOM

P~M! ln P~M!

1 OM

P~M! ln P~MuX, Y~T~k21!!!G (3)

under the constraintSM P(M 5 M) 5 1, whereb 51t, and t is the temperature, which determines the

annealing schedule. Appendix A2 develops theclosed form solution to the optimal probability dis-tribution Pk(M) in Equation (3). The annealingschedule, as formulated above, can also be under-stood in terms of estimation of the prior probabilityof the segmentation of the portal image (prior in-formation should reflect our state of knowledgeabout the random variables17). Initially, for smallvalues ofb, irrespective of the conditional proba-bility estimate, the above formulation incorporatesour knowledge of total ignorance about the pixellabels. This is essential as our initial estimate of thetransformation parameters is expected to be poorand thus lead to a poor estimate of the conditionaldensity and the component density functions esti-mated from the given dataset. However, as ourconfidence in the transformation parameters im-proves, the estimated component density functionswill be close to true densities. Now the pixels canbe classified into the two classes using themini-mum-error-rateclassification,8 (which is actuallythe Bayes decision rulefor the zero-one loss func-tion).

Thus, the annealing schedule, as proposedabove, by using a simple term1

b, allows us to

incorporate, analytically and systematically, the

Fig. 1. The Minimax Entropy Algorithm


above-discussed subjective information into the es-timation of the probability of a segmentation of aportal image. Note also that the current estimate ofthe posterior probability is used in themaxstep inorder to update our knowledge on the segmentationof the portal image. This updated knowledge of theportal image segmentation is then used as the priorinformation in the nextmaxstep for the estimationof the posterior probability. In summary, we areupdating our information on the segmentation ofthe portal image using the only objective informa-tion we have,P~MuX,Y;T~k21!!. The subjective in-formation about thegoodnessof this posteriorprobability is incorporated into the maximum en-tropy framework using the1

bterm. Given this sub-

jective and objective information about the seg-mentation of the portal image, the maximumentropy estimate of the probability is the only un-biased estimate of our state of knowledge17 aboutthe segmentation of the portal image.

The estimated probability,P(M), is the prob-ability distribution for the ensemble of the portalimages obtained from the patient in a particularposition. The reason we can estimate such a distri-bution from a single portal image, rather than re-quiring a whole ensemble of images, is because weassume that the intensities of the pixels belongingto the class are distributed according to the sameprobability density function; that is, we are assum-ing a stationary random field. In other words, thisassumes that the intensities of the pixels of a classin a portal image are representative of the intensi-ties at a particular pixel in the ensemble.

The min step is developed to be as follows(see Appendix A2 for details):

Min Step:

Tk 5 arg minT

3 S Oa[A

S 1

N2 Oi51

N2

Pik~a!DHa~ x, y! 2 H~ y!D (4)

See Reference 1 for the development of themin step. The component density function for classa, pa(x, y), is estimated as the weighted sum

of Gaussian kernels,Gc~x! 5 ~2p!2n

2 ucu212 exp

~212xTc21x!, using the Parzen window method as

follows:

pak~ x, y! <

1

¥~ xi,yi![IPik~a!

3 O~ xi,yi![I

Pik~a!GCa~ x 2 xi, y 2 yi! (5)

wherePik 5 Pk~mi 5 a! is the probability that the

ith pixel in the portal image belongs to classa,estimated in themaxstep, Equation (2), andCa isa 23 2 covariance matrix, which is assumed to bediagonal (note that this assumption does not meanthat the random variablesx andy are independent).I andJ denote sets of sizesNI andNJ, respectively,of pixels sampled at random from the portal image,X, and the DRR,Y. The joint entropy functions,which are the expected value of the log of the jointprobability density functions, are approximated asthe statistical expectations using the Parzen win-dow density estimates as follows:

Ha~x, y! 5 2EE pa~x, y! ln pa~x, y! dx dy

< S 21

¥~ xj,yj![JPjk~a!D O

~ xj,yj![J

Pjk~a!

3 ln S 1

¥~ xi,yi![IPik~a!

O~ xi,yi![I

Pik~a!GCa~xj 2 xi, yj 2 yi!D

(6)

Similarly, the entropy of the DRRs,H(y), isestimated as:

H~ y! 5 2E p~ y! ln p~ y! d y

< S21

NJ D Oyj[J

ln S 1

NIOyi[I

GC~ yj 2 yi!DAppendix B provides a coordinate descent

interpretation of the proposed minimax entropyregistration framework. Since the coordinate de-scent strategies tend to converge to a local opti-mum, the minimax entropy algorithm will, intu-itively, also converge to a local optimum.

It is easy to see the relation between theminstep and the mutual information (MI) measure,currently popular in the medical image analysiscommunity. We have found it to be more robustthan other interesting measures (e.g., correlation.See Reference 2, where we present the relationship


between the two measures and show that MI ismore sensitive to mis-registration than correlation).Because MI, as implemented in the medical imageprocessing literature, assumes that pixels are inde-pendent and identically distributed (i.i.d.), this isgenerally a problem, which we get around by usingmixture densities. We note that Studholme et al.26

register images with mutual information as a matchmeasure while incorporating segmentation infor-mation on one of the images. However, the imagewas pre-hand-segmented, and thus remains fixedthroughout the registration. In our proposed algo-rithm, the portal image segmentation is estimatedsimultaneously and iteratively with the transforma-tion parameter estimations.

Optimization strategyWe follow the optimization strategy of stochasticgradient descent,29 which is described below andadapted for the inclusion of segmentation informa-tion. From Equation (4), let the cost function bedenoted byF(T); that is,

F~T! 5 Oa[A

S 1

N2 Oi51

N2

Pik~a!D Ha~ x, y! 2 H~ y! (7)

which is minimized to estimate the transformationparameters, given the current estimates of the seg-mentation of the portal image. The transformationparameters are updated according to the followingrule, until a pre-specified number of iterations:

I 4 Set of pixels, of sizeNI,

drawn randomly fromX, Y

J 4 Set of pixels, of sizeNJ,

drawn randomly fromX, Y

T 4 T 1 ld

dTF~T! (8)

wherel is called thelearning factor. Note againthat the randomly sampled sets of pixels,I andJ,are used to estimate both the Parzen window esti-mates of the component density functions (Equa-tion (5)) and the joint entropy terms (Equation (6)).The parameterl needs to be set empirically. In oursimulations,l was set to 0.02. The parameter up-date strategy, Equation (8), is analogous to thegradient descent optimization strategy, where thestochastic nature of the updates arises from the fact

that the cost function,F(T), is only a stochasticapproximation of the true energy function. It isexpected that the stochastic nature of the parameterupdate may help the algorithm escape a local opti-mum and help converge to the global optimum,though there is no general proof.

Estimating covariance parametersThe covariance matrices of the Gaussian kernels inthe Parzen window estimates of the joint densityfunctions are assumed to be diagonal. However,note that this does not mean the marginal densityfunctions of the pixel intensities of the two imagesare independent. In the Parzen window estimates ofthe probability density functions, the same Gauss-ian kernel is used for all the sampled pixel inten-sities. Thus, only two parameters need to be esti-mated for each joint density. The covarianceparameters are estimated by minimizing the empir-ical entropy using the stochastic gradient descentmethod.

RESULTSIn this section we evaluate the accuracy and robust-ness of the proposed minimax algorithm using bothreal and simulated data. A Plexiglas pelvic bonephantom is scanned to provide the 3D CT dataset(the phantom consists of real human pelvic boneencased in Plexiglas of density close to the densityof soft tissue). The phantom is then moved to theradiotherapy treatment room to obtain real portalimages using treatment energy X-rays (6 MV). Thesimulated portal images are obtained in the follow-ing fashion. First, the 3D CT voxel values aremapped from diagnostic energy values to the val-ues at the treatment energy X-rays using the atten-uation coefficient tables.16 Second, the 3D CT data-set is transformed by known transformationparameters. Third, the digitally reconstructed radio-graphs (DRRs) are rendered from the CT dataset,both in the anterior-posterior (AP) and the left-lateral (LL) directions. Two different sets of simu-lated portal images are then obtained from theresulting DRRs. To obtain the first set of simulatedportal images, varying amounts of i.i.d. Gaussiannoise are added to the DRRs. To obtain the secondset of simulated portal images, the DRRs areblurred using blurring kernels of increasing width,which simulates the finite size of the radiationsource and the low contrast and low sharpness ofthe real portal images. The 3D CT dataset is thenset to its undeformed position and the algorithm isexecuted to estimate the transformation parameters.Note again that this is a 2D/3D registration prob-


lem, as the 3D CT dataset is being registered to the2D portal images for the estimation of the registra-tion parameters. Since the true registration param-eters are known for the simulated portal images,these datasets are used to study the accuracy androbustness of the algorithm under increasing noiseand blur in the images. The perspective parametersare assumed to be known.

Figure 2(a) shows the surface rendering of thepelvic bone of the phantom dataset, while Figure2(b) shows the real portal image obtained by im-aging the phantom with 6-MV X-rays. The realportal image was contrast-enhanced by histogramequalization.

As the results below suggest, the proposedalgorithm is not robust to the estimation of the

out-of-plane transformation parameters when usingonly a single AP portal image. Thus, the proposedalgorithm has been extended to utilize dual portalimages by using two forms of Equation (2) for eachportal segmentation and a summed version ofEquation (4) for both portal images. Note that thetwo portal images can be obtained from two differ-ent views which are not necessarily orthogonal. InFigure 3 we show the setup of the pelvic phantomand the AP and LL portal images in the 3D space.As is clear from the figure, the in-plane translationsfor the AP portal image consist of translationsalong the X and Y axes, and the in-plane rotation isthe rotation about the Z axis. For the lateral portalimage, the in-plane translations are the translationsof the 3D CT dataset along the Y and Z axes, andthe in-plane rotation is the rotation about the Xaxis. Note that by using two portal images, only therotation about the Y axis now remains as the in-plane transformation for both the images.

Three-dimensional analysis of the patientsetup was performed using both a single AP portalimage and dual AP and LL portal images. The 3Danalysis leads to determination of 3 translations and3 rotations which will correctly align the patient tothe treatment beam in the treatment room. How-ever, as mentioned in the introduction, patient setupverification in many radiotherapy departments isstill carried out by aligning the 2D simulator im-ages to the 2D portal images, and such an analysiscan estimate only the in-plane transformation pa-

Fig. 2. (a) Volume-rendered 3D CT phantom for whichthe simulated portal images were calculated. (b) Real portalimage of the phantom obtained by taking 6MV X-ray of thephantom in the treatment room.

Fig. 3. The setup of the radiation source, 3D CT data, and the simulated portal image.


rameters. Thus, we have also modified our regis-tration algorithm to register 2D simulator images to2D portal images and have tested it out in thisenvironment also.

2D/2D RegistrationAs noted above, for 2D analysis of the patient, the2D simulator image is registered to the 2D portalimage. Figure 4(a) shows a 2D simulator image.The simulated portal image in Figure 4(b) is ob-tained by first transforming the simulator imageand then blurring it with a uniform blurring kernelof width 11. The first row of the table in Figure 4shows the true transformation parameters, wheretxandty denote, in pixels, the translations along the Xand Y axis, respectively.u is the in-plane rotation,in degrees. The second row of the table shows theparameters as estimated by the proposed algorithm.Note that even though the portal image was blurredby 11 pixels, the algorithm was able to estimate theX translation within 2 pixels and the Y translationwithin 0.25 pixels. The rotation angle was alsoestimated very accurately. The segmentation of theportal image as estimated by the proposed algo-rithm is shown in Figure 4(c), showing an intu-itively plausible result.

3D/2D RegistrationThree-dimensional analysis of the patient setup isdone using both single (AP) and dual (AP and LL)portal images. We first present results using the realportal image and then results using the simulatedportal images. The simulated portal images areobtained as explained above. The proposed regis-tration algorithm estimates six transformation pa-rameters, where the three translations, in voxels,

are denoted astx, ty, and tz, (along the X axis, Yaxis, and Z axis, respectively) and the three rota-tions, in degrees, are denoted asuYZ, uXZ, anduXY

(about the X axis, Y axis, and Z axis, respectively).

3D analysis using a single portal imageIn this subsection, the results of registering a realAP portal image of the pelvic phantom to its 3D CTdataset are presented. To study the effect of noiseon the accuracy of the estimated parameters, wethen register simulated AP portal images, with in-creasing noise, to the 3D CT dataset.

Figure 2(b) shows the real portal image whichis registered to the 3D CT dataset. The parametersestimated by the proposed algorithm, to bring thetwo datasets into alignment, are shown in the tablein Figure 5. Figure 5(b) shows the DRR obtainedfrom the 3D CT dataset at the parameters estimated

Fig. 4. (a) Simulator image. (b) Simulated portal image. The simulator image was rotated and translated, and then blurredusing a low pass filter of width 11. (c) Segmentation of the portal image estimated by the proposed algorithm.

Fig. 5. (a) Portal image with hand-drawn contours. (b)DRR with mapped contours.


by the minimax entropy algorithm. To verify thegoodness of the estimated parameters, several con-tours are hand-drawn on the portal image (Fig.5(a)) by an expert, matching closely to key fea-tures. These undeformed contours are then mappedonto the DRR (Fig. 5(b)). Note that the contoursmatch closely to the corresponding features in theDRR, verifying the accuracy of the estimated pa-rameters for this dataset.

Different segmentations of the portal imageare shown in Figure 6. Bright pixels denote esti-mates of bone and dark pixels denote background.Figure 6(a) shows the segmentation estimated bythe minimax entropy algorithm, where the segmen-tation labels are being estimated using the jointprobability density function. Gray values denotethe pixels whose labels could not be determined.The estimated segmentation is compared to twoother segmentations: a simple threshold-based seg-mentation (Fig. 6(b)) and a segmentation obtainedby a clustering algorithm in a commercial program(MEDx25) (Fig. 6(c)). In this clustering algorithm,the image is segmented into three classes based onthe nearness of the gray values to the user-specifiedvalues.

As noted earlier, to study the robustness ofthe algorithm against noise, simulated portal im-

ages with increasing i.i.d. Gaussian noise are reg-istered to the 3D CT dataset. Examples of simu-lated portal images with different amounts of noiseare shown in Figures 7(a) and (b). The image inFigure 7(a) is obtained by first rotating the 3D CTdataset by 5° along the Z axis and then addingGaussian noise of standard deviation (s) 10.0 to therendered DRR. Gaussian noise ofs 5 30.0 wasadded to obtain the image in Figure 7(b). Figure7(c) shows the graph of estimated parameters withincreasing noise in the simulated portal image.Simulated portal images for the graph labeledtx areobtained from the 3D CT in the following fashion.The 3D CT dataset is translated along the X axis by5 voxels and then a DRR is rendered. Varyingamounts of Gaussian noise are then added to theDRR. The difference between the estimated andtrue parameters is plotted along the Y coordinate inFigure 7(c), with noise standard deviation along theX coordinate. The results can be summarized asfollows. The minimax entropy algorithm is quiterobust against noise for in-plane translations of the3D CT dataset, i.e., translation along the X and Yaxes. However, for both in-plane and out-of-planerotations, the estimation of the parameters becomespoor with increasing noise. Thus, we have extendedour algorithm to use dual portal images, obtainedfrom two different views, to improve the accuracyand robustness of the algorithm.

3D analysis using dual portal images

Based on the conclusions from the previous sec-tion, the algorithm has been further developed toutilize dual portal images simultaneously to esti-mate the transformation parameters. In this section,dual simulated portal images in the AP and LLdirections are used to study the accuracy and ro-bustness of the algorithm.

Figure 8 shows the simulated AP and LL

Fig. 6. Estimated segmentation results of the true portalimage. (a) Results of using the proposed minimax entropyalgorithm. (b) Simple threshold. (c) Clustering algorithm incommercial package (MEDx25) for comparison.

Fig. 7. (a) Simulated portal image with Gaussian noise of std (s) 10. (b) s 5 30. (c) Error in estimated parameter withincreasing noise. The dynamic range of pixel intensities is 255.


DRRs registered to the 3D CT dataset. The first rowof the table in the figure shows the true registrationparameters and the second row shows the parame-ters estimated by the algorithm. Note that the esti-mated parameters are within half a voxel of the trueparameters. The rotations have also been accuratelyestimated. The estimated segmentations of theDRRs are shown in Figure 8(c) and (d).

The simulated portal images are blurred byusing a uniform blurring kernel of width 11 toobtain the portal images in Figure 9(a) and (b).Figure 9(c) and (d) show the corresponding seg-mentation of the portal images estimated by thealgorithm. Again, the table in the figure shows thetrue and estimated parameters. Note that the esti-mated translations are within 1 voxel of the truevalues, even in the presence of a blur of 11 pixels.The estimates of the rotation parameters are within0.5° of the true values.

Figure 10(a) and (b) show the simulated por-tal images with Gaussian noise ofs 5 30.0. Note

again the accuracy of the parameters estimated bythe algorithm. Figure 10(c) and (d) show the seg-mentation of the portal images as estimated by thealgorithm.

Figure 11(c) and (d) show the graphs of errorin the estimated parameters when the algorithm isinitialized with varying amounts of rotational andtranslational setup variations, in the presence ofGaussian noise ofs 5 20.0 in the simulated portalimages. These graphs are obtained in the followingfashion. First, the 3D CT dataset is transformed bya known amount and AP and LL DRRs are ob-tained. Then i.i.d. Gaussian noise ofs 5 20.0 isadded to the DRRs to obtain the simulated portalimages. Example portal images used for thesegraphs are shown in Figure 11(a) and (b). For thegraph labeleduYZ, only the parameteruYZ, whichdenotes rotation about the X axis, is varied toobtain the DRRs. The error in the estimated param-eter as compared to the true parameters is thenplotted. The graphs show that, for this dataset, the

Fig. 8. (a) Simulated AP DRR. (b) Simulated left-lateral DRR. (c) Segmentation of the AP portal image estimated by theproposed algorithm. (d) Estimated segmentation of the LL portal image. The second row of the table shows the parametersestimated by the minimax entropy algorithm.

Fig. 9. (a) Simulated AP portal image. (b) Simulated left-lateral portal image. (c) Estimated segmentation of the AP portalimage. (d) Estimated segmentation of the LL portal image. Estimated and the true parameters are shown in the table.


algorithm could accurately estimate rotation anglesup to 50°. For translations, the estimates for thethree translations were accurate up to very differentlimits. These figures also show that either the al-gorithm is quite accurate in estimating the param-eters or it breaks down completely; that is, theestimated parameters are totally different from the

true parameters. This shows that the algorithm getstrapped in a local minimum if the global minimumis very far from the initial starting position. Forlarge translations, very little pelvic structure re-mains in the image, making it easier for the algo-rithm to get stuck into a local minimum. Graphs inFigure 11(e) and (f) show the performance of the

Fig. 10. Simulated portal images with noise. (a) AP with standard deviation 30.0. (b) Left-lateral with std 30.0. Estimatedsegmentation of (c) AP portal image. (d) LL portal image. The table show the true and the parameters estimated by thealgorithm.

Fig. 11. (a), (b) Example simulated portal images with Gaussian noise of 20.0. (c) Graph of estimated versus true rotationangles. (d) Graphs of estimated translation versus true translation. (e) Error in estimated rotation angles with increasing noise.(f) Error in estimated translation with increasing noise.


algorithm under increasing noise. The AP and LLportal images, e.g., for the graph labeleduXY, areobtained by first rotating the 3D CT data by 15°about the Z axis and then rendering the DRRs inboth the AP and the LL directions. Varyingamounts of noise are then added to the DRRs toobtain the simulated portal images. The 3D CTdataset is then initialized to its undeformed positionand the algorithm is run to estimate the transfor-mation parameters. The graph shows the error inestimated transformation parameters for variousamounts of noise. Similarly, for the graphs labeleduYZ, uXY, tx, ty, tz, the 3D CT dataset was trans-formed by 30°, 25°, 20 voxels, 20 voxels, and 15voxels, respectively, to obtain the DRRs in the APand the LL directions.

Performance on Real DataFigure 12 shows the results of running the proposedalgorithm on real patient data. Figure 12(a) and (b)show histogram equalized AP and LL portal im-ages, respectively. The DRRs projected through the3D CT data in its original pose are shown in Figure12(c) and (d). Running the algorithm estimates anew pose of the 3D CT dataset, which differs fromthe original pose byuXY5 23.2°,uYZ5 2.93°,uXZ

5 21.93°, tx 5 4.47 voxels,ty 5 227.5 voxels,and tx 5 214.54 voxels. The DRR projections inthe new pose are shown in Figure 12(e) and (f).Segmentations of the AP and LL portal images,estimated by the algorithm, are shown in Figure12(g) and (h), respectively. Note that the segmen-

tation of the portal images are being used implicitlyby the algorithm. Because of the poor quality ofthese digitized portal film images, the segmentationstep was initialized manually in several regionswhere the soft tissue and bone were hard to distin-guish. To assess the accuracy of the estimated poseof the 3D CT dataset, contours are hand-drawn onthe portal images, matching visible features. Thesecontours are then mapped onto the DRRs, in Figure12(c–f), undeformed. Note that the contours areused only to visually assess the goodness of theestimated pose. They match closely to the featuresin DRRs obtained at the pose estimated by thealgorithm.

The algorithm as implemented took about15–20 minutes on the SGI Indigo R1000 to esti-mate registration parameters using dual portal im-ages.

DISCUSSION AND FUTURE WORKIn this paper we have presented an informationtheoretic framework in which segmentation andregistration are carried out simultaneously and it-eratively, with segmentation results helping in theregistration and vice versa. Feature-based registra-tion methods proposed in the literature carry outportal image segmentation as a pre-processing stepin the registration process. Our approach of simul-taneously segmenting and registering the images,using a unified framework, leads to an accurate androbust algorithm.

The mutual information match measure

Fig. 12. Recovery of setup variation using actual patient data and an early version of the proposed registration algorithm.(a,b) Digitized portal images, (c,d) DRRs of 3D CT in original pose, (e,f) DRRs in corrected pose, (g,h) implicit segmentationof portals. See more detailed explanation in the text.


overcomes the assumption of a linear relation-ship between the pixel intensities of the imagesto be registered, an underlying assumption in thenormalized cross-correlation match method. Inthe mutual information-based registration, therelationship between the pixel intensities is esti-mated from the given data itself, and thus canregister images from different modalities. At anestimated set of transformation parameters, ajoint density between the images to be registeredcan be estimated from the given data. The mutualinformation measure assigns a number to eachsuch estimated density. The transformation pa-rameters corresponding to the density having thelargest mutual information are chosen as the pa-rameters estimated by the algorithm. The EMalgorithm provides an iterative framework to es-timate the parameters of a distribution in thepresence of missing data, but the algorithm re-quires that the parametric form of the variouscomponent density functions be known.

The proposed minimax entropy algorithmovercomes this restriction of the EM algorithm byborrowing the idea from the mutual informationmethod of estimating the joint distribution from thegiven data, and emphasizes the fact that the distri-bution on the segmentation labels is the maximumentropy distribution satisfying the given con-straints. This fact allows us to invoke other con-straints on the distribution systematically, whichwe used to impose an annealing schedule. In ourfuture work, we will be imposing a regularizationconstraint on the segmentation within the sameframework.

We studied the robustness of the algorithmagainst noise by increasingly adding Gaussian i.i.d.noise to the simulated portal image. In the initialiterations of the algorithm with no segmentationinformation on the portal image, the proposed al-gorithm basically searches for the transformationparameters that increase the mutual informationmeasure. During later iterations, with estimates onthe segmentation labels, the algorithm maximizesthe mutual information in the separate regions es-timated in the entropy maximization step.

Our experiments showed that, by using onlyone portal image, estimated registration parameterswere not accurate for the out-of-plane rotations andtranslations. We therefore extended our algorithmto enable the use of dual portal images (which neednot be orthogonal). The results of the algorithm arevery encouraging for the simulated dataset. Thealgorithm was also extended to register a 3D sim-ulator image to a 2D portal image. For the 2D

registration experiments, the algorithm leads tovery accurate registration and reasonably good seg-mentation.

Note that, in our proposed method, the pa-tient setup verification was achieved largely byusing the bony anatomy visible in the portalimages and the 3D CT dataset. However, for theproblem of radiotherapy treatment of prostatecancer, the real goal is to identify and localizesoft tissue position. The prostate position mayvary on a day-to-day basis relative to the bonyanatomy, depending on bladder and rectum dis-tension. Thus, there are inherent errors in usingthe bony anatomy to localize the prostate and thegains in patient positioning accuracy may benegated by the fundamental uncertainty associ-ated with using bony anatomy to direct irradia-tion of a mobile soft-tissue organ. Thus, there isa need to incorporate organ motion compensationinto prostate cancer radiotherapy. In future work,we hope to integrate intra-treatment ultrasoundfor measuring prostate motion.

Our future research also includes measures tospeed up the algorithm, and validation of the accu-racy and robustness of the algorithm, especially incomparison to mutual information-based registra-tion and the ridge-based algorithm proposed byGilhuijs.11 We also feel that inclusion of the edgesand the whole boundary information will likelylead to more accurate results, and this extension isalso planned.

APPENDICES

A. DERIVATIONS OF THE MINIMAXENTROPY STEPS

The maximum likelihood (ML) estimates of thetransformation parameters,T, are estimated ac-cording to Equation (1):

T 5 arg maxT

Oi

@ln p~ xi, yi~T!! 2 ln p~ yi~T!!#

The expectation-maximization (EM) algo-rithm6 can be used to estimate the optimal MLparameters. The EM steps to estimateT can beshown to be:

E-Step:

Q~T, T~k21!!

5 OiF O

a[A

^zai&k ln pa~ xi, yi! 2 ln p~ yi!G ~9!


M-Step:

Tk 5 arg maxT

Q~T, T~k21!! (10)

where

^zai&k 5 S ^zai&

k21pa~ xi, yi!

¥b[A ^zbi&k21pb~ xi, yi!

D (11)

whereyi 5 y(i, T(k21)). Note thatyi 5 y~i,T~k21!!can be easily shown to be the conditional probabil-ity, given the dataset and the current estimate of thetransformation parameters, that theith pixel be-longs to classa, that is,P~mi 5 auxi,yi!.

A1. Derivation of the max step

Since each pixel is assumed to be independentlydistributed, the posteriori distribution of a segmen-tation,M , of a portal image, givenX andY, can bewritten as:

P~MuX, Y~Tk21!! 5 PiP~m i 5 auxi, yi!

However, we need to estimateP(M), theprobability distribution of the segmentation of aportal image. We expect thatP(M) is close toP~MuX,Y~Tk21!! and satisfies any other constraints.We invoke theprinciple of maximum entropytoestimateP(M) which is closestto P~MuX,Y~Tk21!!and has maximum entropy under given constraints.Thus, to estimateP(M), we maximize the functionh, as a function ofQ(z):

h~P~M!! 5 2OM

P~M! ln P~M!

1 Q~P~M!, P~MuX, Y~Tk21!!! (12)

with SM P(M) 5 1. The first term on the right-handside of Equation (12) is the entropy of the distri-bution P(M), and the second term ensures that thetwo distributions P(M) and P~MuX,Y~Tk21!! areclose. Following the formulation in Reference 4,the functionQ(z) can be determined so thath canbe written as:

h~P~M!! 5 2OM

P~M! ln P~M!

1 OM

P~M! ln P~MuX, Y~Tk21!! (13)

Sinceh(P(M)) is maximized under the con-straintSM P(M) 5 1 to estimateP(M), themaxstepis formulated to be:

Max Step:

Pk~M! 5 arg maxP~M!

F2OM

P~M! ln P~M!

1 OM

P~M! ln P~MuX, Y~Tk21!!Gunder the constraintSM P(M) 5 1.

To incorporate subjective information intothe problem, themaxstep is modified with the newfunction h9 being defined as:

h1~P~M!! 5 21

bOM

P~M! ln P~M!

1 OM

P~M! ln P~MuX, Y~Tk21!! (14)

We utilize principles of calculus of variations toestimateP(M) which maximizesh1(P(M)), withconstraintSM P(M) 5 1. We define a function:

h91~P~M!! 5 21

bOM

P~M! ln P~M!

1 OM

P~M! ln P~MuX, Y~Tk21!!

1 l S OM

P~M! 2 1Dwhere l is a Lagrangian multiplier which alsoneeds to be estimated. Thus,P(M) andl are esti-mated by solving the following set of equations:

dh91dP~M!

5 0 (15)

dh91dl

5 0 (16)

Equation (15) leads to

P~M! 5 ~lb 2 1! P~MuX, Y~Tk21!!b (17)

and Equation (16) yields


OM

P~M! 5 1 (18)

Equations (15) and (16) are used to estimatel,which is easily seen to be:

l 51

b F1 11

¥M P~MuX, Y!bGThus, the estimated probability distributionPk(M)can be written as:

Pk~M! 5P~MuX, Y!b

¥M P~MuX, Y!b (19)

As the image pixels are assumed to be indepen-dently distributed, Equation (19) can be written inthe factored form as:

Pk~M!

5 Pi

Pk21~m i 5 auxi, yi!b

¥b Pk21~m i 5 buxi, yi!b

5 Pi

@ pk21~ xi, yium i 5 a! Pk21~m i 5 a!#b

¥b @ pk21~ xi, yium i 5 b! Pk21~m i 5 b!#b

5 Pi

@ pak21~ xi, yi! Pi

k21~a!#b

¥b@ pbk21~ xi, yi! Pi

k21~b!#b

5 PiPk~m i 5 a!

Thus, the distributionPk(mi 5 a) can be evaluatedas:

Pk~m i 5 a! 5@ pa

k21~ xi, yi! Pik21~a!#b

¥b @ pbk21~ xi, yi! Pi

k21~b!#b (20)

A2. Derivation of the min stepThe EM algorithm, in theM-StepEquation (10),requires that the form of various component densityfunctions,pa(xi, yi), is known. As noted in the maintext of the paper, it is very difficult to knowa priorithe form of the variouspa(xi, yi), especially for themulti-modality image registrations. Thus, we pro-pose to estimate the density functions from thegiven dataset itself. However, since these densityfunctions cannot be directly used in theM-StepEquation (10), we propose to optimize the expectedvalue ofQ(T, T(k21)) with respect top(X, Y). Tak-

ing the expected value ofQ(T, T(k21)) with respectto the density functionp(X, Y), it is easy to showthat, using the Kullback-Leibler inequality,

E@Q~T, T~k21!!#

# 2F Oa[A

S 1

N2 Oi51

N2

Pik~a!DHa~ x, y! 2 H~ y!G

where the equality is satisfied exactly for the setvalues ofPi

k(a) 5 {0, 0.5, 1}, that is, during theinitial iterations (when the values are 0.5) and thefinal iterations (when the values are 0 or 1) of thealgorithm.

An upper bound on the expected value, whichturns out to be the joint conditional entropy2 H~M, XuY!, is chosen to be maximized (i.e.,H~M, XuY! is minimized) to estimateT.

Thus, themin step is defined to be:Min Step:

Tk 5 arg minT

H~M, XuY!

5 arg minT

F Oa[A

S 1

N2 Oi51

N2

Pik~a!DHa~ x, y! 2 H~ y!G

B. COORDINATE DESCENTINTERPRETATION

The minimax entropy algorithm above is developedwithin a probabilistic framework. However, withinthe optimization framework the algorithm can beviewed as a coordinate descent approach whichseeks to optimize a cost function by iterative esti-mation of the parameters along different coordi-nates. Let

F~P, T! 5 2H~M, XuY! 1 H~M!

5 EE dX dYOM

p~M, X, Y! ln p~M, XuY!

2 OM

P~M! ln P~M!

5 EE dX dYOM

p~X, YuM! P~M! ln p~M, XuY!

2 OM

P~M! ln P~M! (21)


Note thatF(P, T) is a functional (function offunction) which is to be optimized to estimate prob-ability, distribution P(M) and the parameters,T.Optimizing F(P, T) using the coordinate descentapproach leads to the following two steps:

Step 1:

Pk~M! 5 arg maxP

F~P, Tk21!

under the constraintSM Pk(M) 5 1.Step 2:

Tk 5 arg maxT

F~Pk, T!

Step 1, where the energy functionalF(P, T) isbeing optimized to estimateP(M), utilizing thetransformation parametersTk21, is equivalent tothe max step. Thus, estimation of the probabilitydistribution P(M), a variational calculus problemwithin the optimization framework, is interpretedas amaximum entropyestimation of a distributionwithin the probabilistic framework. Step 2, wherewe optimizeF(P, T) to estimateT, utilizing currentestimates ofP(M), is equivalent to themin step, asthe marginal entropy term,H(M), is independent ofthe parametersT. A coordinate descent optimiza-tion approach converges to a local optimum. Sincethe minimax entropy algorithm can be viewed as acoordinate descent optimization of the cost func-tion in Equation (21), this intuitively appeals toconvergence of the proposed registration frame-work to a local optimum.

REFERENCES1. Bansal R, Staib L, Chen Z, Rangarajan A, Knisely J,

Nath R, Duncan JS. A novel approach for the regis-tration of 2D portal and 3D CT images for treatmentsetup verification in radiotherapy. In: Wells WM,Colchester A, Delp S, editors. Medical image comput-ing and computer-assisted intervention (MICCAI’98).Proceedings of First International Conference, Cam-bridge, MA, October 1998. LNCS 1496. Berlin:Springer, 1998. p 1075–1086.

2. Bansal R, Staib L, et al. A novel approach for theregistration of 2D portal and 3D CT images for treat-ment setup verification in radiotherapy. Technical Re-port 1998-1, Department of Electrical Engineering andDiagnostic Radiology, Yale University, January 1998(http://noodle.med.yale.edu/;bansal/techreps/regis.ps.gz).

3. Berger M, Gerig G. Motion measurements in low-contrast X-ray imagery. In: Wells WM, Colchester A,Delp S, editors. Medical image computing and com-

puter-assisted intervention (MICCAI’98). Proceedingsof First International Conference, Cambridge, MA,October 1998. LNCS 1496. Berlin: Springer, 1998. p832–841.

4. Bevensee RM. Maximum entropy solutions to scien-tific problems. PTR Prentice Hall, 1993.

5. Bijhold J, Gilhuijs KG, van Herk M, Meertens H.Radiation field edge detection in portal images. PhysMed Biol 1991;36:1705–1710.

6. Dempster AP, Laird NM, Rubin DB. Maximum like-lihood from incomplete data via EM algorithm. JRoyal Statistical Soc Ser B 1977;39:1–38.

7. Dong L, Boyer AL. An image correlation procedurefor digitally reconstructed radiographs and electronicportal images. Int J Radiation Oncology Biol Phys1995;33:1053–1060.

8. Duda RO, Hart PE. Pattern Classification and SceneAnalysis. John Wiley & Sons, 1973.

9. Feldmar J, Malandain G, Ayache N, Ferna´ndez-VidalS, Maurincomme E, Trousset Y. Matching 3D MRangiography data and 2D X-ray angiograms. In: Troc-caz J, Grimson E, Mo¨sges R, editors. Proceedings ofFirst Joint Conference, Computer Vision, Virtual Re-ality and Robotics in Medicine and Medical Roboticsand Computer-Assisted Surgery (CVRMed-MR-CAS’97), Grenoble, France, March 1997. LNCS1205. Berlin: Springer, 1997. p 129–138.

10. Fritsch DS, Chaney EL, Boxwala A, McAuliffe MJ,Raghavan S, Thall A, Earnhart JR. Core-based portalimage registration for automatic radiotherapy treat-ment verification. Int J Radiation Oncology Biol Phys1995;33:1287–1300.

11. Gilhuijs K. Automated verification of radiation treat-ment geometry. Ph.D. thesis. University of Amster-dam, Radiotherapy Department, The Netherlands,1995.

12. Gilhuijs K, Drukker K, Touw A, van de Ven PJ, vanHerk M. Interactive three-dimensional inspection ofpatient setup in radiation therapy using digital portalimages and computed tomography data. Int J Radia-tion Oncology Biol Phys 1996;34:873–885.

13. Gueziec A, Kazanzides P, Williamson B, Taylor RH.Anatomy-based registration of CT-scan and intraop-erative X-ray images for guiding a surgical robot.IEEE Transactions on Medical Imaging 1998;17:715–728.

14. Hanley J, Mageras GS, Sun J, Kutcher GJ. The effectsof out-of-plane rotations on two dimensional portalimage registration in conformal radiotherapy of theprostate. Int J Radiation Oncology Biol Phys 1995;33:1331–1343.

15. Hiristov DH, Fallone BG. A gray-level image align-ment algorithm for registration of portal images anddigitally reconstructed radiographs. Med Phys 1996;23:75–84.

16. Hubble JH. Photon cross sections, attenuation coeffi-cients, and energy absorption coefficients from 10


KeV to 100 GeV. Nat Stand Ref Data Ser, Nat BurStand (US), August 1969.

17. Jaynes ET. Prior probabilities. In: Rosenkrantz RD,editor. ET Jaynes: Papers on probability, statistics andstatistical physics, Volume 158. Boston, MA: D Re-idel Publishing Company, 1983. p 114–130.

18. Kirkpatrick S, et al. Optimization by simulated anneal-ing. Science 1983;220:671–680.

19. Lavallee S, Szeliski R, Brunie L. Anatomy-based reg-istration of three-dimensional medical images, rangeimages, X-ray projections, and three-dimensionalmodels using octree-splines. In: Taylor RH, Lavalle´eS, Burdea GC, Mo¨sges R, editors: Computer-Inte-grated Surgery: Technology and Clinical Applications.Cambridge, MA: The MIT Press, 1996. p 115–143.

20. Lemieux L, Jagoe R, Fish DR, Kitchen ND, ThomasDG. A patient-to-computed-tomography image regis-tration method based on digitally reconstructed radio-graphs. Med Phys 1994;21:1749–1760.

21. Leszczynski KW, Loose S, Boyko S. An image reg-istration scheme applied to verification of radiationtherapy. Br J Radiology 1998;71:413–426.

22. Meertens H, Bijhold J, Strackee J. A method for themeasurement of field displacement errors in digitalportal images. Phys Med Biol 1990;35:299–323.

23. Mubata CD, Bidmead AM, Ellingham LM, ThompsonV, Dearnaley DP. Portal imaging protocol for radicaldose-escalated radiotherapy treatment of prostate can-cer. Int J Radiation Oncology Biol Phys 1998;40:221–231.

24. Neve WD, van den Heuvel F, Coghe M, Verellen D,De Beukeleer M, Roelstraete A, De Roover P, Thon L,Storme G. Interactive use of on-line portal imaging inpelvic radiation. Int J Radiation Oncology Biol Phys1993;25:517–524.

25. Sensor Systems, Inc. MEDx™: Multimodality Radio-logical Image Processing For Unix Workstations,1998. Version 2.1.

26. Studholme C, et al. Incorporating connected regionlabeling into automated image registration using mu-tual information. In: Proceedings of MMBIA’96,1996. p 23–31.

27. Van den Elsen PA, Viergever MA. Medical imagematching—a review with classification. IEEE Eng inMed and Biol 1993;12:26–39.

28. Vemuri BC, et al. An efficient motion estimator withapplication to medical image registration. Medical Im-age Analysis 1998;2:79–98.

29. Viola P, Wells WM. Alignment by maximization ofmutual information. Fifth International Conference onComputer Vision, 1995. p 16–23.

30. Weese J, Penney GP, Buzug TM, Fassnacht C, LorenzC. 2D/3D registration of pre-operative CT images andintra-operative x-ray projections for image guided sur-gery. In: Lemke H, Vannier MW, Inamura K, editors.Computer Assisted Radiology and Surgery. Proceed-ings of the 11th International Symposium and Exhi-bition (CAR’97). Amsterdam: Elsevier, 1997. p 833–838.


A minimax entropy registration framework for patient setup verification in radiotherapy

Documents

Transcript of A minimax entropy registration framework for patient setup verification in radiotherapy