IOP PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY

Phys. Med. Biol. 56 (2011) 117–137 doi:10.1088/0031-9155/56/1/008

Non-rigid registration between 3D ultrasound and CTimages of the liver based on intensity and gradientinformation

Duhgoon Lee1, Woo Hyun Nam1, Jae Young Lee2 and Jong Beom Ra1

1 Department of Electrical Engineering, KAIST, Daejeon, Korea2 Department of Radiology, Seoul National University Hospital, Seoul, Korea

E-mail: jbra@ee.kaist.ac.kr

Received 30 June 2010, in final form 20 October 2010Published 30 November 2010Online at stacks.iop.org/PMB/56/117

AbstractIn order to utilize both ultrasound (US) and computed tomography (CT) imagesof the liver concurrently for medical applications such as diagnosis and image-guided intervention, non-rigid registration between these two types of imagesis an essential step, as local deformation between US and CT images existsdue to the different respiratory phases involved and due to the probe pressurethat occurs in US imaging. This paper introduces a voxel-based non-rigidregistration algorithm between the 3D B-mode US and CT images of theliver. In the proposed algorithm, to improve the registration accuracy, weutilize the surface information of the liver and gallbladder in addition tothe information of the vessels inside the liver. For an effective correlationbetween US and CT images, we treat those anatomical regions separatelyaccording to their characteristics in US and CT images. Based on a novelobjective function using a 3D joint histogram of the intensity and gradientinformation, vessel-based non-rigid registration is followed by surface-basednon-rigid registration in sequence, which improves the registration accuracy.The proposed algorithm is tested for ten clinical datasets and quantitativeevaluations are conducted. Experimental results show that the registrationerror between anatomical features of US and CT images is less than 2 mm onaverage, even with local deformation due to different respiratory phases andprobe pressure. In addition, the lesion registration error is less than 3 mm onaverage with a maximum of 4.5 mm that is considered acceptable for clinicalapplications.

(Some figures in this article are in colour only in the electronic version)

1. Introduction

The ultrasound (US) imaging system is popularly utilized in various clinical areas, as thistype of imaging is non-invasive, occurs in real-time and is less expensive than other imaging

0031-9155/11/010117+21$33.00 © 2011 Institute of Physics and Engineering in Medicine Printed in the UK 117

118 D Lee et al

modalities such as computed tomography (CT) or magnetic resonance imaging (MR imaging).Especially in the case of the liver, US scanning is frequently performed for clinical applicationssuch as diagnoses, biopsies, percutaneous ethanol injection therapies and radio-frequencyablations. However, due to lower overall image quality of US images than that of CT or MRimages, clinicians often have difficulty in examining target organ(s) in their applications. Toovercome this limit of US imaging, there have been several attempts to display or fuse a USimage with its corresponding high-quality CT (or MR) image to provide clear informationregarding the anatomy and/or target lesion(s) (Lange et al 2004, 2009, Lee et al 2008, Penneyet al 2001, 2004, Wein et al 2008, 2010). Given that lesions in a US image are often nearlyinvisible, the corresponding CT image is helpful for providing lesion information to cliniciansfor diagnosis and planning of US-guided intervention. To achieve a CT image preciselyaligned to a US image for the liver, non-rigid registration between US and CT images isrequired.

An objective function, such as a similarity or dissimilarity measure, must be properlydefined for a successful registration. However, as the relationship between US and CT imagesvaries with organs, the objective function between US and CT depends on the organ inwhich the registration is applied, although there have been efforts made in defining an organ-independent function (Zhang et al 2006). The registration between the US and MR images ofthe brain is introduced by correlating the intensities of the US image with both the intensitiesand gradient magnitudes of the MR image based on a bivariate correlation ratio (Roche et al2001). For the registration of the US and CT prostate images, mutual information has beenadopted as a similarity measure (Castro-Pareja et al 2005). A registration algorithm for theUS and CT images of the spine was also introduced (Brendel et al 2002). In that paper, afteran estimation of the surface of the spine in both images, the objective function is defined asthe average intensity value of the points in the US image corresponding to sample points onthe estimated surface in the CT image.

For the registration of the US and CT images of the liver, several algorithms have beensuggested. Two of these registration algorithms adopt a voxel-based approach (Penney et al2004, Wein et al 2008). As the image characteristics of a US liver image differ from thecharacteristics of a CT liver image, these methods typically convert the US image and/orCT/MR image into a different type of image(s) to correlate the two images. More specifically,one algorithm converts US and MR images into vessel probability images by remapping theintensities based on a set of training data (Penney et al 2004). The algorithm then usesthe normalized cross-correlation similarity measure between the two probability images tooptimize the rigid transformation parameters. The other algorithm generates a simulated USimage from the CT image on the basis of US physics (Wein et al 2008). Registration is thenperformed by examining the similarity between the simulated and original US images basedon the correlation ratio. The semi-affine transformation model is used in this registration. Afeature-based approach was also used (Penney et al 2001). In this approach, which uses thevessel centerline and liver surface as features, an ICP-based rigid registration algorithm isdeveloped in which the objective function is given as the average closest distance between thefeature samples of the US and CT images. In this approach, a segmentation step is requiredfor the extraction of features in both US and CT images.

It is known that the liver undergoes respiratory motion and that its degree of movementvaries with the subject (Brandner et al 2006, Shimizu et al 1999). Furthermore, the respiratorymotion of the liver consists of non-rigid deformation as well as rigid translation. It has beenfound that in cases involving deformation of the liver, there is an average registration error ofapproximately 10 mm between the inspiration and expiration phases after rigid registration.Non-rigid registration can successfully compensate for this deformation (Rohlfing et al 2004).

Non-rigid registration between 3D ultrasound and CT images of the liver 119

In a CT scan, an image is usually acquired at either the end-inspiration phase or theend-expiration phase in accordance with the imaging protocol. However, it is practically verydifficult to obtain a US image at the exactly same respiratory phase that corresponds to a CTimage. Moreover, in some applications, multiple registrations of a CT image to US imagesin various respiratory phases are required. Hence, it should be considered in the registrationprocess that the US and CT images may be acquired at different respiratory phases. In addition,the probe pressure during US scanning can cause local deformation of organs (Mcgahan et al2004, Treecea et al 2002). Therefore, local deformation due to the US probe pressure can alsobe a source of registration error in any registration between the US and CT images of the liver.This error should also be compensated for. However, all of the aforementioned registrationalgorithms between the US and CT images of the liver are based on a global transformationbased on rigid or semi-affine parameters. Hence, they cannot sufficiently compensate for localdeformation due to the probe pressure and the different respiratory phases between US andCT scans.

To describe the spatial relationship between US and CT image precisely with consideringlocal deformations, several trials of non-rigid registration have been attempted. An ICP-basednon-rigid registration algorithm (Lange et al 2004) used an additional power Doppler (PD)US image to register the CT image to a B-mode US image, adopting multilevel B-splines(Xie and Farin 2001) for modeling the non-rigid transformation. In this algorithm, only thevessel information or the samples of vessel centerlines were utilized for the registration. Anon-rigid registration algorithm was also developed that used combined landmark-intensityinformation (Lange et al 2009). In this algorithm, a few corresponding landmark pairs aremanually selected in advance. Based on this prior knowledge of landmarks and on thenormalized gradient field (Haber and Modersitzki 2006) between PD US and CT images, anenergy functional is defined and minimized to estimate the transformation. Instead of usingPD US images, B-mode US images were directly used in a couple of non-rigid registrationalgorithms (Lee et al 2008, Wein et al 2010). Preliminary research was conducted on thenon-rigid registration between the B-mode US and CT images of the liver by using a similaritymeasure based on intensity and gradient information (Lee et al 2008). They adopt a parametrictransformation based on a B-spline free form deformation (FFD) model and estimate it byminimizing a cost function based on the similarity. Wein et al (2010) have recently extendedtheir previous registration algorithm (Wein et al 2008) to the non-rigid registration. Theypropose a two-component similarity measure based on local cross-correlation and maximizeit using a variational registration framework for obtaining a deformable transformation.

The present paper proposes an accurate voxel-based registration algorithm between the3D B-mode US and CT images of the liver. The proposed algorithm also focuses on non-rigidregistration and aims to improve the registration accuracy by considering local deformationsdue to patient respiration and the US probe pressure, which cannot be compensated for using aglobal registration algorithm. In the proposed algorithm, to improve the registration accuracy,both the vessel information and the surface information of the liver and gallbladder (GB)are used for the registration of the liver image. Based on the correlation of the intensityand gradient information between US and CT images, different similarity measures areconsidered according to the image characteristics of the feature regions, including the livervessels and the surfaces of the liver and GB in the two images. To describe the similaritymeasures between US and CT images, an objective function based on intensity and gradientinformation that we had originally introduced for the registration between CCD and IR imagesis adopted (Kim et al 2008). The proposed registration algorithm sequentially performsthe vessel-based and surface-based registration processes based on corresponding similaritymeasures.

120 D Lee et al

Coronal Sagittal

Axial

Figure 1. GB surface attached to the liver.

The paper is organized as follows. In section 2, the registration algorithm based on thevessel and surface information is described in detail. Section 3 describes the experimentalresults for the clinical datasets. Finally, the conclusions are given in section 4.

2. The proposed method

2.1. Anatomical features for registration

For the two given images of a US image and a contrast-enhanced CT image, we aim to finda transformation that aligns the CT onto the US image through a registration process. Forthis process, we utilize several anatomical features in the liver, specifically the vessels, liversurface and GB surface attached to the liver. Vessels are well-known features in registrationsbetween the US and CT images of the liver because they are visible in both images (Langeet al 2004, 2009, Penney et al 2001, 2004, Wein et al 2008). The liver surface is alsoidentifiable in both images; hence, some researchers have used it as a meaningful feature forregistration (Lee et al 2008, Penney et al 2001, Wein et al 2008, 2010). Moreover, clinical3D US images often include a part of the GB in the region of interest, and it is known thata certain part of the GB is attached to the liver (Moore and Agur 2007). The CT image infigure 1 demonstrates that the GB surface is attached to the liver on the solid lines. We willattempt to use this GB surface as an additional effective feature for the registration of the liverimage.

To correlate the US and CT images of the liver, we focus on the intensity and gradientinformation for each feature region. The intensity values of vessels in a conventional B-modeUS image are lower than those of the neighboring area of the vessels, as shown in figure 2(a).

Non-rigid registration between 3D ultrasound and CT images of the liver 121

(a) (b)

(c) (d)

Figure 2. Relationship between US and CT images of vessels: (a) US image, (b) contrast-enhancedCT image, (c) and (d) gradient orientations of a vessel region in US and CT images, respectively.

In a contrast-enhanced CT image, the intensity values of vessels are higher than those of softtissue, as shown in figure 2(b). Therefore, in a statistical sense, the intensity values of a USimage are likely related to those of a CT image in the vessel region. The edge orientation canalso be a common feature of US and CT vessels, as shown in figures 2(c) and (d). Thus, it canalso be used as important information in the registration.

The liver surface has high reflectivity to the US beam if the acoustic impedance differenceis large between the regions inside and outside the liver. Hence, the corresponding liver surfaceprovides high intensity values in US images. Meanwhile, the CT liver image shows a definiteboundary for the liver surface by providing different intensity values between regions insideand outside the liver. Therefore, the liver surface in the CT image provides high gradientvalues, which can be related to the high intensity values in the US image, as shown in the solidboxes in figures 3(a) and (b). In addition, the edge orientation is useful information, as it isavailable in both US and CT images and is strongly correlated between them. Figures 3(c)and (d) demonstrate the edge orientation via arrows within a liver surface region.

In contrast to the liver surface with a large acoustic impedance difference, the GB surfaceattached to the liver shows a definite boundary with different intensity values between regionsinside and outside the GB in both US and CT images, as shown in the dotted boxes infigures 3(a) and (b). Hence, the intensity values are directly correlated with each other. Edgeorientations in the GB surface region are also highly correlated, as shown in figures 3(e) and(f).

2.2. Overview of the proposed algorithm

Based on the image characteristics described above, we propose an accurate registrationalgorithm between the US and CT images of the liver. A diagram of the proposed algorithm

122 D Lee et al

(a) (b)

(c) (d)

(e) (f)

Figure 3. Relationship between the US and CT images of the liver and GB surface regions: (a)US image, (b) contrast-enhanced CT image, (c) and (d) gradient orientations of the liver surfaceregion in US and CT images, respectively, and (e) and (f) gradient orientations of the GB surfaceregion in US and CT images, respectively.

is given in figure 4. We first perform preprocessing, which includes an automatic denoisingprocess in the US image and automatic denoising and semi-automatic anatomical featureextraction in the CT image, respectively. The registration process is then conducted onlyusing the vessel information. Based on the vessel-based registration result, registration isperformed again using the surface information. Finally, a transformation is estimated usingboth the vessel- and surface-based registration results and a CT image registered to the USimage is generated via the estimated transformation. Each step is described in detail in thefollowing subsections.

In the algorithm, the registration process is divided into the two parts: vessel-basedregistration and surface-based registration. These are performed sequentially. As vesselscontain salient features in the 3D space, the non-rigid registration based on the vesselinformation can provide accurate alignment of the vessels between the US and CT images. Incontrast, the liver and GB surfaces have smooth and convex shapes in US images. Locating

Non-rigid registration between 3D ultrasound and CT images of the liver 123

Final estimation of the transformation

Non-rigid registration

PreprocessingUS image

CT imageNon-rigid

registration

Estimation of the transformation

Filtered US image

Filtered CT image & mask of vessels

Rigid registrationCT segments of vessels Trigid_vessel

Treg_vessel

Test_vessel

Preprocessing

Filtered CT image & masks of liver and GB surfaces

Registered CT

Treg_surface

Automatic procedure

Semi-automaticprocedure

Based on vessel information

Based on surface information

Based onvessel and surface information

Test_v&s

CT segments of surfaces

Figure 4. The registration process.

Surface in US imageSurface in CT image

(a) (b) (c)

Figure 5. Schematic illustration of surface alignment ambiguity: (a) relative surface positionsof US and CT images before non-rigid registration, (b) and (c) two different but well-alignednon-rigid registration results.

salient features on these surfaces is difficult. The liver and GB surfaces appearing in aUS image are also only a part of the entire surface in the CT image. Therefore, non-rigidregistration based only on the surface information cannot guarantee a correct alignment inthe region near the surfaces, although the surfaces of US and CT images appear to be wellaligned in the 3D space after registration due to the high degree of freedom (DOF) in the non-rigid registration process. Figure 5 illustrates this ambiguity in the surface registration of theliver.

To overcome this inherent limit of a surface registration and guarantee a more reliableregistration process for the surface region, a good initial transformation prior to the non-rigidregistration between the surfaces is necessary. Hence, in the proposed registration process,for a more reliable registration, the initial transformation is estimated first based on the vessel-based registration. This result is then applied to the surface-based registration.

124 D Lee et al

Figure 6. Anatomical features in a 3D CT image for the registration.

2.3. Preprocessing

Given that the proposed registration algorithm is based on the intensity value of each voxel inUS and CT images, a denoising process is used before registration. To remove the noise whilesimultaneously preserving edges even with low contrast, for both the US and CT images,we apply a noise-constrained filtering algorithm with anisotropic diffusion that considers thenoise characteristics of each image (Krissian et al 2005).

As mentioned above, for registration between the US and CT images of the liver, it isnecessary to manipulate the anatomical information of the vessels, liver and GB that areidentifiable in both images. To contend with this anatomical information, we segment thevessels, liver and GB only in the CT image. Here, the segmentation process in the CT imageis not necessarily automatic; it can be considered as a preoperative processing step that usesvarious complex and/or user-interactive approaches for a satisfactory segmentation result. Inthis paper, to segment the organs, we first extract the organs by applying a conventional region-growing scheme (Adams and Bischof 1994) to a filtered CT image. The segmented organs arethen manually refined using 3D sculpting software. Figure 6 shows a volume-rendered imageof segmented CT organs. Based on the segmentation results, we generate masks MCT_V,MCT_LS and MCT_GS of the anatomical features of the vessels and surfaces of the liver andGB, respectively. The masks include voxels within the regions which are obtained simply bymorphologically dilating the segmented anatomical features with a margin of 3 mm.

2.4. Registration based on vessel information

To utilize the vessel information in the registration process, we establish a relationship of theintensity and the edge orientation in the vessel region between the US and CT images andadopt a similarity measure based on this relationship. By minimizing the cost as defined viathe similarity measure, we first find the parametric transformation Treg_vessel through the rigidand non-rigid registrations that aligns the vessel region in the CT image onto that in the USimage. The transformation Test_vessel for the whole liver, to be used as the initial transformationin the subsequent process, is then estimated based on the obtained Treg_vessel. Details of theprocedure are given below.

Non-rigid registration between 3D ultrasound and CT images of the liver 125

Transform the image & mask.

Updatelocal transformation

parameters.Calculate the cost.

Calculategradient information

in the mask.

No

EndYes

Generate a mask.

Minimum?

Filtered US image Filtered CT image &a mask of anatomical feature

Initial transformation

Calculategradient information

in the mask.

Figure 7. The proposed non-rigid registration scheme used in figure 4.

2.4.1. Rigid registration. To obtain the rigid transformation, Trigid_vessel, between the USand CT images of the liver, an ICP-based rigid registration algorithm is applied that uses thecenterlines of the vessels in the liver (Nam et al 2010). To extract the centerlines of segmentedvessels in the CT image, we apply Bitter’s skeletonization method to the pre-segmented vessels(Bitter et al 2001). For the centerlines of the vessels in the US image, we adopt our previouslypresented automatic feature extraction algorithm in a 3D B-mode US image (Nam et al 2008).

2.4.2. Non-rigid registration. The rigid transformation Trigid_vessel is used as the initialtransformation to obtain the non-rigid transformation Treg vessel. This can be represented as

Treg vessel (x) = Trigid vessel (x) + Tlocal (x) . (1)

Here, x denotes a point in the US image and Tlocal denotes the local transformation.Additionally, Tlocal is obtained through the proposed non-rigid registration algorithm. Inthe algorithm, a B-spline FFD model known to be appropriate for liver motion modeling(Rohlfing et al 2004) is also adopted.

The overall structure of the proposed non-rigid registration algorithm is illustrated infigure 7. As inputs to the algorithm, the filtered US and CT images and the CT mask ofanatomical feature are applied through the preprocessing step described in subsection 2.2.Initial transformation is applied through the rigid registration described in subsection 2.4.1.Note that by focusing only on the features in the mask, we can avoid interferences fromoutside the mask in the registration procedure. First, a mask of the corresponding US featureis automatically generated from the filtered 3D US image. The CT mask is also deformed basedon the updated parameters and the gradient information in the mask regions of both the USand the deformed CT images is then calculated. Using the intensity and gradient informationin the overlapping mask region, the cost is calculated. This consists of an objective functionand a constraint. It is minimized through an optimization process to determine Treg_vessel. Indetermining Treg_vessel, as only masked vessel regions are used, it is possible not only to reducethe computation time but also to alleviate any undesirable effects of noisy non-vessel regionsduring the registration process.

126 D Lee et al

Mask. An accurate CT vessel mask can be determined because user-interactive vesselsegmentation is allowed in the preprocessing step. On the other hand, for a given accurateCT vessel mask, a relatively low accuracy of the US vessel mask is acceptable because onlythe overlapping mask between US and CT is used for registration. Hence, we determine USvessel candidates using only an adaptive threshold scheme, which is a part of a complicatedvessel extraction algorithm for 3D B-mode US images (Nam et al 2008). In the scheme, thethreshold is given as

Th1(n) = μLN(n) − σLN

(n), (2)

where μLN(n) is the mean of the intensities in a 3D local window with N voxels centered

at voxel n and σLN(n) is the standard deviation of the intensities in the local window. The

voxels with intensities that are lower than the threshold Th1 are regarded as vessel candidates.The vessel mask MUS_V is then determined as a dilated version of the vessel candidates. Thegradient information of the US and deformed CT images is then extracted only in the vesselmasks and the cost is calculated only in the overlapping region of the two vessel masks fromthe US and deformed CT images.

Edge orientation. To obtain the edge orientation information, it is necessary to calculate agradient vector at each voxel. To obtain the gradient vectors, we adopt an eigen analysis ofa structure tensor (Harris and Stephens 1988). The structure tensor S at each voxel can bewritten as

S(∇I )= Kρ ∗⎛⎝Gx

Gy

Gz

⎞⎠ (Gx Gy Gz )=

⎛⎝ Kρ ∗ G2

x Kρ ∗ GxGy Kρ ∗ GxGz

Kρ ∗ GyGx Kρ ∗ G2y Kρ ∗ GyGz

Kρ ∗ GzGx Kρ ∗ GzGy Kρ ∗ G2z

⎞⎠ , (3)

where Gx, Gy and Gz denote the intensity gradients along the x, y and z directions in image I,respectively. Additionally, the Gaussian convolution kernel, Kρ, with the standard deviationρ can be expressed as

Kρ(x, y, z) = (2πρ2)−1 · exp

(−x2 + y2 + z2

2ρ2

). (4)

Through eigenvalue decomposition, S can be rewritten as

S(∇I ) = (ω1 ω2 ω3 )

⎛⎝μ1 0 0

0 μ2 00 0 μ3

⎞⎠

⎛⎝ωT

1

ωT2

ωT3

⎞⎠ . (5)

Here, the eigenvectors ω1 and ω3 represent the directions of the maximum and minimumintensity variations, respectively, while the eigenvalues μ1 and μ3 denote the magnitudes ofω1 and ω3, respectively. A voxel with an anisotropic nature has a large difference between itsmaximum and minimum eigenvalues and is likely to locate on an edge. Hence, we regard ω1

as the edge orientation and μ1 – μ3 as the gradient magnitude. To alleviate the noise effect,we discriminate the edge region R in a given mask M as follows:

R = {x |μ1 (x) − μ3 (x) > Th2, x ∈ M } . (6)

Objective function. To manipulate a feasible relationship of the vessel intensity andedge orientation between the US and CT images of the liver for registration (refer tosubsection 2.1), an objective function previously introduced by the authors is adopted.This function was developed for CCD and IR image registration (Kim et al 2008). As therelationship between those multimodal images is similar to that between US and CT images

Non-rigid registration between 3D ultrasound and CT images of the liver 127

in the liver, the adopted objective function utilizing both the intensity and edge orientationinformation is considered proper for a registration between the US and CT images of the liver.

To obtain the objective function, we first construct a 3D joint histogram, h(a,b,o(�θ )).Here, a and b denote the intensity values of images A and B, respectively, and o(�θ ) denotesthe orientation coincidence of the two images. This is represented as

o(�θ) = 1 + cos(2�θ)

2, (7)

where �θ is the difference between the edge orientation angles in images A and B. Theobjective function F is then defined as

F(A,B) = W(A,B) · E(A,B). (8)

Here, the entropy term E is determined as

E(A,B) = H(A,B,O) − M(A,B), (9)

where M(A,B) represents the mutual information and H(A,B,O) is the 3D joint entropycalculated from this 3D joint histogram. The weighting term is determined as

W(A,B) = 1 − C(A,B), (10)

where the coincidence measure C is given as

C(A,B) =∑

x∈(RA∩RB) 2 · o (�θx)

NA + NB

. (11)

Here, RA and RB denote the edge regions which are obtained by using equation (6) in imagesA and B, respectively, and NA and NB denote the numbers of voxels in RA and RB, respectively.The weighting term W represents the disagreement in the orientation. When two images arecorrectly registered, it is expected that W is minimized.

Based on equations (7)–(11), the objective function regarding the vessels can be writtenas

Fvessel(IUS, I′CT) = W(IUS, I

′CT) · E(IUS, I

′CT)

=

⎧⎪⎨⎪⎩1 −

∑x∈(RUS V∩RCT V)

(1 + cos(2�θx)

NUS + NCT

⎫⎪⎬⎪⎭

· {H(IUS, I′CT,O) − M(IUS, I

′CT)}, (12)

where IUS and I′CT denote the US and deformed CT images, respectively, RUS_V and RCT_V

denote the regions corresponding to the vessel edges in the two vessel masks MUS_V andM′

CT_V of IUS and I′CT, respectively, and NUS and NCT denote the numbers of pixels in RUS_V

and RCT_V, respectively. While the weighting term W is determined in the overlapping regionof RUS_V and RCT_V, the entropy term E is determined only for the overlapping region of MUS_V

and M′CT_V of IUS and I′CT, respectively.

Cost. We now define a cost function that provides the minimum value when two images arecorrectly registered:

CFvessel = Fvessel + λvessel · Fconstraint. (13)

As a constraint function, Fconstraint, we adopt an incompressibility constraint (Rohlfing andMaurer 2003) which is used for regularization in the non-rigid registration of the liver images(Rohlfing et al 2004). The constraint function is expressed as

Fconstraint = 1

ND

∑x∈D

|log (JT(x))|, (14)

128 D Lee et al

where D denotes the entire region of a US image, ND is the number of voxels in D and JT

denotes the Jacobian determinant. To find the minimum value of the cost function whichprovides the optimized transformation parameters, the gradient descent scheme (Rueckertet al 1999) is adopted in this paper.

2.4.3. Transformation estimation. Although the transformation Treg_vessel is determined forthe entire region of a US image, the transformation is accurate and reliable only in the vesselregions since the vessel information is used for registration. Hence, it is not sufficient to usethis as the initial transformation of the subsequent surface registration procedure. To obtaina better initial transformation, we re-estimate the transformation on the basis of a thin-platespline (TPS) model because this model tends to provide a smoothly varying transformationamong control points (Hajnal et al 2001).

To use the TPS model, we choose sample points p in the US image that are mapped tovoxels p′ in the pre-segmented vessels of the CT image. We regard p and p′ as correspondingcontrol point pairs. The transformation based on a TPS is then given as

Test vessel(x) = A · x +n∑

i=1

{biφ (|pi − x|)}, (15)

where A denotes the affine matrix, bi denotes the non-affine coefficients, φ represents a radialbasis function and n is the number of vessel point correspondences. Here, A and bi aredetermined using the corresponding pairs p and p′. The transformation Test_vessel is consideredas a refined version of Treg_vessel. It is a good initial transformation for further refinement.

2.5. Registration based on surface information

Although at this point we have obtained the transformation Test_vessel, its level of accuracy isnot satisfactory because it relies only on the vessel information. To improve the registrationaccuracy, we use the liver surface as additional useful information because it is clearlyidentifiable in both US and CT images of the liver. To improve the accuracy further, wealso adopt the GB surface which is attached to the liver as additional surface information.

2.5.1. Manipulation of the liver surface. Based on the liver surface relationship betweenUS and CT images as described in subsection 2.1, we define an objective function for theregistration in the liver surface region. While the objective function in the vessel region,Fvessel, is determined using the intensity values and edge orientations from both the US andthe CT images, this function in the liver surface region, Fliver_surface, is determined using thegradient magnitudes instead of the intensity values in the CT image:

Fliver surface(IUS, I′CT) = W(IUS, I

′CT) · E(IUS,

∣∣∇I ′CT

∣∣)

=

⎧⎪⎨⎪⎩1 −

∑x∈(RUS LS∩RCT LS)

(1 + cos(2�θx)

NUS + NCT

⎫⎪⎬⎪⎭

· {H(IUS, |∇I ′CT|,O) − M(IUS, |∇I ′

CT|)}. (16)

Here, RUS_LS and RCT_LS denote the edge regions in the two liver surface masks MUS_LS andM′

CT_LS of IUS and I′CT, respectively, and |∇I′CT| denotes the gradient magnitude of the imageI′CT. As expressed by equation (12), the entropy term E is determined only for the overlappingregion of MUS_LS and M′

CT_LS of IUS and I′CT.

Non-rigid registration between 3D ultrasound and CT images of the liver 129

Figure 8. A US image and overlay of a mask for the liver surface on the US image. ‘A’ indicatesan unwanted high echogenic structure.

According to equation (16), the objective function is calculated only within the overlappingmask region. To determine the mask of IUS automatically, we first segment the liver surface byadopting an algorithm based on a Hessian matrix analysis (Nam et al 2008) and then expandthe surface with a certain margin. The segmentation of the liver surface is straightforwardfor the liver surface attached to the diaphragm, which is located far from the probe contactposition. However, the segmentation of the liver surface beneath the probe contact position isdifficult due to unwanted high echogenic structures coming from several sources, such as thefat layer of the abdominal wall (see layer A in figure 8). To solve this problem, we first removethe segment corresponding to the liver surface attached to the diaphragm with a distance largerthan a certain value (a value of 25 mm is heuristically obtained from our datasets) from theprobe contact position. We then select the surface located farthest from the probe contactposition from among the remaining segments. After segmenting the liver surface attached tothe diaphragm and the surface near the probe contact position, we generate a liver surfacemask MUS_LS of the US image which includes the voxels within the region obtained by dilatingthe extracted surfaces, as demonstrated in figure 8.

2.5.2. Manipulation of the GB surface. As the GB surface relationship between US and CTimages is similar to that of the vessels, as described in subsection 2.1, we adopt the objectivefunction of the vessels as the objective function for the GB surface region:

FGB surface(IUS, I′CT) = W(IUS, I

′CT) · E(IUS, I

′CT)

=

⎧⎪⎨⎪⎩1 −

∑x∈(RUS GS∩RCT GS)

(1 + cos(2�θx)

NUS + NCT

⎫⎪⎬⎪⎭

· {H(IUS, I′CT,O) − M(IUS, I

′CT)}. (17)

Here, RUS_GS and RCT_GS denote regions corresponding to GB edges in the two GB masksMUS_GS and M′

CT_GS of IUS and I′CT, respectively. Note also that the entropy term Eis determined only for the overlapping region of MUS_GS and M′

CT_GS of IUS and I′CT,respectively.

As mentioned in subsection 2.3, the GB surface attached to the liver was by this pointobtained through the pre-processing step of the CT image. Hence, the corresponding maskcan be generated by expanding the GB surface. Meanwhile, the GB surface mask of the USimage can be determined using the same scheme used for the US liver surface mask.

130 D Lee et al

2.5.3. Application of the surface information to the registration. Based on the aboveobjective functions for the surface information, non-rigid registration is performed toalign those surfaces. This non-rigid registration process is nearly identical to that insubsection 2.4.2. We define the two cost functions for the liver surface and the GB surface asfollows:

CFliver surface = Fliver surface + λliver surface · Fconstraint, (18)

CFGB surface = FGB surface + λGB surface · Fconstraint. (19)

To obtain the optimal transformation, it may be necessary to minimize the two costssimultaneously. However, instead of doing this, the transformation can be obtained byalternately minimizing the two costs. The degree of local deformation of the surfaces, whichshould be offset in the non-rigid registration process, depends on the initial transformation.Moreover, this degree can differ depending on the surface. Hence, alternate minimizationof the two costs is considered more efficient for obtaining the desired transformation, as itallows more DOF in the deformation of each surface in the registration with B-spline FFDtransformation. We also use a gradient descent optimization scheme for minimization, as inthe vessel-based registration.

2.6. Final estimation of the transformation

Through the registration processes based on the vessel and surface information, we can achieveaccurate correspondences of the vessels, liver surface and GB surface attached to the liverbetween US and CT images. As in the vessel case in subsection 2.4.3, pairs of correspondingpoints on the surfaces can also be obtained simply by choosing the sample points in the USimage that are mapped to voxels of the pre-segmented CT image surfaces. Using those pairson vessels and surfaces, we attempt to estimate the final transformation between the US andCT images for the whole liver. The final transformation Test_v&s is determined by referring toequation (15).

3. Experiments

3.1. Data acquisition and experimental conditions

To evaluate the performance of the proposed algorithm, we use ten datasets obtained fromvolunteers. The study protocol was approved by the Institutional Review Board of SeoulNational University Hospital, and written informed consent was obtained from each volunteer.For evaluation of lesion registration performance, we use five datasets that include clearlyvisible lesion(s) in both US and CT images so that it (they) can be segmented in both images.

In each dataset, contrast-enhanced 3D CT images were acquired at the end-inspirationwith a breath-hold using a Siemens Sensation 16 CT scanner. The typical image dimension andvoxel size in these images were 512 × 512 × 220 and 0.65 × 0.65 × 1.0 mm3, respectively,although there were slight variations. 3D B-mode US images were also acquired using aMedison Accuvix V10 ultrasound scanner equipped with a mechanical 3D US probe. Atypical image dimension is 200 × 124 × 148 in the Cartesian coordinates with an isotropicvoxel size of 1.82 mm3. For the experiment, we acquired 3D US images at the respiratoryphase of close-expiration with a breath-hold. This point is clearly distinct from the respiratoryphase in the CT images.

To extract the gradient information, the value of Th2 was empirically set to 10 in all of theUS images. Meanwhile, in the CT images, the value of Th2 was set to 100, which is common

Non-rigid registration between 3D ultrasound and CT images of the liver 131

Table 1. Registration accuracy test for ten clinical datasets.

Value: mean (±STD) (mm)Rigid (vessels) Non-rigid (vessels) Non-rigid (vessels and surfaces)

Average Vessels Surface Average Vessels Surface Average Vessels Surfaces

4.95 (2.26) 3.78 (0.72) 5.67 (2.60) 3.81 (2.47) 1.88 (0.27) 5.02 (2.48) 1.78 (0.37) 1.90 (0.25) 1.71 (0.43)

for vessel and GB surface regions. It was set to 500 for the liver surface regions. Those valuesof Th2 were determined to be low enough to extract the desired gradient information withintheir masks. The 3D joint histograms used to calculate the entropy values of the objectivefunctions were generated with 32 × 32 × 32 bins, as in a recent study (Kim et al 2008). TheB-spline FFD transformation was defined by using a 13 × 9 × 10 mesh of control points withuniform spacing of 20 voxels for the US image. The weighting values of λvessel, λliver_surface

and λGB_surface were also empirically set to 10, 50 and 50, respectively, for all datasets.

3.2. Experimental results

Figures 9 and 10 show the registration results between the 3D US and CT images. For a simplevisual assessment of the registration accuracy, we indicate the corresponding locations withseveral pairs of arrows in both images. The top-left image in each figure is a US image, thetop-right image is the rigidly registered CT image, the bottom-left image is an intermediateresult obtained from the registration based on the vessel information and the bottom-rightimage is the final result of the proposed algorithm. The figures demonstrate that accuratecorrespondences can be obtained in the liver surface region (and the GB surface region infigure 10) as well as in the vessel regions. A comparison of the top-right and bottom-leftimages verifies that local deformation in the vessel region can be properly compensated forby a non-rigid registration. The bottom-right images clearly demonstrate the improvement ofthe registration accuracy due to the additional use of the surface information.

For the quantitative evaluation of the proposed registration algorithm, we adopt a distance-based measure on the basis of the anatomical features of the vessels, the liver surface and theGB surface. The distance measure for a pair of US and CT images can be written as follows:

DM = 1

NA

∑xUS∈A

minxCT∈B

{d (Test v&s(xUS), xCT)}. (20)

Here, A and B denote the set of feature samples in the US and CT images, respectively. NA

is the number of samples in A, and d(·) denotes a function representing the distance betweenthe two input points. Set A (or B) consists of vessel centerlines, liver surface and GB surfacein the US image (or the CT image). Set A for the US image is mainly extracted on the basisof the feature extraction algorithm (Nam et al 2008). Some manual segmentation tasks areperformed for refinement of those features. Meanwhile, set B for the CT image is determinedby using the segmentation method that is described in subsection 2.3. To extract the vesselcenterlines in the CT image, we apply Bitter’s skeletonization method to the segmented vessels(Bitter et al 2001). Table 1 shows the averaged DM and its standard deviation (STD) for theclinical datasets. In the table, to verify the improvement of the registration accuracy ofthe proposed algorithm, we represent quantitative errors for rigid registration using vessels(subsection 2.4.1), for non-rigid registration using vessels (subsection 2.4.3) and for theproposed registration using both vessels and the surface information (subsection 2.6). It isclear that the overall DM for both regions of vessels and the surface is less than 2 mm, evenwith greatly different respiratory phases between the US and CT images. The DM for surface

132 D Lee et al

USRigid

(vessels)

Non-rigid(vessels)

Non-rigid(vessels + surface)

USRigid

(vessels)

Non-rigid(vessels)

Non-rigid(vessels + surface)

(a)

(b)

Figure 9. Comparison of vessel-based rigid, vessel-based non-rigid and vessel and surface-basednon-rigid registration results. The arrows show several typical positions for comparison. (a) Axialslice, (b) sagittal slice.

regions decreases significantly after being processed by the proposed registration algorithm.This shows that the algorithm can be helpful in improving the registration accuracy of a targetlesion even when the lesion is located far from any vessels but near the surface.

We also measured the fiducial registration error (FRE) by defining it as the root-mean-square of the distances between corresponding fiducial points. The fiducial pointcorrespondences in the liver were selected on vessel bifurcations by an expert. For eachdataset, four to six point correspondences were obtained for the FRE measurement. Table 2

Non-rigid registration between 3D ultrasound and CT images of the liver 133

USRigid(vessels)

Non-rigid(vessels)

Non-rigid(vessels + surface)

Figure 10. Comparison of vessel-based rigid, vessel-based non-rigid and vessel- and surface-basednon-rigid registration results. The white arrow shows the GB surface attached to the liver in theUS image; the black arrows are located at the position of the white arrow in each registered image.

Table 2. Fiducial registration error for the datasets in table 1.

Value: mean (±STD) (mm)Rigid (vessels) Non-rigid (vessels) Non-rigid (vessels and surfaces)

4.80 (2.62) 2.43 (1.13) 2.39 (1.06)

shows the averaged FRE and its standard deviation for the ten clinical datasets. In the table,after the non-rigid registration process, the FRE decreases significantly.

As mentioned earlier, lesions are visible both in the US and CT images in the five datasets.To evaluate the registration accuracy of lesions, they were manually segmented and delineatedin both images by an expert via a software tool. We then measured the distance betweenthe centers (DC) of a pair of corresponding lesions, USL and CT′

L, in the US and registeredCT images. In addition, to observe the degree of overlap between the lesions in the US andregistered CT images, we adopted an overlap measure (OM) (Camara et al 2007). This isdefined as follows:

OM = n(USL ∩ CT′L)

min {n(USL), n(CT′L)} × 100. (21)

As the volume of a segmented lesion can vary depending on the imaging modality and onimaging conditions such as the degree of the diffusion of the contrast agent in the CT case, theUSL and CT′

L volumes can differ from each other despite the fact that they originate from thesame lesion. Therefore, the minimum value between the USL and CT′

L volumes is used as thedenominator in equation (21).

Table 3 shows the DC and OM values for the five datasets. In datasets C and E, twolesions are identified in each case, while one lesion is identified in the remaining datasetsA, B and D. Hence, seven lesions are evaluated in total to quantitatively examine the lesionregistration error. As shown in the table, the average DC value is 2.84 mm and the DC values

134 D Lee et al

Table 3. Accuracy test of lesion registration for five patient datasets.

Rigid(vessels)

Non-rigid(vessels)

Non-rigid(vessels and

surfaces)

Lesion type DC OM DC OM DC OMPatient (diameter (mm)) (mm) (%) (mm) (%) (mm) (%)

A Hemangioma (10.3) 3.42 68 0.52 100 0.47 100B Hemangioma (23.2) 10.50 62 4.52 95 4.48 94C Cyst_1 (12.2) 4.25 71 1.21 99 1.36 99

Cyst_2 (22.7) 5.08 81 4.54 88 2.75 90D Cyst (18.3) 12.40 20 2.56 87 3.31 86E Cyst_1 (6.2) 6.44 0 6.43 10 3.86 88

Cyst_2 (18.9) 7.85 59 5.25 86 3.62 90Average 7.13 52 3.58 81 2.84 92

range from 0.47 to 4.48 mm, which are considered to be acceptable for image-guided liverintervention (Lange et al 2004). Additionally, in cyst_2 in C and in both cysts in E, the use ofthe surface information improves the registration accuracy. Meanwhile, it is noted in cyst_1in C and in the cyst in D, the DC values of the proposed algorithm show a slight increasecompared to those of the non-rigid registration process based on the vessels. However, it isconsidered that the accuracy of lesion registration is not degraded in the proposed algorithm,as the OM values are mostly preserved. Among the datasets, we select the registrationresults for datasets C and E that include two identifiable lesions. These are shown infigure 11. In this figure, the white contours of lesions of a US image are superimposedonto the registered CT image. These contours help to illustrate that the proposed algorithmprovides very good registration accuracy for lesions and for the anatomical features of thevessels and the surface.

Based on the quantitative evaluations above, the proposed algorithm is shown to provideaccurate non-rigid registration results. These accurate results stem from the efficientmanipulation of the anatomical information in the non-rigid registration framework. Asthe proposed registration algorithm can offset local deformation of the liver due to the USprobe pressure and/or due to the different respiration phases between US and CT images, weexpect that the algorithm can be applied to a range of clinical areas in which high registrationaccuracy is required, such as liver surgery. The proposed algorithm can provide new clinicalsolutions pertaining to image-guided diagnoses and intervention strategies.

To obtain the final registered CT image in our experiments, user interaction was notrequired except for anatomical feature segmentation in the CT preprocessing step which canbe considered a preoperative procedure. Hence, the proposed algorithm is fully automatic in theintraoperative procedure. For real-time intraoperative applications, however, the computationtime of the algorithm is a bottleneck, since two main non-rigid registration steps in figure 4requires several hours of execution time on a standard PC. The acceleration of computationtime is required as future work for real-time applications.

An application example of the proposed algorithm is a real-time concurrent displaysystem for 3D US and CT images. In such a system, multiple-respiratory-phase CT imagesare obtained through the non-rigid registration of a pre-scanned single-respiratory-phase CTimage to real-time multiple-respiratory-phase US images. This display would be very usefulfor image-guided intervention involving the liver. Given that computation time of the 3D

Non-rigid registration between 3D ultrasound and CT images of the liver 135

USRigid(vessels)

Non-rigid(vessels)

Non-rigid(vessels + surface)

USRigid

(vessels)

Non-rigid(vessels)

Non-rigid(vessels + surface)

(a)

(b)

Figure 11. Registration results for two patient datasets. From the top left, a B-mode US slice andits registered CT slices are shown. The superimpositions of the contours of the lesions extractedfrom the US slice are depicted for (a) patient C and (b) patient E in table 3.

non-rigid registration process is too slow for real-time execution, however, it is necessaryto improve the computation time or to consider a system consisting of a preoperative stagerequiring a high computational load and an intraoperative stage requiring a low computationalload (Blackall et al 2005, Huang et al 2009). These approaches using the proposed algorithmare left for future work.

4. Conclusions

An accurate registration algorithm for the 3D B-mode US and CT images of the liver ispresented in this paper. For accurate registration, we utilize the anatomical features of the

136 D Lee et al

liver surface and the GB surface in addition to that of the vessels. To correlate the US and CTimages in the regions corresponding to these anatomical features, the intensity informationand the gradient information, including the magnitude and orientation, are incorporated intoa cost function using a previous 3D-joint-histogram-based objective function proposed bythe authors. To obtain intermediate parametric transformations, the registration algorithmproposed here sequentially performs vessel-based and surface-based registration processeson the basis of the corresponding cost functions. The algorithm then estimates the finaltransformation based on the obtained parametric transformations. Through quantitativeevaluations, the proposed algorithm was demonstrated to be capable of providing registrationresults which are meaningful for clinical applications such as image-guided surgery andintervention. It should be emphasized that the proposed algorithm can accurately register a 3DCT image to a US image regardless of local deformation which can occur due to the differentrespiratory phases between the two images and due to the pressure from the US probe whenapplied to the contact area of the patient.

Acknowledgments

The authors would like to thank Medison Co. Ltd for the financial support of this work.Thanks also to Professor B I Choi in the Department of Radiology, Seoul National UniversityHospital, and to D G Hyun in Medison R&D for the helpful discussions, collaboration, andfor their involvement with the data acquisition stages of this study.

References

Adams R and Bischof L 1994 Seeded region growing IEEE Trans. Pattern Anal. Mach. Intell. 16 641–7Bitter I, Kaufman A E and Sato M 2001 Penalized-distance volumetric skeleton algorithm IEEE Trans. Vis. Comput.

Graphics. 7 195–206Blackall J M, Penney G P, King A P and Hawkes D J 2005 Alignment of sparse freehand 3-D ultrasound with

preoperative images of the liver using models of respiratory motion and deformation IEEE Trans. Med.Imaging 24 1405–16

Brandner E D, Wu A, Chen H, Heron D, Kalnicki S, Komanduri K, Gerszten K, Burton S, Ahmed I and Shou Z 2006Abdominal organ motion measured using 4D CT Int. J. Radiat. Oncol. Biol. Phys. 65 554–60

Brendel B, Winter S, Rick A, Stockheim M and Ermert H 2002 Registration of 3D CT and ultrasound datasets of thespine using bone structures Comput. Aided Surg. 7 146–55

Camara O, Delso G, Colliot O, Moreno-Ingelmo A and Bloch I 2007 Explicit incorporation of prior anatomicalinformation into a nonrigid registration of thoracic and abdominal CT and 18-FDG whole-body emission PETimages IEEE Trans. Med. Imaging 26 164–78

Castro-Pareja C R, Zagrodsky V, Bouchet L and Shekhar R 2005 Automated prostate localization in external-beamradiotherapy using mutual information-based registration of treatment planning CT and daily 3D ultrasoundimages Int. Congr. Ser. 1281 435–40

Haber E and Modersitzki J 2006 Intensity gradient based registration and fusion of multi-modal images MedicalImage Computing and Computer-Assisted Interventation—MICCAI 2006 (Copenhagen, Denmark, Oct. 2006)

Hajnal J V, Hill D L G and Hawkes D J 2001 Medical Image Registration (Boca Raton, FL: CRC Press)Harris C and Stephens M 1988 A combined corner and edge detector Alvey Vision Conf. 1988 (Manchester, UK, Aug.

1988)Huang X, Ren J, Guiraudon G, Boughner D and Peters T M 2009 Rapid dynamic image registration of the beating

heart for diagnosis and surgical navigation IEEE Trans. Med. Imaging 28 1802–13Kim Y S, Lee J H and Ra J B 2008 Multi-sensor image registration based on intensity and edge orientation information

Pattern Recognit. 41 3356–65Krissian K, Kikinis R, Westin C F and Vosburgh K 2005 Speckle-constrained filtering of ultrasound images IEEE

Conf. on Computer Vision and Pattern Recognition 2005 (San Diego, California, USA, June 2005)

Non-rigid registration between 3D ultrasound and CT images of the liver 137

Lange T, Eulenstein S, Hunerbein M, Lamecher H and Schlag P 2004 Augmenting intraoperative 3D ultrasoundwith preoperative models for navigation in liver surgery Medical Image Computing and Computer-AssistedInterventation—MICCAI 2004 (Saint-Malo, France, Sept. 2004)

Lange T, Papenberg N, Heldmann S, Modersitzki J, Fischer B, Lamecher H and Schlag P M 2009 3D ultrasound-CTregistration of the liver using combined landmark-intensity information Int. J. CARS. 4 79–88

Lee D, Nam Kim Y S, Lee J H and Ra J B 2008 Non-rigid registration of 3D ultrasound and CT images in the liverusing intensity and gradient information Proc. CARS 2008 (Barcelona, Spain, June 2008)

Mcgahan J P, Ryu J and Fogata M 2004 Ultrasound probe pressure as a source of error in prostate localization forexternal beam radiotherapy Int. J. Radiat. Oncol. Biol. Phys. 60 788–93

Moore K L and Agur A M R 2007 Essential Clinical Anatomy 3rd edn (Baltimore, MD: Lippincott Williams andWilkins)

Nam W H, Kang D-G, Lee D and Ra J B 2008 Anatomical feature extraction in 3D B-mode ultrasound liver imagesfor CT-ultrasound affine registration Proc. CARS 2008 (Barcelona, Spain, June 2008)

Nam W H, Kang D-G, Lee D and Ra J B 2010 Robust registration of 3-D ultrasound and CT images of the liver forimage-guided intervention IEEE ISBI 2010 (Rotterdam, The Netherlands, April 2010)

Penney G P, Blackall J M, Hayashi D, Sabharwal T, Adam A and Hawkes D J 2001 Overview of an ultrasound to CTor MR registration system for use in thermal ablation of liver metastases Proc. Medical Image Understandingand Analysis 2001 (Birmingham, UK, July 2001)

Penney G P, Blackall J M, Hamady M S, Sabharwal T, Adam A and Hawkes D J 2004 Registration of freehand 3Dultrasound and magnetic resonance liver images Med. Image Anal. 8 81–94

Roche A, Pennec X, Malandain G and Ayache N 2001 Rigid registration of 3-D ultrasound with MR images: a newapproach combining intensity and gradient information IEEE Trans. Med. Imaging 20 1038–49

Rohlfing T and Maurer C R 2003 Volume-preserving nonrigid registration of MR breast images using free-formdeformation with an incompressibility constraint IEEE Trans. Med. Imaging 22 730–41

Rohlfing T, Maurer C R, O’Dell W and Zhong J 2004 Modeling liver motion and deformation during the respiratorycycle using intensity-based nonrigid registration of gated MR images Med. Phys. 31 427–32

Rueckert D, Sonoda L I, Hayes C, Hill D L G, Leach M O and Hawkes D J 1999 Nonrigid registration using free-formdeformations: application to breast MR images IEEE Trans. Med. Imaging 18 712–21

Shimizu S, Shirato H, Xo B, Kagei K, Nishioka T, Hashimoto S, Tsuchiya K, Aoyama H and Miyasaka K 1999Three-dimensional movement of a liver tumor detected by high-speed magnetic resonance imaging Radiother.Oncol. 50 367–70

Treecea G M, Pragera R W, Geea A H and Berman L 2002 Correction of probe pressure artifacts in freehand 3Dultrasound Med. Image Anal. 6 199–214

Wein W, Brunke S, Khamene A, Callstrom M R and Navab N 2008 Automatic CT-ultrasound registration fordiagnostic imaging and image-guided intervention Med. Image Anal. 12 577–85

Wein W, Kutter O, Aichert A, Zikic D, Kamen A and Navab N 2010 Automatic non-linear mapping of pre-procedureCT volumes to 3D ultrasound IEEE ISBI 2010 (Rotterdam, The Netherlands, April 2010)

Xie Z and Farin G E 2001 Deformation with hierarchical B-splines Mathematical Methods for Curves and Surfaces(Nashville, TN: Vanderbilt University) pp 545–54

Zhang W, Noble J A and Brady J M 2006 Real time 3-D ultrasound to MR cardiovascular image registration using aphase-based approach IEEE ISBI 2006 (Arlington, Virginia, USA, April 2006)