Int J CARSDOI 10.1007/s11548-014-1097-6

ORIGINAL ARTICLE

3D femur model reconstruction from biplane X-ray images:a novel method based on Laplacian surface deformation

Vikas Karade · Bhallamudi Ravi

Received: 7 January 2014 / Accepted: 12 June 2014© CARS 2014

AbstractPurpose Conventional methods for 3D bone model recon-struction from CT scans can require high-radiation dose, costand time. A 3D model generated from 2D X-ray images maybe a useful alternative. Reconfiguring a 3D template surfacemesh model to match bone shape in orthogonal radiographsis a common technique for 3D reconstruction. A computa-tionally efficient 3D bone modeling algorithm was developedand tested.Method An algorithm for bone template reconfigurationis proposed, which uses Kohonen self-organizing maps for2D–3D correspondence between input X-ray images and thetemplate. Laplacian surface deformation is then used forfinal deformation of the template. In the literature, Laplaciandeformation has been shown to perform better than thin-platesplines and free form deformation in terms of computationtime and mesh quality. The method was applied to 22 sets ofsimulated input contours generated from 3D models of thedistal femur.Results An acceptable range of reconstruction error: 1.5mm of RMS-P2S (root-mean-square point-to-surface) dis-tance and 1.2 mm mean-P2S distance errors was observedbased on comparison with the corresponding reference mod-els/ground truth. Computation time for the 3D bone modelingalgorithm was less than a minute for each case.Conclusion The new template reconfiguration algorithmbased on Laplacian surface deformation provided acceptablereconstruction accuracy and high computation efficiency for3D modeling of the distal femur using biplane radiographs.

V. Karade (B) · B. RaviOrthoCAD Lab, Department of Mechanical Engineering,Indian Institute of Technology Bombay, Powai, Mumbai 400076,Maharashtra, Indiae-mail: vicky731@gmail.com

This algorithm may provide a useful option for orthopedicmodeling applications.

Keywords 3D reconstruction · X-ray · Laplacian meshdeformation · Self-organizing maps · Medical modelingsoftware

Introduction

Three-dimensional model reconstruction of bone anatomyfrom medical images has been proven to be very useful forpreoperative planning and computer-aided surgeries [1,2].The conventional methods of reconstruction of 3D modelfrom CT or MRI images mostly deal with image process-ing and visualization techniques, and the 3D information isalready present in the images [3]. These methods, however,have drawbacks such as expensive imaging techniques, highvolume of data, more time to generate 3D model and lesseravailability. In addition, CT scanning also involves draw-backs of high radiation. Hence, 3D modeling from two X-rayimages using some prior information about the bone shapeand shape variation has become a topic of research interest.

The methods of 3D reconstruction from X-ray images canbe classified according to the information used for recon-struction. At one extreme, there are methods in which onlythe X-ray images are used as the inputs. In these methods,the contours extracted from the calibrated X-ray images areback-projected to approximate the cross section of the boneshape [4–6]. On the other extreme, there are methods whichuse a mathematical model containing information of the boneshape as well as its variation [7–12]. This is referred to asstatistical shape model (SSM), created by statistical analy-sis of a collection of bone shapes. The SSM has informa-tion about the average shape geometry and shape variations

123

Int J CARS

[13,14]. The average shape can be deformed by applying alinear combination of principal modes of shape variationscontained in the SSM. The deformation is performed untilthe projection of the deformed shape matches the informationfrom X-ray images. Ehlke et al. also included a volume-basedstatistical shape and intensity models (SSIMs), which resultin additional information of volumetric bone density in thereconstructed 3D model [15]. In between the two extremes,there are methods which use the bone shape informationin the form of 3D template bone model [16–19]. The tem-plate model is taken as the initial 3D model which is thenreconfigured/deformed according to the contour informationextracted from the calibrated X-ray images. A method shownby Lee et al. [19] includes scaling, shearing and transfor-mation of different regions of the femur template accordingto the anatomical parameters measured from X-ray images.In methods presented by Laporte et al. [16], the templatebone is elastically deformed using nonstereo correspondingcontours (NSCC) algorithm. This method, however, requiresmanual identification of stereo corresponding points on thecalibrated X-rays, which form the constraints for NSCCalgorithm-based deformation. In a few other methods, thetemplate bone model is deformed using freeform deforma-tion (FFD) technique where the template is assumed to beenclosed in a bounding box that acts as a control volume[18,20]. By modifying the control volume parameters, thetemplate inside can be deformed until its projection boundarymatches the bone contours extracted from the X-ray images.With a single bounding, box only eight parameters (eightcorners of the bounding box) can be modified to deform theembedded shape, which may not be sufficient for getting thedesired shape. Gunay et al. have showed a method based onFFD (Hierarchical FFD) where the bounding box is itera-tively subdivided into smaller boxes, which can be individ-ually modified to get a shape closest to the desired shape[18]. Another FFD-based method shown by Koh et al. [21]had inclusion of some volumetric information in the form ofsparse CT data.

Methods based on inputs from X-ray images alone donot give good accuracy since the images have limited infor-mation. Methods based on SSM have shown more accurateresults but they require a large database. The database canbe classified according to age, gender, and ethnicity. Thetemplate-based methods using FFD for template reconfigura-tion have been considered to be a viable option for developingsoftware applications for 3D reconstruction [20]. However, ithas been shown that volume-based deformation such as FFDwith a few control points is poor in preserving the detailedfeatures of the shape [22]. Hierarchical FFD (for accurateshape matching) requires complex and time-consuming algo-rithms, and does not consider the topology of the shape underdeformation [23]. An alternate method of template reconfig-

uration is desirable that is simpler, faster and preserves thelocal shape features of the template during deformation.

The main objective of this study is to investigate the appli-cation of Laplacian surface deformation (LSD) for templatereconfiguration for 3D reconstruction from two X-ray images(taken at ML and AP directions), applied to femur boneshape. LSD is known for preserving the overall topology andshape features and has been recently researched for com-puter graphics and animation applications [24,25]. In med-ical applications, a study presented by Zhang et al. [26] showsapplication of LSD for 3D surface reconstruction of left ven-tricle from sparse TMRI. LSD is easier to implement andtakes less computation time compared to other deformationmethods such as FFD or TPS (thin-plate spline). The LSDmethod is introduced for the first time in the field of 3D recon-struction from 2D X-ray images. For validation of the recon-figuration method, simulated contours of X-ray projectionimages, i.e., outer contours generated from the projectionsof 3D models (generated from CT data) on ML (Medial-Lateral) and AP (Anterior-Posterior) planes, have been used.The algorithm was also applied to a real set of X-ray imagesto check its feasibility with the real X-ray images.

Methods

The 3D bone model reconstruction problem that we arefocusing on is as follows (Fig. 1):

Inputs: (1) simulated contours of X-ray images of femurbone taken from ML (Medio-Lateral) and AP (Anterio-Posterior) views, and (2) a femur template 3D bone model

Fig. 1 Flowchart showing the overall process of template model recon-figuration

123

Int J CARS

Fig. 2 Detection of boundary points and edges of the projection of the 3D point cloud to generate the input contour

belongs to clinically normal person in the form of 3D meshwith triangular elements.

Output: The template model reconfigured into a shape thatmatches the input contours.

The input contours were generated from projections of 3Dmodels (generated from CT data of femur) on ML and APplanes. This mimics the conditions of biplane X-ray imagingtechnique, which can give two X-ray images at exact 90◦.For a given camera calibration K , i.e., location and orienta-tion of the X-ray source w.r.t. the image acquisition planes(ML and AP), and the given 3D bone model (its centroid),perspective projections were calculated on the image planes.The bone model was in the form of 3D point cloud, andhence, the projection was also a point cloud in 2D imageplane. From the projection points, a 2D mesh with triangularelements was created and the boundary edges of the meshwere extracted by identifying those edges in the 2D mesh,which were shared by only one triangular element. The mesh-ing was performed using Constraint Delaunay Triangulationwhere the edges with length more than a certain threshold (2mm) are eliminated from the meshing obtained from normalDelaunay Triangulation. This method gives a smooth bound-ary curve (edges), since the concave features in bone shapesare much larger than the threshold. Figure 2 shows the mesh-ing of the projected points and detected boundary points. Theset of extracted boundary edges and corresponding projec-tion points together form the input contour of the projection.Input contours extracted in both ML and AP image planeswere the inputs to the algorithm. The second input, i.e., thetemplate model, was in the form of 3D triangular surfacemesh created from CT scans of a clinically normal personusing medical modeling software.

The template model of the femur was truncated accordingto the shortest length of femur measured from the distal end,available in the X-ray images. At first, the ML X-ray imagewas assumed to have the shortest length of femur and takenas the reference. The ratio (ML aspect ratio) of ML width

of the femur to its length was measured on the ML contour.The femur template model was truncated along the shaft tomaintain the measured ML aspect ratio. After truncation, theratio (AP aspect ratio) of AP width of femur template to itstruncated length was measured and the femur contour of theAP X-ray image was truncated accordingly. If the AP contouris found to be shorter, the process was reversed by taking APcontour as the reference.

The template model reconfiguration is performed in threesteps (Fig. 1). The first step involves initial alignment of thetemplate in the image acquisition space. The second stepis to find the 2D–3D correspondence between points of theinput contours and the silhouette vertices (both ML and APplanes). Third step is to deform the template mesh usingLSD according to the calculated 2D–3D correspondences.In this step, the template is deformed in such a way thatthe shape of the projections of the silhouette of the templateon both the image planes (ML and AP) matches that of thecorresponding input contours. The detailed methodology ofeach step is presented next.

Initial alignment of template

For a given input contour and corresponding camera calibra-tion K , perspective projection of the vertices of the templatemesh is calculated on its image plane. The outer contour ofthe template projection (henceforth referred to as templateprojection contour) can be calculated using the same method(with threshold = 5 mm) as used for calculating the inputcontours. The vertices of the template model which corre-spond to the template projection contour points are the sil-houette vertices (projections of the silhouette vertices are thetemplate projection contour points). The template has to bescaled and aligned (similarity transformation) in such a waythat the template projection contour is best aligned/matchedto the input contour without deforming its shape. To simplifythe process, the template projection contour is first aligned

123

Int J CARS

Fig. 3 Determining 3D-3D point pairs (asm , bs

m) according to the positions of the corresponding projection contour points p pm before and after the

initial alignment of template projection contour with the input contour

Fig. 4 Template model a before, and b after initial alignment

with the input contour using iterative closest point (ICP) algo-rithm, and then, the corresponding transformations (transla-tion, scaling and rotation) are applied to the template so thatits projection on the image plane will match with the newlyaligned template projection contour.

To apply corresponding transformations to the template,3D–3D point pairs (Fig. 3) are determined after the finalalignment of template projection with the contour points ofboth ML and AP planes. For each mth (m = 1, . . ., M; M =total number of silhouette vertices) silhouette vertex of thetemplate with its initial position as

m , a closest position bsm on

the projection ray rm joining the X-ray point source and thecorresponding aligned template projection point p p

m is cal-culated. In this way, total M numbers of 3D–3D point pairs

(asm, bs

m) are found. ICP algorithm was applied on these pointpairs (as

m, bsm) to find the transformations of silhouette ver-

tices for their optimal superimposition and applied to thewhole template model. Figure 4 shows the template modelbefore and after the alignment.

In the iterative process of the ICP method, after each stepof the iteration, a new corresponding points pair betweentemplate projection and input contour was determined. Aftereach step of the iteration, the mean absolute distance (MAD)between the points of template projection contour and theircorresponding closest points of the input contour was mea-sured. The iteration was stopped when the difference in MADof the two consecutive steps of iterations was below 0.0001mm. The MAD between the input contour and the template

123

Int J CARS

Fig. 5 Variation of MAD (in mm) with the number of iterations duringthe initial alignment

projection contour is minimized (Fig. 5) through the steps ofthe iteration. The corresponding alignment of the 3D tem-plate was then applied at once. The alignment with respectto the ML plane does not affect the alignment with respectto the AP plane since both the image planes are orthogonal.

2D–3D correspondence

After the initial alignment of the template model, a 2D–3Dcorrespondence is determined between the points of the 2Dinput contour and the silhouette vertices of the aligned 3Dtemplate model (for both ML and AP planes simultaneously).Using this 2D–3D correspondence, the silhouette verticeswill be updated to new positions (desired final positions)such that their projection, i.e., template projection contourwill match with the input contour.

First, a 2D–2D correspondence between the points of tem-plate projection contour points and the input contour pointsis found. We used a nonrigid registration approach insteadof rigid registration-based method like ICP because ICP cangive wrong correspondence for complex contour shapes [27].One of the nonrigid registration methods based on Koho-nen self-organizing maps (SOM) algorithm was successfullyapplied by Ferrarini et al. in their GAMEs approach to find3D–3D shape correspondence [28,29]. We applied a similarmethod to find 2D shape correspondence. The template pro-jection contour points (p p) are adapted onto the input contourpoints (pc) using SOM algorithm. After the adaptation, thetemplate projection contour points represent the shape of theinput contour. The number of the template projection contourpoints and their topology (connectivity) is preserved in theSOM method. Hence, the positions of the template projec-tion contour points before and after the adaptation give therequired 2D-2D correspondence.

The use of the SOM algorithm allows smooth changes inthe shape formed by the template projection contour points.In this method, for each input contour point, a best match-ing template projection contour point p p

winner (nearest tothe input contour point) is determined and its position isupdated toward the input contour point (Eq. 1). When the

Fig. 6 Variation of MAD (in mm) with the number of iterations duringSOM method

template projection contour adapts to the input contour, themotion of the best matching template projection contourpoint p p

winner affects the neighbor template projection contourpoints as well. This is controlled by the neighborhood func-tion n(p p

winner, p pm), which is an exponential function whose

value is high for the template projection contour points thatare closer to the p p

winner and small for points which are fartheraway (Eq. 2). The neighborhood function is responsible fortopology preservation during the adaptation. The adaptationof all the projection contour points is performed with respectto every input contour point. The adaptation of every templateprojection contour point and its effect on the neighbor pointsdecrease exponentially. This is controlled by the learning ratel(t), which is a function that makes the adaptation processdie smoothly with time. In our method, the learning rate con-stant decreased from 0.5 to 0.1. The whole process (adapta-tion of template projection contour points with respect to allthe input contour points) is also repeated through number ofcycles (iterations) until the MAD value between the points oftemplate projection contour and their corresponding closestpoints of the input contour goes below a threshold (0.15 mmin our case). Figure 6 shows an example of variation in theMAD which was calculated after the adaptation of all thetemplate projection contour points with respect to each inputcontour point through all the iterations.

The steps involved are as follows:For each (kth) input contour point pc

k(k = 1, . . ., K ; K =total number of contour points):

1. Calculate distance between the input contour point pck

and all the template projection contour points

dk,m = ∣∣∣∣ pc

k − p pm∣∣∣∣ (m = 1, . . . , M;

M = total number of projection points) (1)

2. Select the nearest template projection contour point asthe winner p p

winner

p pwinner : dk,winner = min

(

dk,m)

123

Int J CARS

Fig. 7 Determining the desired final positions of the template silhouette vertices by 3D–3D point pair calculation method

3. Update each template projection contour point usingEq. (2):

p p′m = p p

m + l(t) ∗ n(p pwinner , p p

m) ∗ (

pck − p p

m)

(2)

In Eq. (2),

l(t) = 0.5 + t f rac ∗ (0.1 − 0.5)

where,

t f rac = current iteration number/total number of

iterations

n(p pwinner, p p

m) = expdistwinner,m

2∗σ2

where,

distwinner, m = ∣∣∣∣ p p

winner − p pm∣∣∣∣ and

σ = 3 + t f rac ∗ (0.1 − 3.0)

The output of SOM method was the adapted template pro-jection contour points (p p′) onto the input contour. The tem-plate projection contour points before and after the adapta-tion represent the required 2D–2D correspondence. As thetemplate projection contour points are directly associatedwith the silhouette vertices (projection), the 2D–2D cor-respondence (i.e., which template projection contour pointcorrespond to which input contour point) directly gives therequired 2D-3D correspondence (i.e., which silhouette ver-tex of template correspond to which input contour point).

Using the 2D–3D correspondence, the silhouette vertices areupdated to their desired final positions in such a way that theirprojections represent the shape of the input contours. The cor-responding desired final positions vs′

m of the mth silhouettevertices of the template with initial positions vs

m are deter-mined using the same 3D-3D point pair calculating methodused for template alignment (Fig. 7). For an mth adapted tem-plate projection contour point p p′

m (which is lying on the inputcontour), a projection ray rm is determined starting from theX-ray point source meeting the point p p′

m itself. A new posi-tion vs′

m closest to a corresponding mth silhouette vertex withinitial position vs

m is found on the updated projection ray. Thenew position vs′

m is the desired final positions of the mth sil-houette vertices. During template deformation, the silhouettevertices are updated to their desired final positions, accordingto which all other vertices of the template are also updatedwhile preserving the overall shape features. This procedureof template deformation was carried out using Laplacian sur-face deformation, as explained in the next section.

Template deformation

The next step is to deform the transformed template meshin such a way that the silhouette vertices get their desiredfinal position (determined in the previous step) while pre-serving the overall topology and differential property of thetransformed template (Fig. 8). This step is performed usingLaplacian surface deformation (LSD). Each vertex of a meshis represented as a differential coordinate, which is the differ-ence between the position of vertex and that of its neighborvertices [24,25]. In general, the inputs are the initial mesh,a set of anchor points (a few vertices of the initial mesh)

123

Int J CARS

Fig. 8 Template model a before, and b after deformation

Fig. 9 The vector di of differential coordinates at vertex i of the tem-plate mesh model

and target positions of the anchor points. The output is adeformed mesh where the anchor points take the target posi-tions while preserving the local shape features and topologyof the initial mesh. For the template deformation, the tem-plate mesh model is input as the initial mesh, the silhouettevertices (with initial positions vs) are the anchor points, andthe desired final positions of the silhouette vertices (i.e., vs′)are the target positions of the anchor points.

Let the mesh M be described by a pair (V, E), where V =vi ∈ R3|1 < i < n, describes the geometric positions ofvertices and E = {(i, j)|vi is connected to v j } describes theconnectivity. The differential coordinate di for each vertex vi

is defined as the vector from the coordinates of the centroidof its immediate neighbors to its coordinates (Fig. 9).

di = L(vi ) = (d(x)i , d(y)

i , d(z)i )

= vi − 1

|N (i)|∗∑

j∈N (i)

wi, j ∗ v j (3)

where,

N (i) = { j | (i, j) ∈ E}|N (i)| = number of immediate neighbors of iwi, j = weight for point p j such that

∑

j∈N (i) wi, j = 1In our case, wi, j = 1/|N (i)|L is a Laplace operator of M

Assume that vs′m(m ∈ {1, . . ., M}) are the deformed posi-

tions for a subset of the N mesh vertices vsm(m ∈ {1, . . ., M}).

These are actually the desired positions of the silhouette ver-tices which were calculated during 3D–3D point pairs cal-culation. The information of the deformed positions can beused to compute a new (deformed) Laplacian coordinate d ′

i

for each vertex i of the mesh. The deformed positions of themesh vertices v′

i are then obtained by solving the followingquadratic minimization problem:

V = minv′

i

(n

∑

i=1

∥∥L(v′

i ) − d ′i

∥∥

2 + α

M∑

m=1

∥∥vs′

m − vsm

∥∥

2

)

(4)

The first term represents the preservation of local detail, andthe second constrains the positions of those vertices directlyspecified by the user. The parameter α (1 in our case) balancesthe two objectives.

To perform the Laplacian mesh deformation, the Carte-sian coordinates of the mesh vertices are transformed intodifferential coordinates by matrix algebra. A is defined as asquare matrix of size n × n. The matrix elements are set asfollows.

A (i, j) = −1/ |N (i)| , if vertex vi and vertex vj are connected

by an edge

A (i, j) = 0, if vertex vi and vertex vj are not connected

by an edge

A (i, i) = 1

123

Int J CARS

The matrix D of the differential coordinates di is thencalculated using the following equation

D = A ∗ V (5)

The matrix A is singular, and hence, the expression V =A−1 ∗ D is undefined. In order to restore the coordinates ofV , a full-rank system has to be solved. Let there be M numberof anchor points m(m = 1, . . ., M) whose initial positionsare vs

m and the corresponding target positions are vs′m (desired

final positions of the silhouette vertices). Total M additionalconstraints shown in Eq. (6) are assumed and treated in least-square sense: Instead of substituting them in the linear systemof Eq. (5), they are added as additional rows of the linearsystem Eq. (7). The linear system now looks like that inEq. (8) can be denoted as A.

vsm = vs′

m (m = 1, . . . , M) (6)

A(n + m, cm) = w and D(n + m) = w ∗ vs′m (7)

(cm = column belonging to the silhouette vertex m in originalL)

(

Aw ∗ I

)

∗ V ′ =(

Dw ∗ V s′

)

(8)

(I is index matrix of size M × n where, I (m, i) = 1 if i = cm,

otherwise 0V s = (vs

1, . . ., vsM ) is a set of initial positions of the

anchor points mV s′ = (vs′

1 , . . ., vs,M ) is a set of target positions of the

anchor points mV ′ = (v′

1, . . ., v′n) is a set of final updated positions of

the template vertices after deformationIn Eq. (7), V ′ is the deformed mesh. The weights w are set

according to the importance of the positional constraints (w =1 in this case). The (n + M) × n linear system that becomesover-determined (more equations than unknowns) becauseof the additional constraints is now a full-rank system andhence can be solved in the least-square sense (Eq. 4). Thefirst term of the equation tries to preserve the topology anddifferential property, and the second term brings the anchorpoints to the target positions (i.e., silhouette vertices to theirdesired positions).

The analytical least-square solution is expressed in termsof the (n + M) × n system matrix A from Eq. (8), as shownin Eq. (9). In Eq. (9), b is the right-hand side vector of linearsystem.

V = (AT ∗ A)−1 ∗ A

T ∗ b (9)

where, b = (D, w∗vs′1 , . . ., w∗vs′

M )T

The output of the Laplacian deformation is the deformedtemplate mesh with its silhouette vertices updated to their

desired positions whose projection matches the shape of theinput contour. Hence, the deformed template is the final 3Dreconstructed model matching the input contour.

Experiments and results

Three experiments were performed to evaluate the perfor-mance of the proposed 3D reconstruction algorithm. In thefirst experiment, 3D models were generated from 22 sets ofsimulated input contours of distal femur bone shape, usingthe proposed algorithm. To measure the reconstruction accu-racy, the generated 3D models were compared with the ref-erence 3D models of femur bone, which were used to gener-ate the simulated input contours using the method explainedearlier. The reference 3D surface models were created fromthe CT scans using medical modeling software (MIMICS15.0, Materialize, Belgium). All the patients were undergo-ing total knee arthroplasty during the period February 2010to August 2011. The mean age of the patients was 65 yearswith a range of 25–82 years. The CT scans were taken usingSimens_/Emotion_Duo (130 KV, 53 mAs) with slice thick-ness of 1 mm, slice increment of 0.5 mm and a resolutionof 512 × 512 pixels. The template mesh model required bythe algorithm was generated from 3D surface model createdfrom CT scan images of a clinically normal person, using themedical modeling software. The surface model was exportedas a dense point cloud (1 mm point-to-point distance), whichwas then uniformly sampled to sufficient number of points.The sufficient number of points was determined by measur-ing the level of detail of the 3D bone model. The level ofdetail and the volume get reduced after the sampling. Thereduction in level of detail can be determined by measuringthe difference in volume of a closed mesh created from theinitial dense point cloud and that of a closed mesh createdfrom the sampled points. By putting the threshold on the levelof detail (volume reduction of 2 %), the sampling and hencethe sufficient numbers of points were determined. The point-to-point distance at this condition was 3 mm. A 3D mesh withtriangular elements was created from the sampled points andused as the template model for the 3D reconstruction. Thetemplate model was in the form of triangular surface meshwith sets of 1,795 vertices and 3,559 faces.

In the second experiment, the algorithm was applied tofive sets of real X-ray images of distal femur of right kneejoint. The X-ray images included the region around the kneejoint of a clinically normal person (at knee region) taken inML and AP view. The ML image was taken by keeping theleg position in such a way that patella was facing upward. Thedistance between the X-ray source and the image plane wasalso predetermined (1,000 mm). The AP image was takenby rotating the leg around its axis at 90◦ (measured usinggoniometer) with respect to the position of the leg when the

123

Int J CARS

Table 1 Results of the 3D model generation from 22 sets of simulated contours and 6 sets of real X-ray image test cases (Experiment 1 and 2),using the template of normal person

Sr. no. Input Numberof cases

Mean MP2Serror

Mean RP2Serror

Mean MaxP2Serror

Mean VarP2Serror

Mean of computationtime (s)

1 Simulated contours 22 1.2 1.5 6.7 0.7 46

2 Real X-ray images 5 1.2 1.4 5.6 0.5 52

All errors are in mm

Table 2 Results of the leave-one-out test (Experiment 3) for 3D model generation from 22 sets of simulated contours and 6 sets of real X-rayimage test cases

Sr. no. Input Numberof cases

Mean of meanMP2S error

Mean of meanRP2S error

Max of meanMP2S error

STDEV of meanMP2S error

Mean of computationtime (s)

1 Simulated contours 22 1.2 1.5 1.6 0.1 25

2 Real X-ray images 5 1.3 1.6 1.6 0.1 58

All errors are in mmSTDEV standard deviation

ML image was taken. This experiment will check the accu-racy of the algorithm with the real X-ray images taken usingsingle plane imaging technique. The outer contours on the X-ray images were drawn manually using a drawing tool, andthe coordinates of the contour points were given as inputs tothe algorithm. The reference 3D surface model for evaluat-ing the accuracy was created from the CT scans of the sameperson whose X-rays were used as inputs, using the medicalmodeling software. The template used in this experiment isthe same, which was used in experiment 1.

In the third experiment, we performed leave-one-out testwith both simulated contours and the real X-ray images, tocheck the dependency of the results on the template shape.Out of the 22 sets of patients’ data, each time one of thepatient’s 3D reference model was used to create the templateshape to generate 3D models from the X-ray images of therest of the 21 patients. This way the 3D generation processwas repeated 22 times using 22 difference template modelseach time. The template models were in the form of triangularsurface mesh with 1,347 vertices and 2,644 faces on average.Similarly, the 3D generation process with the five sets of realX-ray images was repeated five times using five differencetemplate models each time. The template models were inthe form of triangular surface mesh with 1,602 vertices onaverage (the template model is truncated according to thefemur length available in X-ray images). All the experimentswere carried out on a workstation with Intel Core 2 Quad 2.40GHz processor. All the programming was performed usingMATLAB.

To measure the reconstruction accuracy, the reconstructed3D models were superimposed with the corresponding refer-ence models. The superimposition included rigid geometri-cal transformation and the least-square matching using ICP

method. Point-to-surface (P2S) distance for a point of thereconstructed model is defined as the normal absolute dis-tances from the point to the surface of the corresponding ref-erence model. For the experiments 1 and 2, performed usingthe normal human template, the mean, RMS (root-mean-square), maximum, and variance of the P2S distances for allthe points of the reconstructed model were measured for eachreconstructed model and referred as MP2S, RP2S, MaxP2S,and VarP2S errors, respectively, for that reconstructed model.The mean values of the MP2S, RP2S, MaxP2S, and VarP2Serrors of all the reconstructed models (22 models from simu-lated contours and five models from real X-ray images) werecalculated. These results are summarized on Table 1. The firstrow shows the results of 3D model generation from the 22sets of simulated contours. The second row shows the resultsof 3D model generation performed using the six sets of thereal X-ray images. The mean computation time was calcu-lated in seconds. The mean MP2S for both simulated and realX-ray images was around 1.2 mm with variance of around0.5 mm. Time taken for the reconfiguration of the templatewas less than a minute.

For the third experiment (both with simulated and real X-ray images), the average of the mean values of MP2S andRP2S of all the reconstructions performed using all the tem-plates (22 in case of simulated contours and 5 in case ofreal X-ray images) were calculated. The maximum and stan-dard deviation of the mean values of MP2S error of all thereconstructions performed using all the template were alsocalculated. These results are summarized on Table 2. Thefirst row shows the results of 3D model generation from thesimulated contours, and the second row shows the same per-formed using the real X-ray images. The mean of the MP2Serror for the reconstructed models from all the templates was

123

Int J CARS

Fig. 10 Distribution of the P2Sdistance error at various regionsof the distal femur bone

1.2 mm for simulated contours and 1.3 mm for the real X-rayimages. The standard deviation of the mean MP2S error wasaround 0.1 mm.

Discussion

An application of self-organizing maps (SOM) and Laplacianmesh deformation for an accurate and efficient method for3D bone model reconstruction from two 90◦ X-ray imagesusing a template mesh model has been presented. The 2D–3D correspondence determined using SOM gives the anchorpoints for the Laplacian mesh deformation. Laplacian meshdeformation preserves the mesh topology and shape features,which is the most important reason to choose it as a templatedeformation technique. During deformation, the templatemesh does not get distorted and the differential coordinatesare preserved due to differential constraints in the Laplacianmethod. Hence, the differential features of the regions ofthe template mesh which cannot be updated using any infor-mation from the input contours (when those regions are notvisible in the X-ray image contours) remain preserved duringLaplacian deformation. Laplacian deformation is also easierto apply and take less computation time compared to FFDand TPS.

In experiment 1, the algorithm was applied to the orthog-onal simulated contours, which mimic the biplane imagingtechnique. Experiment 2 shows the feasibility of the algo-rithm in real situations where it may be difficult to obtainexact 90◦ X-ray images using single plane imaging tech-nique. In controlled conditions, however, this is still possi-ble as demonstrated in experiment 2. There may be someerrors when goniometer was used to measure the angle andthe leg was rotated about the axis, to get exact orthogonalX-ray images. However, the results were similar to thoseobtained using the exactly orthogonal simulated contours.Experiment 3 was performed to check the dependency of the

results on the template shape used for the 3D model genera-tion using both simulated contours and the real X-ray images.The reconstruction errors in experiment 3 were similar tothose obtained in the experiments 1 and 2 performed usingthe template shape of normal human, with a very small vari-ance of 0.1 mm.

Figure 10 shows the distribution of the P2S distance errorat various regions of the distal femur bone, calculated byaveraging the distance errors on the vertices of the templatebone reconfigured for all the 22 3D model reconstructions.The reconstruction algorithm was applied to the right kneefemur bone shape. The proposed feature-based algorithm isdriven by the outer contour shape of the bone. In AP viewof the right femur, the outer contour does not include theupper medial region of the lateral condyle, because the lat-eral condyle is always lower than the medial condyle. There-fore, shape information of this region is usually not availablein the input contour for reconfiguring the template model.Hence, the error was the highest in the upper medial sideregion of the lateral condyle. These errors are much higher ifa single imaging view is used as input. For complex regionsor cavities perpendicular to imaging direction, the methodwill not work accurately. Hence, it is required to have twoorthogonal biplanar X-ray images from ML and AP view asinputs. This may solve the problem of nonvisibility of cav-ities perpendicular to the imaging direction in certain cases.For example, the distal intercondylar notch which is not visi-ble in the AP view is visible in the contour of ML view whichgives the positional constraint in LSD. However, the depthof the posterior intercondylar notch cannot be captured bythe contours of both the views. The final results also directlydepend on the 2D–2D correspondence. In addition, the topol-ogy and shape features preservation properties of the Lapla-cian Surface Deformation which are used for the templatereconfiguration are also being investigated in this study.

The proposed method was validated using 22 sets of sim-ulated contours generated from the projections of 3D mod-

123

Int J CARS

Tabl

e3

Com

pari

son

ofdi

ffer

entm

etho

dsof

3Dre

cons

truc

tion

from

X-r

ayim

ages

appl

ied

knee

bone

shap

es

Sr.n

o.A

utho

rM

etho

dSh

ape

Inpu

t(re

alor

sim

ulat

ed)

Con

tour

gene

ratio

nN

umbe

rof

case

sM

ean

P2S

erro

rR

MS

P2S

erro

rC

ompu

tatio

ntim

e(s

)Sy

stem

confi

gura

tion

Lev

elof

deta

il

1K

arad

eet

al.

LSD

-SO

MD

ista

lfem

urR

eal

Man

ual

51.

21.

452

Inte

l,2.

4G

Hz,

8GB

1,79

5

2K

arad

eet

al.

LSD

-SO

MD

ista

lfem

urSi

mul

ated

Aut

omat

ic22

1.2

1.5

46In

tel,

2.4

GH

z,8G

B1,

795

3Fi

lippi

etal

.[20

]FF

DFu

llfe

mur

Sim

ulat

edM

anua

l5

1.4

––

––

4G

unay

etal

.[18

]FF

DT

ibia

Rea

l–

––

–10

0A

MD

,1G

Hz

1,90

6

5B

aka

etal

.[8]

SSM

Dis

talf

emur

Rea

lA

utom

atic

(can

ny)

10–

1.68

300

2.4

GH

z,2

GB

931

5Z

huet

al.[

11]

SSM

Dis

talf

emur

Rea

lM

anua

l10

0.9

178

Inte

l2.6

7G

Hz,

9G

B30

00

6Fl

eute

etal

.[12

]SS

MD

ista

lfem

urSi

mul

ated

Aut

omat

ic–

–l.0

60N

A50

0

7G

amag

eet

al.[

23]

TPS

Full

fem

urR

eal

Aut

omat

ic6

0.9

––

––

8L

apor

teet

al.[

14]

NSC

CD

ista

lfem

urR

eal

Sem

i-au

tom

atic

8–

1.4

––

556

9Ta

nget

al.[

9]H

ybri

dat

las

Dis

talf

emur

Sim

ulat

edA

utom

atic

22.

0–

––

–

Err

ors

are

inm

m;S

yste

mco

nfigu

ratio

n:pr

oces

sor,

spee

d,R

AM

;Lev

elof

deta

il:nu

mbe

rof

poin

tsIt

alic

valu

esar

eth

ose

ofou

rst

udy

els in ML and AP view, with known calibration parameters.The real X-ray images acquired using a single plane imagingtechnique were also used to check the feasibility of the algo-rithm with real situations. The results were similar with bothsimulated and the real input images. The validation proce-dure with simulated contours is similar to the one presentedby Filippi et al. [20], which was based on hierarchical FFD(free form deformation) performed on five projection con-tours. Our method showed similar accuracy (mean P2S error= 1.2 mm) compared to the method presented by Filippi et al.(mean P2S error = 1.4 mm). In their method, complete femurshape was reconstructed. The computation time required byour algorithm to create the final 3D shape is less (49 s) com-pared to that required by FFD-based methods (100 s) [18].However, the system configuration was different. The methodpresented by Gunay et al. [18] was applied to complete tib-ial shape, but the number of vertices used to represent theshape (1,906) was comparable to that used in our method(1,795). A method presented by Baka et al. [8] based on sta-tistical shape models applied on distal femur showed accu-racy of the mean RMS error of 1.68 mm. This method wasapplied on 90◦ biplane low-quality fluoroscopic images ofdistal femur and required around 300 s of computation time.A method by Zhu et al. [11] also based on statistical shapemodels applied on distal femur showed accuracy of the aver-age error close to 1 mm with standard deviation of 1.5 mmwith computation time of 178 s, applied on 10 cases. AnotherSSM-based method by Fleute et al. showed RMS error of 1mm with computation time of less than a minute [12]. How-ever, it is not clear that how many cases have been studied.Gamage et al. [23] presented a method based on iterativenonrigid 2D point-matching process and thin-plate spline-based deformation applied on 6 cadaver complete femursand showed Euclidean distance error of 0.9 mm. Very few3D reconstruction methods applied to distal femur shape havereported accuracy comparable to that in our method. Laporteet al. [14] applied their semi-automated method called asnonstereo corresponding contour (NSCC) and reported themean RMS error of 1.4 mm on eight femur shapes, which issame as in our method. In a method presented by Tang et al.[9], a hybrid atlas was fitted to three 2D projection images,and the mean RMS error was about 2 mm.

The proposed method gives comparable accuracy for thesame shape (distal femur). However, the computation timecannot be compared directly because it is also depending onthe system configuration, level of detail (number of surfacepoints), and length of distal femur bone. Table 3 summarizesthe comparison of the proposed method with other publishedmethods (our algorithm is referred to as LSD-SOM).

The simulated outer contours of experiment 1 are compa-rable to the outer contour which could be calculated usingreal biplane orthogonal X-ray images. In case of real X-rayimages, also the contours were drawn manually. This ignores

123

Int J CARS

the reconstruction error which may be included due to errorsin identifying outer contours from a real X-ray image. Thereis a need of automatic contour detection from the input X-rayimages. Also, the calibration parameters (K ) were alreadyknown in simulated contours, which simulate the biplaneorthogonal X-ray imaging technique. In case of real X-rayimage, it will require a special effort (e.g., goniometer tomeasure 90◦ angle) and standardized conditions (e.g., 1,000mm distance between source and image plane) to get theseconditions (90◦ X-ray images) as demonstrated in experi-ment 2, which showed the similar accuracy as shown in theexperiment 1 with simulated contours.

In summary, a novel algorithm for reconfiguring the tem-plate model to reconstruct 3D model of bone shape from afew X-ray images has been presented here. Since the algo-rithm uses a template model and outer contour from the X-ray image, it can be applied to any part of the skeleton. Thedistal femoral region of arthritic knees (used in the study)were deformed or worn out due to the disease which leadsto misalignment of knee axes. Our method can also be eas-ily applied to the less complex shape of tibia bone also. A3D model of complete knee joint can help to visualize themisalignment in the knee joint axes. Once the calibrationparameters and the input contours are provided, the rest ofthe process is totally automatic. The acceptable reconstruc-tion accuracy and computation time less than a minute andno need for manual intervention (in the template reconfigu-ration process) make the algorithm a good option for use inmedical software.

Acknowledgments The authors would like to thank Dr. Vijay Shetty(Hiranandani Hospital, Mumbai), Dr. Raju Sharma and Dr. Devasen-athipathy (AIIMS, Delhi) for providing the medical imaging data for thestudy. The authors would also like to thank Mr. Anurag Khaire (M.Tech,IIT-Bombay, India), Mr. Darshan Shah (M.Tech, IIT-Bombay, India)and Mrs. Hepsiba Seeli (Research Assistant, IIT-Bombay) for assistingin creating 3D models from the CT data.

Conflict of interest Vikas Karade and Bhallamudi Ravi declare thatthey have no conflict of interest.

References

1. Otomaru I, Nakamoto M, Kagiyama Y, Takao M, Sugano N,Tomiyama N, Tada Y, Sato Y (2012) Automated preoperative plan-ning of femoral stem in total hip arthroplasty from 3D CT data:atlas-based approach and comparative study. Med Image Anal16(2):415–426

2. Ellis RE, Tso CY, Rudan JF, Harrison MM (1999) A surgical plan-ning and guidance system for high tibial osteotomy. J ComputAided Surg 4(5):264–274

3. Kim YH, Kim JK, Choi C (2004) Three-dimensional reconstructionof human femur using consecutive computer tomography imagesand simulated implantation system. J Med Eng Technol 28(5):205–210

4. Zhang B, Sun S, Sun J, Chi Z, Xi C (2010) 3D reconstructionmethod from biplanar radiography using dlt algorithm: applicationto the femur. In: Proceedings of 1st international conference onpervasive computing signal processing and applications (PCSPA2010); 2010 Sep 17–19; Harbin, China, pp 251–254

5. Caponetti L, Fanelli AM (1990) 3D Bone reconstruction from twoX-Ray views. In: Proceedings of twelfth annual international con-ference of the IEEE engineering in medicine and biology society(EMBS 1990); 1990 Nov 1–4; Philadelphia, PA, USA, pp 208–210

6. Fuente M, Schkommodau E, Lutz P, Neuss M, Wirtz DC, Raderma-cher K (2005) 3D reconstruction and navigated removal of femoralbone cement in revision THR based on few fluoroscopic images.In: Proceedings of computer assisted radiology and surgery (CARS2004); 2005 June 23–26; Chicago, USA, pp 626–631

7. Zheng G, Gollmer S, Schumann S, Dong X, Feilkas T, GonzálezBallester MA (2009) A 2D/3D correspondence building method forreconstruction of a patient-specific 3D bone surface model usingpoint distribution models and calibrated X-ray images. Med ImageAnal 13(6):883–99

8. Baka N, Kaptein BL, de Bruijne M, van Walsum T, Giphart JE,Niessen WJ, Lelieveldt BP (2011) 2D–3D shape reconstruction ofthe distal femur from stereo X-ray imaging using statistical shapemodels. Med Image Anal 15(6):840–850

9. Tang T, Ellis R (2005) 2D/3D deformable registration using ahybrid atlas. In: Proceedings of medical image computing andcomputer-assisted intervention (MICCAI 2005); 2005 Oct 26–29;Palm Springs, CA, USA. Springer, Berlin, pp 223–230

10. Benameur S, Mignotte M, Parent S, Labelle H, Skalli W, de Guise J(2003) 3D/2D registration and segmentation of scoliotic vertebraeusing statistical models. Comput Med Imaging Graph 27(5):321–337

11. Zhu Z, Li G (2011) Construction of 3D human distal femoral sur-face models using a 3D statistical deformable model. J Biomech44(13):2368–2362

12. Fleute M, Lavallée S (1999) Nonrigid 3-D/2-D registration ofimages using statistical models. In: Proceedings of the second inter-national conference on medical image computing and computer-assisted intervention (MICCAI ’99), 1999, pp 138–147

13. Hraiech N, Boichon C, Rochette M, Marchal T, Horner M (2010)Statistical shape modeling of femurs using morphing and principalcomponent analysis. J Med Devices 4(2):027534–027534

14. Bredbenner TL, Eliason TD, Potter RS, Mason RL, Havill LM,Nicolella DP (2010) Statistical shape modeling describes variationin tibia and femur surface geometry between Control and Inci-dence groups from the osteoarthritis initiative database. J Biomech43(9):1780–1786

15. Ehlke M, Ramm H, Lamecker H, Hege HC, Zachow S (2013)Fast generation of virtual X-ray images for reconstruction of 3Danatomy. IEEE Trans Vis Comp Graph 19(12):2673–2682

16. Laporte S, Skalli W, de Guise JA, Lavaste F, Mitton D (2003)A biplanar reconstruction method based on 2D and 3D contours:application to the distal femur. Comput Methods Biomech BiomedEng 6(1):1–6

17. Le Bras A, Laporte S, Bousson V, Mitton D, De Guise JA, LaredoJD, Skalli W (2004) 3D reconstruction of the proximal femur withlow-dose digital stereoradiography. Comput Aided Surg 9(3):51–57

18. Gunay M, Shim MB, Shimada K (2007) Cost- and time-effectivethree-dimensional bone-shape reconstruction from X-ray images.Int J Med Robot 3(4):323–335

19. Lee MK, Lee SH, Kim A, Youn I, Lee TS, Hur N, Choi K (2008)The study of femoral 3D reconstruction process based on anatom-ical parameters using a numerical method. J Biomech Sci Eng3(3):443–451

20. Filippi S, Motyl B, Bandera C (2008) Analysis of existing methodsfor 3D modeling of femurs starting from two orthogonal images

123

Int J CARS

and development of a script for a commercial software package.Comput Methods Prog Biomed 89(1):76–82

21. Koh K, Kim YH, Kim K, Park WM (2011) Reconstruction ofpatient-specific femurs using X-ray and sparse CT images. ComputBiol Med 41(7):421–426

22. Feng J, Shao J, Jin X, Peng Q, Forrest AR (2006) Multiresolu-tion free-form deformation with subdivision surface of arbitrarytopology. Visual Comput 22(1):28–42

23. Gamage P, Xie, SQ, Delmas, P.; Xu, P (2009) 3D reconstructionof patient specific bone models from 2D radiographs for imageguided orthopedic surgery. In: Proceedings of international con-ference on digital image computing: techniques and applications(DICTA 2009), 2009 Dec 1–3; Melbourne, VIC, pp 212–216

24. Masuda H, Yoshioka Y, Furukawa Y (2007) Preserving formfeatures in interactive mesh deformation. Comput Aided Design39(5):361–368

25. Zhang S, Huang J, Metaxax D (2011) Robust mesh editing usingLaplacian coordinates. Graph Model 73(1):10–19

26. Zhang S, Xiaoxu W, Metaxas D, Ting C, Axel L (2009) LV surfacereconstruction from sparse TMRI using Laplacian surface defor-mation and optimization. In Proceedings of international sympo-sium of biomedical imaging: from nano to macro (ISBI 2009) 2009Jun 28–Jul 1, Boston, pp 698–701

27. Heimann T, Meinzer HP (2009) Statistical shape models for3D medical image segmentation: a review. Med Image Anal13(4):543–563

28. Kohonen T (1982) Self-organised formation of topologically cor-rect feature maps. Biol Cybern 43:59–69

29. Ferrarini L, Olofsen H, Palm WM, van Buchem MA, Reiber JH,Admiraal-Behloul F (2007) GAMEs: growing and adaptive meshesfor fully automatic shape modeling and analysis. Med Image Anal11(3):302–314

123