Three-dimensionalModel-basedSegmentation of Brain MRI...Three-dimensionalModel-basedSegmentation of...
10
Three-dimensional Model-based Segmentation of Brain MRI Andr´ as Kelemen, G´ abor Sz´ ekely, Guido Gerig Swiss Federal Institute of Technology Communication Technology Laboratory ETH-Zentrum, CH-8092 Zurich, Switzerland [email protected] Abstract This paper presents a new technique for the automatic model-based segmentation of 3-D objects from volumetric image data. The development closely follows the seminal work of Cootes et al. [5] but presents various new solutions to come up with a true 3-D technique rather than a slice- by-slice 2-D processing. The segmentation system includes both the building of statistical models and the automatic segmentation of new image data sets via a restricted elas- tic deformation of models. Geometric models are derived from a sample set of image data which have been segmented by experts. The surfaces of these binary objects are con- verted into a parametric surface net which is normalized to get an invariant object-centered coordinate system. Sur- face descriptions are expanded into series of spherical har- monics which provide parametric representations of object shapes. Gray-level information is represented by 1-D pro- files normal to the surface. The alignment is based on the well-accepted stereotactic coordinate system since the driv- ing application is the segmentation of brain objects. Shape statistics are calculated from the parametric shape repre- sentations rather than from the spatial coordinates of sets of points. After initializing the mean shape in a new data set on the basis of the alignment coordinates, the model elasti- cally deforms in accordance to displacement forces across the surface but is restricted only by shape deformation con- straints. The technique has been applied to segment left and right hippocampal structures from a large series of 3-D magnetic resonance scans taken from a schizophrenia study. 1 Introduction Segmentation of anatomical objects from large 3D med- ical data sets, obtained from routine Magnetic Resonance Imaging (MRI) examinations, for example, represents a necessary yet difficult issue in medical image analysis. In some limited applications, segmentation can be achieved with minimal user interaction by applying simple and effi- cient image processing methods, which can be applied rou- tinely [7]. In many clinical applications such as computer assisted neurosurgery or radiotherapy planning, a large number of organs have to be identified in the radiological data sets. While a careful and time-consuming analysis may be ac- ceptable for outlining complex pathological objects, no real justification for such a procedure can be found for the de- lineation of normal, healthy organs at risk. Delineation of organ boundaries is also necessary in var- ious types of clinical studies, where the correlation between morphological changes and therapeutical actions or clini- cal diagnosis has to be analyzed. In order to get statisti- cally significant results, a large number of data set has to be segmented. For such applications manual segmentation becomes questionable not only because of the amount of work, but also with regard to the poor reproducibility of the results. The necessity of obtaining high reproducibility and the need to increase efficiency motivates the development of computer-assisted, automated procedures. Elastically deformable contour and surface models, so- called snakes [8], have been proposed as tools for support- ing manual object delineation. While such procedures can be extended to 3-D [24, 3], their initialization is a critical issue. Most often, the initial guess must be very close to the sought contour to guarantee a satisfying result [14]. An excellent overview of elastically deformable models can be found in [11]. The primary reason for the need of a precise snake initialization is the presence of disturbing attractors in the image. These do not belong to the desired object contour but force the snake into local energy minima. If the deformation of a snake could be limited to shapes within the normal variation of a class of object boundaries, the proce- dure would be more robust. Elastically deformable parametric models offer a straightforward way for the inclusion of prior knowledge in the image segmentation process. This is done by incor-
Transcript of Three-dimensionalModel-basedSegmentation of Brain MRI...Three-dimensionalModel-basedSegmentation of...
Thr ee-dimensionalModel-basedSegmentationof Brain MRI
AndrasKelemen,GaborSzekely, GuidoGerigSwissFederalInstituteof Technology
CommunicationTechnologyLaboratoryETH-Zentrum,CH-8092Zurich,Switzerland
Abstract
This paperpresentsa new techniquefor the automaticmodel-basedsegmentationof 3-D objectsfrom volumetricimage data. Thedevelopmentcloselyfollows the seminalworkof Cootesetal. [5] but presentsvariousnew solutionsto comeup with a true 3-D techniquerather than a slice-by-slice2-D processing. Thesegmentationsystemincludesboth the building of statistical modelsand the automaticsegmentationof new image data setsvia a restrictedelas-tic deformationof models. Geometricmodelsare derivedfroma samplesetof imagedatawhich havebeensegmentedby experts. Thesurfacesof thesebinary objectsare con-vertedinto a parametricsurfacenet which is normalizedto get an invariant object-centeredcoordinatesystem.Sur-facedescriptionsareexpandedinto seriesof sphericalhar-monicswhich provideparametricrepresentationsof objectshapes.Gray-level informationis representedby 1-D pro-filesnormal to thesurface. Thealignmentis basedon thewell-acceptedstereotacticcoordinatesystemsincethedriv-ing applicationis thesegmentationof brain objects.Shapestatisticsare calculatedfrom the parametricshaperepre-sentationsrather than from the spatial coordinatesof setsof points.After initializing themeanshapein a new dataseton thebasisof thealignmentcoordinates,themodelelasti-cally deformsin accordanceto displacementforcesacrossthesurfacebut is restrictedonlybyshapedeformationcon-straints. The techniquehas beenapplied to segmentleftandright hippocampalstructuresfroma largeseriesof 3-Dmagneticresonancescanstakenfroma schizophreniastudy.
1 Intr oduction
Segmentationof anatomicalobjectsfrom large3D med-ical datasets,obtainedfrom routineMagneticResonanceImaging (MRI) examinations,for example, representsanecessaryyet difficult issuein medicalimageanalysis.In
somelimited applications,segmentationcan be achievedwith minimal userinteractionby applyingsimpleandeffi-cientimageprocessingmethods,whichcanbeappliedrou-tinely [7].
In many clinical applicationssuchascomputerassistedneurosurgery or radiotherapy planning,a large numberoforganshave to be identified in the radiologicaldatasets.While a carefuland time-consuminganalysismay be ac-ceptablefor outliningcomplex pathologicalobjects,norealjustificationfor sucha procedurecanbe found for the de-lineationof normal,healthyorgansat risk.
Delineationof organboundariesis alsonecessaryin var-ioustypesof clinical studies,wherethecorrelationbetweenmorphologicalchangesand therapeuticalactionsor clini-cal diagnosishasto be analyzed. In order to get statisti-cally significantresults,a large numberof datasethastobesegmented.For suchapplicationsmanualsegmentationbecomesquestionablenot only becauseof the amountofwork, but alsowith regardto thepoorreproducibilityof theresults.Thenecessityof obtaininghigh reproducibilityandthe needto increaseefficiency motivatesthe developmentof computer-assisted,automatedprocedures.
Elasticallydeformablecontourandsurfacemodels,so-calledsnakes[8], have beenproposedastools for support-ing manualobjectdelineation.While suchprocedurescanbe extendedto 3-D [24, 3], their initialization is a criticalissue. Most often, the initial guessmustbe very closetothesoughtcontourto guaranteea satisfyingresult[14]. Anexcellentoverview of elasticallydeformablemodelscanbefoundin [11]. Theprimaryreasonfor theneedof a precisesnake initialization is the presenceof disturbingattractorsin the image. Thesedo not belongto the desiredobjectcontourbut forcethesnake into localenergy minima.If thedeformationof asnakecouldbelimited toshapeswithin thenormalvariationof a classof objectboundaries,theproce-durewouldbemorerobust.
Elastically deformable parametric models offer astraightforward way for the inclusion of prior knowledgein the imagesegmentationprocess.This is doneby incor-
poratingprior statisticsto constrainthevariationof thepa-rametersof the elasticmodel. Suchprocedureshave beendevelopedby VemuriandRadisavljevic [26] usingahybridprimitive calleddeformablesuperquadricsconstructedin amulti-resolutionwaveletbase,or by StaibandDuncan[18]for deformableFouriermodels.
For complex shapesdescribedby alargenumberof oftenhighly correlatedparameters,theuseof suchpriorsmaybe-cometedious.Themodalanalysisasproposedby PentlandandSclaroff [15] offers a promisingalternative by chang-ing the basisfrom the original modelingfunctionsto theeigenmodesof thedeformationmatrix. Thedominantpartof the deformationscanthusbe characterizedby only thesmallesteigenmodes,substantiallyreducingthedimension-ality of the objectdescriptorspace.Methodsusingmodalanalysishave beensuccessfullyappliedto medicalimageanalysisby Sclaroff andPentland[17] andNastarandAy-ache[13], for example.
Cooteset al. [5] combinedthe power of parametricde-formableshapedescriptorswith statisticalmodalanalysis.They useactive shapemodels,which restrict the possibledeformationsusingthe statisticsof training samples.Ob-ject shapesare describedby the point distribution model(PDM) [4, 6], which representstheobjectoutlineby a sub-set of boundarypoints. Theremustbe a one-to-onecor-respondencebetweenthesepointsin the differentoutlinesof the trainingset. After normalizationto size,orientationandposition,they provide thebasisfor thestatisticalanal-ysisof theobjectshapedeformations.Themeanpoint po-sitions and their modesof variation (i.e. the eigenvectorscorrespondingto thelargesteigenvaluesof their covariancematrix) areusedfor limiting the objectdeformationsto areasonablelinearsubspaceof thecompleteparameterspace.
For a large training set containingseveral anatomicalstructures,the generationof the PDM parametrizationbe-comesverytediousand,becauseof thelackof areasonableautomatization,canbea sourceof errors,suggestingalter-nativeapproachesfor form parametrization.StaibandDun-canhave alreadydemonstratedsegmentationby parametri-cally deformableelasticmodelsfor 2-D outlines[18] and3-D objectsurfaces[19, 20] usingFourier-descriptors.In ourpreviouswork [21] wecombinedthestatisticalmodalanal-ysiswith parametrizationbasedon2-D Fourier-descriptors.UsingspatialnormalizationbasedonthegenerallyacceptedTalairachcoordinatesystem[23] wedemonstratedthatfullyautomaticsegmentationof organ contourson 2-D imageslicescan be achieved. In this previous paperthe feasi-bility of a 3-D extensionof this methodhasalreadybeeninvestigated.We have demonstratedthat basedon a gen-eralsurfaceparametrizationscheme[2] theconceptcanbegeneralizedfor 3-D organsurfaceswith sphericaltopologyusingsphericalharmonicsasshapedescriptors.This papersummarizesthe basicconceptsof the newly developed3-
D segmentationsystemandalsopresentsevaluationresultsusingacollectionof 22volumetricMR braindatasets.
2 Segmentationconcept
The3-D segmentationdiscussedhereis basedon a sta-tisticalmodel,generatedfrom acollectionof manuallyseg-mentedMR imagedatasetsof differentsubjects.Thepro-cesscanbedividedinto two majorphases;amodel-buildingstage,andtheautomaticsegmentationof largeseriesof datasets.� In thetrainingphase,theresultsof interactivesegmen-
tationof sampledatasetsareusedto createastatisticalshapemodelwhichdescribestheaverageaswell asthemajorlinearvariationmodes.� Themodelis placedinto new, unknown datasetsandis elasticallydeformedto optimally fit the measureddata.
Thegenerationof thestatisticalmodelwill bediscussedin detail in the following section. The purely geometri-cal statisticalmodelproposedin our earlierpaper[21] hasbeenextendedby incorporatinggray-valuedprofilesacrossthe organsurface,implementingthe conceptproposedbyCootesandTaylor [5, 6] to thethird dimension.
The matchingprocessis initialized using the averagegeometricalmodel resulting from this training phase. Atwo-stagealgorithm,describedin section4, is usedto de-form this modelto optimally fit the featuresof a new dataset,while still restrictingthe deformationsto the variabil-ity allowedby thestatisticalmodel. This algorithmmakesfull useof thegray-valueprofilesalongthesurface,whichbecomespossibleonly by using a dual representationofthe object both as a collection of samplepoints and as aparametrizedsurface.
3 Generationof 3-D statistical model
The conceptproposedin this paperresultsin an auto-maticselectionof a largesetof labeledsurfacepoints.Thisis doneusinga uniform parametrizationof closedsurfacesandby calculatingan invariantobjectcentereddescription(Brechbuhleret al. [1, 2]). Thealignementof parametrizedobjectsurfacesapproximatesa correspondenceof surfacepoints. The training setconsistsof a seriesof segmentedvolumetric objectsobtainedby experts using interactivesegmentation.
3.1 Interacti veexpert segmentation
Today’s routinepracticefor 3-D segmentationinvolvesslice-by-sliceprocessingof high-resolutionvolume data.
a d
b c
Figure 1. Model building: Interactive segmen-tation (a), reconstruction from shape descrip-tor s up to degree one (b), reconstruction upto degree ten (c) and normalization in objectspace (d).
Working on large series of similar scans, human ob-serversknowlegablein anatomybecomeexpertsandpro-ducehighly reliablesegmentationresults,althoughat thecostof aconsiderableamountof timeperdataset.Realisticfiguresareseveralhoursto onedaypervolumedatasetforonly a smallsetof structures.Regionsin 2-D imageslicescorrespondingto cross-sectionsof 3-D objectsareoutlinedandpaintedby aninteractive tool called”slice editor”. Theseriesof binaryregionssegmentedfrom consecutive slicesform volumetricvoxel objects.Figure1aillustratesthere-sultof anexpertsegmentationof theleft hippocampus.
3.2 Surfaceparametrization
Thesurfaceof aclosedvoxel objectis mostoftenstoredasameshbasedonverticeshaving threespatialcoordinates,althoughpresentingtwo degreesof freedom. Brechbuhleret al. [2] developeda surfaceparametrizationof arbitrarysimply connectedobjectsbasedon thosetwo parameters.Theembeddingof aconvolutedobjectsurfacein thesurfaceof theunit sphereof theparameterspacewasformulatedasaconstrainednon-linearoptimizationproblem.Themethodachievesa homogeneousdensityandminimal distortionoftheparameternetprojectedontotheobjectsurface.Further,objectsof similarshapeproduceverysimilarparameternets(up to a rotationin parameterspace;seebelow) dueto theuniquenessof the solution and robustnessof the iterativeoptimization[2].
3.3 Sphericalharmonic shapedescriptors
The parametization gives us three explicit func-tions defining the object surface as x
�θ � φ ����
x�θ � φ ��� y � θ � φ ��� z� θ � φ ��� . This surface description is
usedto expanda 3-D shapeinto acompletesetof sphericalharmonics.Theseriestakestheform
x�θ � φ ��� K
∑l 0
l
∑m� l
cml Ym
l�θ � φ � � (1)
Thecoefficientscml arethree-dimensionalvectorswith com-
ponentscxml , cy
ml andcz
ml with degreel andorderm. A de-
taileddescriptioncanbefoundin Brechbuhleretal. [2]. Allthecm
l with components�x � y� z� definethevector
c � �cx
00 � cy
00 � cz
00 � cx
11 � cx
01 � cx
11 � cy
11 � cy
01 � cy
11 �
cz 11 � cz
01 � cz
11 ������� cx
KK ����� cz
KK � T �
3.4 Surfacecorrespondenceand objectalignment
The surfaceparametrization,i.e., the representationofthe surfaceby a parameternet with homogeneouscells, isso far only determinedup to a 3-D rotation in parameterspace.However, apointtopointcorrespondenceof surfacesof differentobjectswould requireparameterswhich do notdependon therelativepositionof theparameternet.
Thepositionandorientationof objectsin original coor-dinatespacehasto benormalizedbeforestartingacompar-ison. For example,parametersfor aligning objectscanbeobtainedby calculatingan object-centeredcoordinatesys-tem. Thesegmentationmethoddescribedin this papercanincorporatesmall deviationsof translationandorientationinto theshapestatistics.This allows usto reproduciblyde-fine a globalcoordinatesystembasedon a smallsetof sig-nificantlandmarks.
Object-centered invariant surface parametrizationThe object can be rotated to a canonical position inparameterspaceby making useof the hierarchicalshapedescriptionprovided by spherical harmonic descriptors.The coefficients of the spherical harmonic function ofdifferentdegreesprovideameasureof thespatialfrequencyconstituentsthat composethe structure. As higher fre-quency componentsareincluded,moredetailedfeaturesofthe object appear. To definea standardposition we onlyconsiderthe contribution of the sphericalharmonicsofdegreeone, which is an ellipsoid representingthe coarseelongationof the object in 3-D space. We rotate theparameterspaceso that the north pole (θ � 0) will be atoneendof theshortestmainaxis,andthepoint wherethezeromeridian(φ � 0) crossesthe equator(θ � π � 2) is at
oneendof the longestmainaxis. Fig. 1(b,c)illustratesthelocationof the middle main axis on the reconstructionupto degreeoneandtenrespectively.
Objectsof similarshapewill getastandardparametriza-tionwhichbecomescomparable,i.e.,parametercoordinates�θ � φ � are located in similar regions of the object shape
acrossthe set of objects(seeFigure 2). Correspondingpointson differentobject surfacesare thereforefound bycalculatingacanonicalparametrizationratherthanby inter-active selectionof labeledsetsof 3-D points. The assess-mentof the quality of point correspondences,with a viewto potentialimprovements,areresearchquestionsof currentinterest([9, 22, 16]).
a b
c d
Figure 2. Corresponding parameter values forθ � π � 2, φ � 0 � π, and φ � π � 2 � 3π � 2 (thic k lines)illustrated on an ellipsoid (a) and on three in-dividual left-hippocampi.
Alignment in object space Our driving application istheautomaticsegmentationof brain objects. We begin bychoosingthe standardstereotacticcoordinatesystempro-posedby Talairachfor globalalignmentof theheadimagedatasets.Basicfeaturesusedfor alignmentaretheapprox-imation of the inter-hemisphericfissureby a midsagittalplaneandthe definitionof theanteriorandposteriorcom-missure(AC-PC)(seeFig. 3). Eachdatasetis transformedinto canonicalcoordinatesby 3-D rotationandscaling.
3.5 Shapestatistics
After transformationto canonicalcoordinates,theobjectdescriptorsarerelatedto thesamereferencesystemandcanbedirectlycompared.An establishedprocedurefor describ-ing a classof objectsfollows [5], wherethecalculationare
Figure 3. Stereotactic coor dinate systemused for object space normalization
carriedout in the domainof shapedescriptorsratherthanthe Cartesiancoordinatesof points in object space. Themeanmodelis determinedby averagingthe descriptorsc j
of theN individualshapes(seeFig. 4).
c � 1N
N
∑j 1
c j (2)
Eigenanalysisof the covariancematrix Σ resultsin eigen-valuesandeigenvectorsrepresentingthesignificantmodesof shapevariation.
Σ � 1N � 1∑
j
�c j � c��� � c j � c� T (3)
Σ � PcΛPTc � �
PcΛ1� 2 � � PcΛ1� 2 � T � (4)
wherethe columnsof Pc hold the eigenvectorsandthediagonalmatrix Λ the eigenvaluesλ j of Σ. Vectorsb j de-scribethedeviation of individual shapesc j from themeanshapeusingweightsin eigenvectorspace,andaregivenbe-low
c j � c � Pcb j (5)
Figure5 illustratesthe largesttwo eigenmodesof the hip-pocampustraining set. Truncatingthe numberof eigen-modes,correspondingto theeigenvaluessortedby size,re-strictsdeformationsto the major modesof variation. Fig-ure7 illustratesthesquarerootof eigenvaluessortedby size(dottedline) andthecomponentsof onetypical vectorb j .
Figure 4. Illustration of all 22 left hippocam-pal structures of the training sets, normaliz edand reconstructed from their descriptor s.
3.6 Modeling the gray level envir onment of sur-facemodels
Followingtheworkof Cootesetal. [5, 6] weaugmentthegeometricshapemodelsby incorporatinginformationaboutthegraylevel environmentof themodelsurface.We exam-ine the statisticsof the imageintensityalong1-D profilesorthogonalto theobjectsurfaceatadiscretesetof samplingpoints. Equalprocessingof eachpartof themodelsurfaceis ensuredby choosingahomogeneousdistributionof sam-pling pointsandprofilesover the3-D surface.Becausetheobjectsareparametrizedby the two sphericalcoordinates(θ � φ), the straightforwardmethodwould be to usea regu-lar meshof theseparameters.This, however, would resultin a highly irregular meshon a sphericalsurfacegiving adensesamplingat the polesand a sparsesamplingalongtheequator(seeFigure8(a)andFigure1(b,c)).A perfectlyregular samplingof a sphericalsurfacedoesnot exist, butwe canfind a goodapproximationby icosahedronsubdivi-
Figure 5. Largest two modes of variation forb j ��� 2 � λ j ����� 2 � λ j . In the mid dle column,b j � 0 represents the mean model.
Figure 6. Mean models of left and right tha-lamus, glob us pallidus, putamen and hip-pocampus.
sion,a techniqueoftenusedin computergraphicsto trian-gulateanddisplayspheresatdifferentscales.Thealgorithmtakesan icosahedroninscribedin a sphere,andsubdividesits facesasshown in Figure8(b). Thenewly introducedver-ticeslie slightly insidethesphere,so we pushthemto thesurfaceby properlynormalizingtheirdistanceto thecenterto unity.
We have chosena subdivision of k � 10 which givesusn � 12 � 30
�k � 1��� 20 � k 1� � k 2�
2 � 1002vertices. Com-putingtheθi andφi valuesateachvertex coordinatei of thesubdividedicosahedronandsubstitutingtheminto
xi � ��xi
yi
zi
�� � K
∑l 0
l
∑m� l
cml Ym
l
�θi � φi ��� (6)
i � 1 ����� 1002 (7)
we obtain a dual descriptionof the object surfaceby thecoordinatesof asetof surfacepointsxi. Theequationabovecanbewritten in a morecompactmatrix form as
5 10 15 20
-0.8
-0.6
-0.4
-0.2
0.2
0.4
Figure 7. Statistics of shape deformation.The dotted line represents the square root ofeigenvalues � λ j sor ted by decreasing size.The contin uous line illustrates the compo-nents of an individual vector b j , whic h de-scribes the deviation of the shape c j from themean shape c.
x � Ac � (8)
wherex representsthe coordinatesin object spaceand cthesphericalharmonicsdescriptors.A consistsof thefunc-tion valuesof Ym
l
�θi � φi � , onefor eachdimension,andde-
scribesthe mappingbetweenshapedescriptionspaceandobjectspacecoordinates.
For everysurfacepoint i in eachdataset j wecanextracta profile wi j of np samplepoints. The distancebetweensamplepointsis the lengthof onevoxel edge.Theprofilesareorientednormalto theobjectsurfaceandcenteredat thesurfacepointsxi j , asillustratedin Figure9. Foreachsamplepoint i we canobtaina meanprofile by averagingover thesampleobjectsN:
wi � 1N
N
∑j 1
wi j � (9)
Wecalculateanp � np covariancematrix wi whichgivesus a statisticaldescriptionof the expectedprofilesat eachsamplepoint.
Cooteset al. [6] proposenormalizedderivative profilesgiving invarianceto uniformscalingof graylevelsandcon-stantshift. For ourapplications,however, we achievedbestresultsusingunnormalizedoriginal gray level profiles,asall our datasetshave beenacquiredunderthe sameimag-ing conditions.Thisallowsusto avoid theinformationlosscausedby any normalizationprocedure.
4 Segmentationby modelfitting
Until now wehave only describedthecreationof a flex-ible 3D modelincludinggeometricshape,gray level envi-
aPi--2
Pi 3 Pi---- 2
2 Pi
φ
Pi--2
0
Pi
θ
b
Figure 8. Sampling methods of spherical sur -faces: regular mesh in spherical coor dinates(a), icosahedr on subdivision (b).
Figure 9. Illustration of an individual lefthippocampal shape with its profile vector ssho wn from the left side of the brain.
ronmentandstatisticsaboutnormalshapevariability. Wenow performthesegmentationstepby elasticallyfitting thismodelto new 3D datasets. This is achievedwith the fol-lowing two steps:� Initialization is doneby transformingthemodel’s co-
ordinatesysteminto thatof thenew dataset.� The surfacewill be elasticallydeformeduntil it bestmatchesthenew grayvalueenvironment.
4.1 Initialization of segmentation
Sincethemodelhasbeenbuilt basedonanormalizationto theTalairachcoordinatesystem,thedeterminationof thesymmetryplaneof thebrainandthepositionof theAC/PCline becomesanintegralpartof theinitialization. Currently
this is donemanuallybut we are planningto replacethedeterminationof the symmetryplaneand the AC/PC lineby automaticmethods[10, 25, 12]. A translationvectorandarotationmatrixarecomputedto transformthemodel’scoordinatesysteminto theimagespaceof thenew dataset.
4.2 Elastic deformation of modelshape
Weintroducedtwo differentrepresentationsof asurface,onebasedon thesphericalharmonicdescriptorsanda sec-ondonebasedon thesubdividedicosahedron.We attemptto usetheadvantagesof bothrepresentationsin our proce-dure.Sphericalharmonicdescriptorswerenecessaryto findacorrespondencebetweensimilarsurfacesandthey alsoal-low theexactanalyticalcomputationof surfacenormalsby
ni � K
∑l 0
l
∑m� l
cml
∂Yml
∂θ � K
∑l 0
l
∑m� l
cml
∂Yml
∂φ� (10)
However, they only representaglobaldescriptionof anob-ject shape. The surfacepoints, on the otherhand,give alocal representation,which is essentialto carryout an iter-ative refinementof the model,aswill be describedin thenext section.Thus,wedecidedto keepbothrepresentationsduring the matchingprocess,the relationbetweenthe twobeingtractablevia thematrixA.
Calculating displacementsfor surfacepoints After ini-tialization of the surfacemodelwe calculatethe displace-mentvectorateachsurfacesamplepointwhichwouldmovethat point to a new position closer to the soughtobject.Since there is a model of a gray level profile for eachpoint,thesearchtriestofind anadjacentregionwhichbettermatchesthis profile. A profile w of lengthl
�� np � normal
to the surfaceis extractedat eachmodelpoint. This newprofile is shiftedalongthemodelprofile in discretestepssto find thepositionof thebestmatch. This is givenasthesquareof theMahalanobisdistance:
d2Maha
�s�!� �
w�s��� w � Σ 1
w�w�s��� w � (11)
wherew(s) representsthesub-interval of theextractedpro-file at steps having a length of np. The location of thebestfit is thustheonewith minimald2
Maha
�s� . Supposesbest
is the shift betweenthe two profilesproviding the bestfit.We choosea displacementvectordx for eachmodelpointwhichis parallelto theprofile,in thedirectionof thebestfitandhasmagnitudesbest.
Adjusting the shapeparameters Having generated3-Ddisplacementvectorsfor eachof then modelpoints
dx � �dx1 � dy1 � dz1 �������"� dzn ��� (12)
wethenadjusttheshapeparametersto movethemodelsur-facetowardsa new position.Sincerotation,translationandscaleare alreadyincorporatedin the model statistics,wedo not have to dealwith themseparately. Of moreconcernarecalculateddisplacementsdx, as thesecould freely de-form theshapeof theobject.In orderto keeptheir resultingshapeconsistentwith thestatisticalmodel,we restrictpos-sibledeformationsby consideringonly thefirst few modesof variation. This will be solved by minimizing a sumofsquaresof differencesbetweenactualmodelpoint locationsandthesuggestednew positions.
The shapestatistics,asdescribedin section3.5, canbeexpressedby
c � c � Pcb � (13)
Multiplying both sidesof the above equationby A we getthedualsurfacedescriptionby asetof surfacepoints:
Ac � Ac � APcb (14)
x � x � Pxb � (15)
wherePx denotestheproductAPc whichrepresentsthema-trix of modesof shapevariationexpressedin objectcoordi-natespace.RecallthatPc is thematrixof eigenvectorsin theshapedescriptorspacedefinedby thecomponentsof theel-liptic harmonicdescriptorsc. Interestingly, weightvectorsbi of individual shapes,which expressthe deviation fromthemeanmodel,staythesamein bothshaperepresentationschemes.
We seekanadjustmentdb to b whichcausesadeforma-tion in eigenspacewhich matchestheoptimaldeformationx ascloselyaspossible.�
x � dx ��� x � Px�b � db �!� (16)
Subtractingeq.(15)from eq.(16)weget
dx � Pxdb � (17)
This is anoverdeterminedsetof linearequationswherethenumberof equations(3n) is muchlarger thanthe numberof variables(thenumberof modesis usuallyrestrictedfromaround5 to 10). Thereforea leastsquaresapproximationtothesolutioncanbeobtainedusingstandardlinearalgebra.
The entire procedureis repeatediteratively and startswith the averagemodelsuchthat bt 0 � 0. At eachiter-ationstep,wecomputeanew setof displacementsfrom thematchof profilesandupdatethe shapedeviation vectorbuntil theshapestopsto vary.
Shapeconstraints Thereare two differentkind of con-straintsweapplyto keeptheresultingshapeconsistentwiththeshapemodel.Ontheonehand,thereis alimited numberof eigenmodesdueto thesmallnumberof individualsand
the restrictionof the numberof modes. And on the otherhand,aftertheweightshavebeenupdatedby
bt # 1 � bt � dbt � (18)
weconstrainthecomponentsbi of b usingthestandardde-viation definedby the statisticalmodel,which is givenbytheeigenvalues$ λi (seeFig. 7). Thus,eachcomponentofbi % t # 1 lying outsideof theinterval & a$ λi will betruncated,wheretheconstanta is setto 2.
5 Results
Figure 10(a) shows the initial placementof the lefthippocampusmodel (white line) togetherwith the hand-segmentedcontour(gray line) on a sagittal2D slice (top)andasa 3D scene(bottom)viewed from the right sideofthehead.Images(b), (c),and(d) show theiterativeprogressof thefit. After 100iterationsthemodelgivesa sufficientlyclosefit to thedata.Themodelusedin this examplehad5degreesof freedom,andthemodelprofileshadtotal lengthof 11 samplepointswhile theextractedprofileshada totallengthof 19 samplepoints. The whole segmentationpro-cesstakesabout2 minutesonaSunUltra 1 workstationandrunsfully automaticallyafter initializing themodelwith anew dataset.
The above procedurehas beenapplied to all 22 datasetswherethehippocampushadbeenmanuallysegmented,andadditionallyto 8 datasetswhereit hadnot. The per-formanceof the automaticsegmentationhas beentestedby comparisonswith manuallysegmentedobject shapes,whicharewell recognizedasagoldstandardgiventhelackof groundtruth. A representsthemodelshapeobtainedbyhumanexperts,B the resultof the new model-basedseg-mentation.
The overlap measure�A ' B �"� � A ( B � shown in Fig-
ure 11 is calculatedon binary voxel maps,createdby in-tersectionof the objectsurfaceswith the voxel grid. Theresultingmeasureis verysensitiveto evensmalldifferencesin overlap,bothinsideandoutsideof theobjectmodel,andthereforea strongtest for segmentationaccuracy. For ex-ample,two voxel cubesof avolumeof 10 � 10 � 10shiftedby onevoxel alongthe spacediagonaldirectionresultsinonly a57%overlap(729/1271),althoughthemeandistanceof surfacesis roughly1 voxel edge.
Thecalculationof themeandistanceof surfacescanbedeterminedin anelegantway directly from thecoefficientsof the sphericalharmonicexpansionusing Parseval’s ex-pression. We thereforeavoid the discretizationerrorsbyprojectingthesurfacesbackto a voxel grid.
)+*x�u � * 2 du � ∞
∑l 0
l
∑m� l , c , 2 � 4π � MSD� (19)
a b
c d
Figure 10. Segmentation results of a left hip-pocampus on sagittal 2D slices and 3D viewsfrom the left hand side . The images a-d weretaken after 0, 30, 60 and 100 iterations.
whereMSD standsfor meansquared distancemeasuredfrom theorigin of thecoordinatesystem.
Figure12nicelyillustrateshow themeandistanceof sur-facesis reducedby the iterative elasticdeformationof themodel. Again, we take the humanexpert’s segmentationasgroundtruth andcompareits surfacewith the resultofthe automaticsegmentation. The barsin light grey illus-tratethemeandistanceof the initialization of themodelina new dataset,andthedarkbarsthefinal meandistanceofsurfacesto themodelsurface.Thehorizontalaxis lists theseriesof 21 normalcontrolsandschizophrenicsthat wereusedin thisstudy.
6 Conclusions
We presenta new 3-D segmentationtechniquethatpro-vides fully automaticsegmentationof objectsfrom volu-metric imagedata. Testswith a largeseriesof imagedatademonstratedthat the methodwasrobust andprovidesre-
1 2 3 4 5 6 7 8 9 101112131415161718192021
20
40
60
Figure 11. Overlap measure�A ' B �"� � A ( B � in
percenta ge calculated between manuall y (A)and automatic (B) segmented left hippocampiof 21 individuals. Bars in light grey illustratethe measure at initialization and in dark greyafter deformation.
producibleresults.The new techniqueuseselasticdeformationof surface
models,which carrystatisticalinformationof normalgeo-metricshapevariationandstatisticsaboutgray levelsneartheobjectsurface.Our modelhasbeenderivedfrom a se-ries of training datasets. Thereby, the modelrepresentsarealisticshaperatherthana simplegeometric3-D figure.Furthermore,information aboutthe statisticsof a normalshapedeformationhelpsto constrainanew solution.Thisisan importantadvantagesince3-D snake andballoontech-niquesareknown to be proneto elasticallydeformto anysmoothobjectshape.
Our approachhasbeensignificantly influencedby theresearchwork of Cootes,Taylor et al. [5, 6]. However, theextensionof their2-D methodto a true3-D volumetricseg-mentationtechniquerequiredvariousnew solutionsto sin-glestepsof theprocedure:
Statistical shapemodels: To overcome the problem ofgettinga reproducibleinteractive definitionof a setofkey pointsin 3-D space,theapproachpresentedhereinproposesan automaticdefinition of surface mesheswith homogeneousdistribution of nodesdefinedin astandard,canonicalposition.
Object alignment: We definethepositionandorientationof objectsin aglobalcoordinatesystemwhichis is de-finedby thetypeof application.Smalltranslationsandrotationsof objectswith respectto thiscoordinatesys-temarepart of thestatisticalmodel. Therefore,wedonot separatea similarity transformfor alignmentandanelastictransformfor remainingshapedeformationsasin [5].
Dual shaperepresentations:Our approachmakesuseof
1 2 3 4 5 6 7 8 9 101112131415161718192021
1
2
3
4
Figure 12. Average distances in mm calcu-lated between manuall y and automatic seg-mented left hippocampi of 21 individuals. Thebars in light grey illustrate the mean distanceof the initialization of the model in a new dataset and the dark bars the final mean distanceof surfaces to the model surface . The lengthof the hippocampus is about 40 mm.
two shaperepresentationswhich are usedin a vice-versafashion,taking advantagesof shapedescriptorsholding a compactglobal objectcharacterizationandof a setof surfacepointsgiving accessto local shapeproperties.
Similar to the experienceof Cooteset al. [5] we toofoundthatthemodellingof graylevel informationneartheobjectboundariesprovidesvaluableadditionalinformationfor a model placementand improves the robustnessandstability of the optimizationscheme.An early versionofoursegmentation[21] usedanenergyminimizationconceptsimilar to standardsnaketechniques.Thismethodwasverysensitive to the quality of the initialization, and pronetobe trappedby local energy minima. Theadditionaluseofgray-levelprofileinformationrepresentsastrongrestrictionto the numberof possiblesolutionsandwasdemonstratedto berobustevenin thepresenceof considerablemismatchbetweeninitializationanda new object.
We noticedthattheconvergenceis fasterif only a smallnumberof modes(usually5) are involved, while a largernumberof modes(usually10) is requiredto find theexactcontour. Thus,we planto applya relaxationmethodwhichgraduallyincreasesthenumberof modes.Theconvergencecriteriais setby thesizeof thedeformationof a surface.
Validationsofarhasbeenonly doneby visuallycompar-ing theautomaticsegmentationwith the resultsof interac-tive outliningby experts,(seeFig. 10),which is a common“gold standard”for comparisons.Wearecurrentlyworkingonaquantitativevalidationstudy.
Thesetof statisticalmodelsandtheautomaticandeffi-cientsegmentationtechnique(only a few minutesperdata
set)opennew possibilitesfor the processinga large num-berof datasetsascollectedin clinical studies,for examplein schizophreniastudies.This will provide new statisticalmodelswith increasednumberof samplesfor normalcon-trolsandfor differentpatientcategories.
Figure6 displaysaveragemodelsof four differentbrainobjects. Thesestatisticalmodelsrepresentthe first stepin building an anatomicalatlasbasedon a setof surfacesof anatomicalshapes. Whereasthe currentsegmentationtechniquewouldsegmenta seriesof objectsindependently,a future developmentcould provide a combinedmodel-ing of severalanatomicalstructures.Therepresentationofanatomicalobjectsby normalizedshapedescriptorsfurtherexploits its accessto morphometricparameters.After seg-mentinga new setof imagedata,morphologicalpropertiesof objectsareavailablefor comparativestudies.
Acknowledgements
RonKikinis andMarthaShenton,BrighamandWomen’sHospital,HarvardMedicalSchool,Bostonkindly providedthe original MR and segmenteddatasets. We are thank-ful to Christian Brechbuhler and JonathanOakley fromour lab for providing thesoftwarefor surfaceparametriza-tion andshapedescription,andfor carefullyproof-readingthemanuscript,respectively. Jens-PeerKuskais acknowl-edgedfor makingusMathview3davailablewhich convertsMathematicaTM 3D objectsto PovRayray-tracerfiles.
References
[1] C. Brechbuhler, G. Gerig, and O. Kubler. Surfaceparametrizationandshapedescription. In VisualizationinBiomedicalComputing, pages80–89,1992.
[2] C. Brechbuhler, G. Gerig, andO. Kubler. Parametrizationof closedsurfacesfor 3-D shapedescription.CVGIP:ImageUnderstanding, 61:154–170,1995.
[3] I. Cohen,L. Cohen,andN. Ayache.UsingDeformableSur-facesto Segment3D Imagesand Infer DifferentialStruc-tures.CVGIP: ImageUnderstanding, 56(2):242–263,1992.
[4] T. Cootesand C. Taylor. Active ShapeModels – ’SmartSnakes‘. In British Mach. Vision Conf., pages266–275.Springer-Verlag,1992.
[5] T. F. Cootes, A. Hill, C. J. Taylor, and J. Haslam.The Use of Active ShapeModels for Locating Struc-tures in Medical Images. Image and Vision Com-puting, 12(6):355–366,July 1994. Electronic version:http://s10d.smb.man.ac.uk/publications/index.htm.
[6] T. F. Cootes,C. J.Taylor, D. H. Cooper, andJ.Graham.Ac-tive ShapeModels- Their TrainingandApplication. Com-puter Vision and Image Understanding, 61(1):38–59,Jan.1995.
[7] G. Gerig,J.Martin, R. Kikinis, O. Kubler,M. Shenton,andF. Jolesz.AutomaticSegmentationof Dual-EchoMR HeadData. In IPMI’91, pages175–187.Wye,GB, 1991.
[8] M. Kass,A. Witkin, andD. Terzopoulos.Snakes: Activecontourmodels.Int. J. Comp.Vision, 1(4):321–331,1988.
[9] A. Kotcheff and C. Taylor. Automatic ConstructionofEigenshapeModelsby GeneticAlgorithm. In InformationProcessingin MedicalImaging, pages1–14.Springer, 1997.
[10] P. Marais,R. Guillemaud,M. Sakuma,A. Zisserman,andM. Brady. VisualCerebralAsymmetry. In VisualizationinBiomedicalComputing, pages411–416,1996.
[11] T. McInerney andD. Terzopoulos.DeformableModelsinMedical ImageAnalysis:A Survey. Medical Image Analy-sis, 1(2):91–108,1996.
[12] S. Minoshima,R. A. Koeppe,M. A. Mintun, K. L. Berger,S. F. Taylor, K. A. Frey, andD. E. Kuhl. AutomatedDe-tectionof theIntercommissuralLine for StereotacticLocal-izationof FunctionalBrain Images.TheJournal of NuclearMedicine, 34(2):322–329,Feb. 1993.
[13] C.NastarandN. Ayache.Frequency-basedNonrigidMotionAnalysis:ApplicationtoFourDimensionalMedicalImages.IEEETransactionsonPatternAnalysisandMachineIntelli-gence, 18(11):1067–1079,Nov. 1996.
[14] N. Neuenschwander, P. Fua,G. Szekely, andO. Kubler. Ini-tializing Snakes. In CVPR’94, pages658–663,1994.
[15] A. Pentlandand A. Sclaroff. Closed-Form SolutionsforPhysicallyBasedShapeModelling andRecognition.IEEEPAMI, 13(7):715–729,1991.
[16] A. Rangarajan,H. Chui, andF. Bookstein. TheSoftassignProcrustesMatchingAlgorithm. In InformationProcessingin MedicalImaging, pages29–42.Springer, 1997.
[17] S.Sclaroff andA. Pentland.On ModalModelling for Med-ical Images:UnderconstrainedShapeDescriptionandDataCompression.In IEEE Workshopon Biomed.Image Anal.,pages70–79,Seattle,Washington,USA, 1994.
[18] L. StaibandJ. Duncan.BoundaryFindingwith Parametri-cally DeformableModels. IEEE PAMI, 14(11):1061–1075,1992.
[19] L. StaibandJ.Duncan.DeformableFouriermodelsfor sur-facefinding in 3D images.In VBC’92, pages90–194,1992.
[20] L. StaibandJ. Duncan. Model-basedDeformableSurfaceFinding for Medical Images. IEEE Trans.Med. Imaging,15(5):1–12,1996.
[21] G.Szekely,A. Kelemen,C.Brechbuhler,andG.Gerig.Seg-mentationof 2-Dand3-D objectsfrom MRI volumedataus-ing constrainedelasticdeformationsof flexible Fouriercon-tourandsurfacemodels.MedicalImageAnalysis, 1(1):19–34,1996.
[22] H. Tagare.Non-rigidCurve Correspondencefor EstimatingHeartMotion . In InformationProcessingin MedicalImag-ing, pages489–494.Springer, 1997.
[23] J.TalairachandP. Tournoux.Co-planarstereotaxicatlasofthehumanbrain. Thieme,Stuttgart,1988.
[24] D. Terzopoulos,A. Witkin, and M. Kass. Symmetry-SeekingModels and 3D Object Reconstruction. Int. J.Comp.Vision, 1(3):211–221,1988.
[25] J. P. Thirion, S. Prima, andG. Subsol. StatisticalAnaly-sis of Dissymmetryin VolumetricMedical Images. Tech-nical Report3178,INRIA, June1997. Electronicversion:http://www.inria.fr/RRRT/RR-3178.html.
[26] B. Vemuri andA. Radisavljevic. MultiresolutionStochas-tic Hybrid ShapeModelswith FractalPriors. ACM Trans.Graphics, 13(2):177–200,1994.
