Real-Time Biomechanical Simulation of Volumetric Brain Deformation for Image Guided...
16
Real-Time Biomechanical Simulation of Volumetric Brain Deformation for Image Guided Neurosurgery Simon K. Warfield , Matthieu Ferrant , Xavier Gallez , Arya Nabavi , Ferenc A. Jolesz and Ron Kikinis Surgical Planning Laboratory http://www.spl.harvard.edu, Department of Radiology, Departmentof Neurosurgery, Brigham and Women’s Hospital and Harvard Medical School, 75 Francis Street, Boston, MA 02115 Telecommunications Laboratory, Centre for Systems Engineering and Applied Mechanics, Universiti´ e Catholique de Louvain, Belgium Abstract We aimed to study the performance of a parallel implementation of an intraoperative nonrigid registration algorithm that accurately simulates the biomechanical properties of the brain and its deformations during surgery. The algorithm was designed to allow for improved surgical naviga- tion and quantitative monitoring of treatment progress in order to improve the surgical outcome and to reduce the time required in the operating room. We have applied the algorithm to two neurosurgery cases with promising results. High performance computing is a key enabling technology that allows the biomechanical sim- ulation to be executed quickly enough for the algorithm to be practical. Our parallel implementa- tion was evaluated on a symmetric multi-processor and two clusters and exhibited similar perfor- mance characteristics on each. The implementation was sufficiently fast to be used in the operating room during a neurosurgery procedure. It allowed a three-dimensional volumetric deformation to be simulated in less than ten seconds. 1 Introduction The key challenge for the neurosurgeon during brain surgery is to remove as much as possible of a tumor without destroying healthy brain tissue. This can be difficult because the visual appearance of healthy and diseased brain tissue can be very similar. It is also complicated by the inability of the surgeon to see critical structures underneath the brain surface as it is being cut. It was our goal to be able to rapidly and faithfully capture the deformation of the brain during neurosurgery, so as to improve intraoperative navigation by allow preoperative data to be aligned to volumetric scans of the brain acquired intraoperatively. Image guided surgery techniques are used in operating rooms equipped with special purpose imag- ing equipment. The development of image guided surgical methods over the past decade has provided a major advance in minimally invasive therapy delivery. Image guided therapy has largely been a visualization driven task. Quantitative assessment of intraoperative imaging data has not been possible in the past, and instead qualitative judgements by experts in the clinical domains have been relied upon. In order to provide the surgeon or interventional radiologist with as rich a visualization environment as possible from which to derive such judgements, previous work has primarily been concerned with image acquisition, visualization and registration of intraoperative and preoperative data. Biomechanically accurate registration of brain scans acquired during surgery, as proposed here, has the potential to be a significant aid to the automatic interpretation of intraoperative images and to enable prediction of surgical changes. 0-7803-9802-5/2000/$10.00 (c) 2000 IEEE.
Transcript of Real-Time Biomechanical Simulation of Volumetric Brain Deformation for Image Guided...
Real-Time Biomechanical Simulation of Volumetric BrainDeformation for Image Guided Neurosurgery
SimonK. Warfield�, MatthieuFerrant
�, Xavier Gallez
�, Arya Nabavi
��� �,
FerencA. Jolesz�
andRonKikinis�
�SurgicalPlanningLaboratoryhttp://www.spl.harvard.edu,Departmentof Radiology,�
Departmentof Neurosurgery, BrighamandWomen’sHospitalandHarvardMedicalSchool,75FrancisStreet,Boston,MA 02115�
TelecommunicationsLaboratory,�
Centrefor SystemsEngineeringandAppliedMechanics,UniversitieCatholiquedeLouvain,Belgium
Abstract
Weaimedto studytheperformanceof aparallelimplementationof anintraoperative nonrigidregistrationalgorithmthataccuratelysimulatesthebiomechanicalpropertiesof thebrainanditsdeformationsduringsurgery. Thealgorithmwasdesignedto allow for improvedsurgical naviga-tion andquantitative monitoringof treatmentprogressin orderto improve thesurgical outcomeand to reducethe time requiredin the operatingroom. We have appliedthe algorithmto twoneurosurgerycaseswith promisingresults.
High performancecomputingis akey enablingtechnologythatallows thebiomechanicalsim-ulationto beexecutedquickly enoughfor thealgorithmto bepractical.Our parallelimplementa-tion wasevaluatedona symmetricmulti-processorandtwo clustersandexhibitedsimilarperfor-mancecharacteristicsoneach.Theimplementationwassufficiently fastto beusedin theoperatingroomduringaneurosurgeryprocedure.It alloweda three-dimensionalvolumetricdeformationtobesimulatedin lessthantenseconds.
1 Introduction
Thekey challengefor the neurosurgeonduringbrainsurgery is to remove asmuchaspossibleof atumorwithout destroying healthybraintissue.This canbedifficult becausethevisualappearanceofhealthyanddiseasedbrain tissuecanbe very similar. It is alsocomplicatedby the inability of thesurgeonto seecritical structuresunderneaththe brain surfaceasit is beingcut. It wasour goal tobe ableto rapidly andfaithfully capturethe deformationof the brain during neurosurgery, so astoimprove intraoperativenavigationby allow preoperativedatato bealignedto volumetricscansof thebrainacquiredintraoperatively.
Imageguidedsurgerytechniquesareusedin operatingroomsequippedwith specialpurposeimag-ing equipment.Thedevelopmentof imageguidedsurgicalmethodsoverthepastdecadehasprovideda majoradvancein minimally invasivetherapy delivery.
Imageguidedtherapy haslargely beena visualizationdriven task. Quantitative assessmentofintraoperative imagingdatahasnot beenpossiblein thepast,andinsteadqualitative judgementsbyexpertsin theclinical domainshavebeenreliedupon.In orderto providethesurgeonor interventionalradiologistwith asrich avisualizationenvironmentaspossiblefrom whichto derivesuchjudgements,previouswork hasprimarily beenconcernedwith imageacquisition,visualizationandregistrationofintraoperative andpreoperative data. Biomechanicallyaccurateregistrationof brain scansacquiredduringsurgery, asproposedhere,hasthepotentialtobeasignificantaidto theautomaticinterpretationof intraoperative imagesandto enablepredictionof surgicalchanges.
0-7803-9802-5/2000/$10.00(c) 2000IEEE.
Early work (reviewedby Jolesz[1]) hasestablishedtheimportanceandvalueof imageguidancethroughthebetterlocalizationof lesions,thebetterdeterminationof tumormargins,andtheoptimiza-tion of thesurgical approach.Previousalgorithmdesignhasbeena steadyprogressionof improvingimageacquisitionandintraoperative imageprocessing.This hasincludedincreasinglysophisticatedmultimodality imagefusionandregistration.Clinical experiencewith imageguidedtherapy in deepbrainstructuresandwith largeresectionshasrevealedthelimitationsof existingrigid registrationandvisualizationapproaches[1].
The changesin brain shapeduring neurosurgery arenow widely recognizedasnonrigid defor-mations.Suitableapproachesto capturethesedeformationsandto allow integratedvisualizationsofpreoperative datamatchedto thebrainasit changesshapeduringthecourseof surgeryarein activedevelopment.Previouswork in capturingbraindeformationsfor neurosurgerycanbecategorizedbythosethatusesomeform of biomechanicalmodel(recentexamplesinclude[2–4])andthosethatapplya phenomenologicalapproachrelyinguponimagerelatedcriteria(recentexamplesinclude[5,6].)
A fast surgery simulationmethodwas describedin [7] which achieved speedby converting avolumetricfinite elementmodel into a modelwith only surfacenodes. This work hadthe goal ofachieving interactive graphicsspeedsat the cost of accuracy of the simulation. Sucha model isapplicablefor computergraphicsorientedvisualizationtasksbut duringneurosurgical interventionson patientswe aim for as high accuracy and robustnessas possibleand useparallel hardware toachieveclinically compatibleexecutiontimes.
A sophisticatedbiomechanicalmodelfor two-dimensionalbraindeformationsimulationusingafinite elementdiscretizationwas proposedin [2]. However this work usedthe pixels of the two-dimensionalimageastheelementsof thefinite elementmesh,andrelieduponmanuallydeterminedcorrespondences.Unfortunately, two-dimensionalresultsarenotusefulin clinical neurosurgicalprac-tice andsucha discretizationapproachis extremelycomputationallyexpensive (evenconsideringaparallelimplementation)if expandedto threespatialdimensionsbecauseof thelargenumberof voxelsin atypicalintraoperativeMRI (256x256x60� 4e06voxels)leadingto alargenumberof equationstosolve. Instead,theuseof a finite elementmodelwith anunstructuredgrid canallow a representationthatfaithfully modelskey characteristicsin importantregionswhile reducingthenumberof equationsto solveby usingmeshelementsthatcoverseveralimagepixelsin otherregions.
Most meshingsoftwarepackagesusedin themedicaldomaindo not allow meshingof multipleobjects[8,9], and often work bestwith regular andconvex objects,which is usuallynot the casefor anatomicalstructures.Therefore,we have implementeda tetrahedralmeshgeneratorspecificallysuitedfor labeled3D medicalimages.Themeshgeneratorcanbeseenasthevolumetriccounterpartof amarchingtetrahedrasurfacegenerationalgorithm.A detaileddescriptionof thealgorithmcanbefoundin [10]. Theresultingmeshstructureis built suchthatfor imagescontainingmultipleobjects,afully connectedandconsistenttetrahedralmeshis obtainedfor everycell. A segmentationof theim-ageindicatesthetypeof anatomicalstructurethecell belongsto. Therefore,differentbiomechanicalpropertiesandparameterscaneasilybeassignedto thedifferentcellsor objectscomposingthemesh.Boundarysurfacesof objectsrepresentedin themeshcanbeextractedfrom themeshastriangulatedsurfaces,which is convenientfor runninganactivesurfacealgorithm,asdescribedbelow.
A numberof imagingmodalitieshave beenusedfor imageguidance.Theseinclude,amongstothers,digital subtractionangiography(DSA), computedtomography(CT), ultrasound(US), andmagneticresonanceimaging(MRI). Intraoperative MR imagingcanacquirehigh contrastimagesofsoft tissueanatomywhichhasprovento beveryusefulfor image-guidedtherapy [11]. Multi-modalityregistrationallowspreoperativedatathatcannotbeacquiredintraoperatively, suchasfunctionalMRI(fMRI) or nuclearmedicinescans,suchasPositronEmissionTomography(PET) or SinglePhotonEmissionComputedTomography(SPECT)scans,or magneticresonanceangiography(MRA) to be
2
Preoperative MRI Preoperativesegmentation
SpatialLocalization
Preparation for Image Guided Therapy
Intraoperative MRI
Rigid Registration
During Image Guided Therapy
TissueClassification
Biomechanicalsimulation of brain deformationAligned preoperative and intraoperative data
Figure1: Schemafor IntraoperativeSegmentationandRegistration
visualizedtogetherwith intraoperativedata.A systemfor intraoperative visualizationhasrecentlybeendeveloped[12]. This systemallows
surfacerenderingof previously preparedtrianglemodelsandarbitraryinteractive resamplingof 3Dgrayscaledata. Thesystemalsoallows for visualizationof virtual surgical instrumentsin thecoor-dinatesystemof thepatientandpatientimageacquisitions.Thesystemsupportsqualitativeanalysisbasedon expertinspectionof theimagedataandthesurgeonsexpectationof whatshouldbepresent(normalanatomy, pathology, currentprogressof thesurgeryetc.)Theability to automaticallycapturethe deformationof the brain during neurosurgery would allow the augmentationof sucha systemto enablethe visualizationspreparedpreoperatively to be updatedto follow the changesthat occurduringsurgery. For example,this might allow previously acquiredfunctionalMRI (which cannotbeacquiredintraoperatively) to betransformedto placethefunctionalinformationin alignmentwith in-traoperatively acquiredmorphologicMRI, preservingtheability to interpretareasof themorphologicMRI baseduponthefunctionalinformation.
We aimedto demonstratethata volumetricthree-dimensionalbiomechanicalsimulationof braindeformationis possibleeven with the time constraintsof neurosurgery and that suchsimulationssignificantlyaddto thevalueof intraoperative imagingandhenceimprovesurgicaloutcomes.
2 Method
In orderto successfullycapturebraindeformationfrom intraoperative imageswe have developedasetof imageprocessingalgorithmsthattakeadvantageof anexistingpreoperativeMR acquisitionandsegmentation.We have experimentedfor severalyearswith a generalimagesegmentationapproachthatusesa 3D digital anatomicalatlasto provide automaticlocal context for classification[13–16].In this applicationthe preoperative dataactsasa patient-specificatlaswhich enablesa robust andreliableintraoperativesegmentationof thebrainsurfaceof thepatientusingourpreviouslydescribedreal-timesegmentationalgorithm[17]. Thebrainsegmentationthenconstitutesa reliabletarget forthebiomechanicalsimulationof thebraindeformation.Figure1 illustratestheprocessingstepsthattakeplacebeforeandduringthetherapy procedure.
Sincepreoperative datais acquiredbeforesurgery, thetime availablefor segmentationis longer.This meanswe canusesegmentationapproachesthatareasrobustandaccurateaspossiblebut aretime consumingand henceimpracticalto usein the operatingroom. In our laboratory, preoper-ative datais segmentedwith a variety of manual[12], semi-automated[18] or automated[15,16]
3
approaches.We attemptto selectthemostrobustandaccurateapproachavailablefor a givenclinicalapplication. Eachsegmentedtissueclassis thenconvertedinto an explicit 3D volumetricspatiallyvaryingmodelof thelocationof thattissueclass,by computingasaturateddistancetransform[19] ofthetissueclass.This modelis usedto provide robustautomaticlocal context for theclassificationofintraoperativedatain thefollowing way.
During surgery, intraoperative datais acquiredand the preoperative data(including any MRI,fMRI, PET, SPECT, MRA thatisappropriate,thetissueclasssegmentationandthespatiallocalizationmodelderived from it) is alignedwith the intraoperative datausingan MI basedrigid registrationmethod[12,20]. The intraoperative imagedatathen togetherwith the spatial localizationmodelforms a multichannel3D dataset. Eachvoxel of the combineddatasetsis then representedbya vectorhaving componentsfrom the intraoperative MR scan,the spatiallyvarying tissuelocationmodeland if relevant to the particularsurgery, any of the otherpreoperative imagedatasets. Forthe first intraoperative scanto be segmenteda statisticalmodel for the probability distribution oftissueclassesin theintensityandanatomicallocalizationfeaturespaceis built. Thestatisticalmodelis encodedimplicitly by selectinggroupsof prototypicalvoxels which representthe tissueclassesto be segmentedintraoperatively (lessthan five minutesof userinteraction). The spatial locationof the prototypevoxels is recordedand is usedto updatethe statisticalmodelautomaticallywhenfurtherintraoperativeimagesareacquiredandregistered.Thismultichanneldatasetis thensegmentedwith
�-NN classification[14,21], a standardclassificationmethodwhichcomputesthetypeof tissue
presentat eachvoxel by comparingthe signalof the voxel to classifywith the signalof previouslyselectedprototypevoxelsof known tissuetype.
The segmentationof the intraoperative datahelpsto establishexplicitly the regions of tissuesthatcorrespondin thepreoperative andintraoperative data.In thepastwe have describedanimage-basednonrigid registrationalgorithm[22,23] which hasbeensuccessfullyappliedto captureshapevariation in schizophrenia[24]. However, our previous approachdoesnot constitutean accuratebiomechanicalsimulationof the deformation,andhenceit is not possibleto effectively model thedifferentmaterialpropertiesof differentstructuresin the head,andit is not possibleto usesuchanapproachfor quantitative predictionof brain deformation. Theselimitations areaddressedby thebiomechanicalsimulationapproachwedescribebelow.
2.1 Method for Biomechanical Simulation of Three-Dimensional VolumetricBrain Deformation
Our intraoperative imageacquisitionandprocessingidentifiestheregion of thebrainvolume.How-ever, its shapechangesduringsurgery. Thegoalof simulatingtheintraoperativebrainvolumedefor-mationis to projectdataalignedwith thebrain in a previousconfigurationontothebrain in its newconfigurationfollowing surgicalchanges.
We usea two stepprocessto achieve this. In thefirst step,anactive surfacealgorithmis usedtoestablishthecorrespondencesbetweenthesurfacesof thebraindata.In thesecondstepthevolumetricbrain deformationimplied by the surfacechangesis computedusinga biomechanicalmodelof thestructureof thebrain.
The deformationfield obtainedfor the surfacesis usedin conjunctionwith the biomechanicalvolumetricmodelto infer the deformationfield insideandoutsidethe surfaces.Thekey conceptisto applyforcesto thevolumetricmodelthatwill producethesamedisplacementfield at thesurfacesaswasobtainedwith theactive surfacealgorithm. Thebiomechanicalmodelwill thencomputethedeformationthroughoutthevolume.
4
2.1.1 Correspondence Detection with an Active Surface
Theactivesurfacealgorithmiteratively deformsthesurfaceof thefirst brainvolumeto matchthatofthe secondvolume. This is doneiteratively by applyingforcesderived from the volumetricdatatoanelasticmembranemodelof thesurface. Thederivedforcesarea decreasingfunctionof thedatagradients,soasto beminimizedat theedgesof objectsin thevolume.To increaserobustnessandtheconvergencerateof theprocess,wehave includedprior knowledgeabouttheexpectedgraylevel andgradientsof theobjectsbeingmatched.Thisalgorithmis fully describedin [25].
2.1.2 Biomechanical Simulation of Volumetric Brain Deformation
Assuminga linear elasticcontinuumwith no initial stressesor strains,the potentialenergy of anelasticbodysubmittedto externallyappliedforcescanbeexpressedthrougha finite elementmodelas[26]: � � �� ��������������� ������ �
(1)
where� �"!$#&%(')%+*-,
is the vector representingthe forcesappliedto the elasticbody (forcesperunit volume,surfaceforcesor forcesconcentratedat thenodesof themesh),
� �.!/#&%0'1%+*2,the
displacementvectorfield wewish to compute,and�
thebodyonwhichoneis workingdescribedbyameshof tetrahedralelements.
�is thestrainvectorand
�thestressvector, linkedto thestrainvector
by thematerial’sconstitutiveequations.In the caseof linear elasticity, with no initial stressesor strains,this relation is describedas� !43652%(3678%(3:9�%(;<5+78%(;=7(9�%>;<5?9?, � A@ �
, where B is the elasticitymatrix characterizingthe material’sproperties[26].
The continuousdisplacementfield�
everywherewithin eachelementof the meshis definedasa functionof thedisplacementat theelement’s nodes
�DCE weightedby theelement’s shapefunctions(interpolatingfunctions)F CE !/GD, , �H!$GD, JI:KML4NPORQSEUT � F CE !$G&,>� CE.V (2)
Theelementswe useto representvolumedataaretetrahedral( FXWZY\[ C\] _^ ), with linearinterpola-tion of thedisplacementfield. Hence,theshapefunctionof node of tetrahedralelementa is definedas F CE !$GD, b2c C)d0e CE ��f CE #g��h CE 'i��� CE *kj (3)
The computationof the volumeof the elementc C
andthe interpolationcoefficientsaredetailedin[26, pages91–92].
Thevolumetricdeformationof thebrainis foundby solvingfor thedisplacementfield thatmini-mizestheenergy describedby Equation1, afterfixing thedisplacementsat thesurfaceto matchthosegeneratedby theactive surfacemodel. We solve thesystemof equationswith the thePortable,Ex-tensibleToolkit for ScientificComputation(PETSc)package[27,28] usingtheGeneralizedMinimalResidual(GMRES)solverwith blockJacobipreconditioning.
2.2 Hardware for Intraoperative Image Acquisition and Parallel Computation
Figure2 showsanopen-configurationmagneticresonancescanneroptimizedfor imagingduringsur-gicalprocedures[1,11]. Operationstakeplacewith thepatientonthebedinsidethetoroidalmagnetsandthesurgeonstandingin thegapbetweenthem.
5
Figure2: Intraoperative magneticresonanceimagingscannerin which imageguidedtherapy takesplace. Theoperatingroomconsistsentirelyof equipmentsafeto usearounda 0.5T magneticfield.For neurosurgerycasesthepatientis typically placedlying insidethe two toroidalmagnetsandthesurgeonstandsin thegapbetweenthem.
6
Item DescriptionCPU CompaqAlpha21164A(ev56)533MHzw/ 8KB+8KB L1 and96K L2
onchipcachesMotherboard Microway ScreamerLX w/ 2MB L3 9nsSRAM cacheanda 128-bit
wide 83MHz memorybus(basedon CompaqAlphaPC164LX moth-erboarddesign)
Memory 768MB , 128bit ECCunbufferedSDRAM 100MHz(1.3 GBytes/secpeaktransferrate)
Harddisk 2.1GB SeagateMedalist2132(ST32132A)IDENetwork Card CompaqDE500Ethernet10/100MbpsRJ45full duplexOS RedHatLinux 6.1
Figure3: Specificationsof workstationsin theDeepFlow cluster.
Measurementsof parallelperformanceweremadeon threedifferentparallelarchitectures.Thefirst of the threearchitectureswasa parallelcluster, called“Deep Flow”, consistingof a clusterofAlpha-basedworkstationsrunningLinux connectedby 100Mbpsfull duplex FastEthernet.Thespec-ificationsof this clusteris describedin Table3. The secondarchitecturewasa SunMicrosystemsUltra HPC6000symmetricmulti-processormachinewith 20250MHzUltraSPARC-II (4MB Ecache)CPUsand5 GB of RAM. Thethird architecturewasa clusterof two symmetricmulti-processorUl-tra 80 workstations,eachwith four 450MHz UltraSPARC-II (4MB Ecache)CPUsand2GB RAMnetworkedwith 100MbpsFastEthernet.
3 Results
In this sectionillustrativevisualizationsof biomechanicalsimulationof intraoperativebrainshift arepresentedalongwith ananalysisof theperformanceof our parallelimplementation.During surgicalproceduresin thebrain,intraoperativeMRI (IMRI) datawasacquiredandstored.Segmentationandnonrigidregistrationwasappliedto this dataafter therapy delivery in orderto allow usto assesstherobustness,accuracy and time requirementsof the approach.In the future we intend to carry outsegmentation,nonrigid registrationandvisualizationusingthe approachdescribedhereduring theinterventionalprocedureswith thegoalof improving imageguidedtherapy outcomes.
3.1 Biomechanical Simulation of Volumetric Brain Deformation
In eachneurosurgerycaseseveral volumetricMRI scanswerecarriedout during surgery. Thefirstscanwasacquiredat the beginning of the procedurebeforeany changesin the shapeof the braintook place,andthenover thecourseof surgeryotherscanswereacquiredasthesurgeoncheckedtheprogressof tumorresection.Thefinal scanin eachsequenceexhibitssignificantnonrigiddeformationandlossof tissuedueto tumorresection.In orderto testourbiomechanicalsimulationapproacheachsubsequentscanwasalignedto thefirst by rigid registrationusingmaximizationof mutualinforma-tion. This methodcomputesa global alignmentaccountingfor positioningdifferencesin the scancoordinatesbut doesnot attemptto correctfor nonrigid deformation. The first scanwasmanuallysegmentedto actasanindividualizedanatomicalmodel.Thelastscanin eachsequencewasthenseg-mentedwith our intraoperative segmentationapproach.Our algorithmfor biomechanicalsimulationof thebraindeformationwasthenexecutedto computethevolumetricdeformationbetweenthetwodatascans.
7
In orderto checkthequalityof theregistration,thedeformationof thefirst MRI scanwasvisuallycomparedwith theMRI to whichit wasmatched.In eachcasetheregisteredbraincloselymatchedtheexpectedlocation.Figure4 is avisualizationof theaccuracy of therecoveredvolumetricdeformationillustratedwith two-dimensionalslices.
Figure4(a)shows theskin bright, thebraingrayandthelateralventriclesdark,in its initial con-figuration. This definesthecoordinatesystemin which preoperative datais alignedandvisualized.Figure4(b) is the correspondingslice of the second3D scanwhich forms the target to which wewish to matchthe first volume. Significantsinkingof the surfaceof the braindueto the surgery isapparent.Figure4(c) illustratestheresultof simulatingthebraindeformationin orderto matchthefirst scanto the secondscan. A 3D visualizationof this matchis shown in Figure5. Figure4(d)showsthedifferencebetweenthesimulateddeformationof thebrainof Figure4(a)andtheMRI scanof Figure4(b) to which it is matched.The closenessof the matchof the simulateddeformationtothe actualdeformationcanbe judgedby the very small intensitydifferencesat the boundaryof thesimulateddeformedbrainandtheair gapinsidetheskull of the target image. Somesmall intensitydifferencesareexpectedbecauseintrinsic MR scannerintensityvariability causesa small variationin theobservedvoxel intensitiesfrom scanto scan.A smallmisregistrationof the lateralventricleson thesideoppositethesurgical resectioncanbeobserved. This occursbecauseour biomechanicalmodeltreatsthebrainasahomogeneousmaterial,but thecerebralfalx (astiff membranebetweenthetwo hemispheres)andthecerebrospinalfluid insidethe lateralventriclesarenot well approximatedby this homogeneousmodel.This misregistrationis not particularlyrelevantfor a surgical resectionon theoppositesideof thebrain.
Figure5 is avisualizationof thesimulateddeformationof thefirst intraoperativescanvolumeontothesecondintraoperativescanvolume.The3D surfacerenderingindicatesthesimulateddeformationof the brain in the first scanandthe cross-sectionalslicesshow the MRI of the secondscan. Theskin appearsbright in the MRI, andthe large dark region betweenthe skin andthe brainsurfaceispart of the deformationdueto the surgery. The closematchof the simulationandthe actualbraindeformationis clear. Thecolor codingindicatesthemagnitudeof thedeformationat every point onthesurfaceof thedeformedvolumeandthebluearrows indicatethemagnitudeanddirectionof thedeformationby showing theinitial andfinal positionof pointsonthesurfaceundergoingdeformation.
3.2 Performance Analysis of Parallel Implementation
The resultsherefocusuponthe biomechanicalsimulationof brain deformationbecausein the pastobtainingsufficiently accurateresultsin a clinically compatiblesmallamountof time hasbeenseenasextremelydifficult. Oftenlessaccuratebut fastmodelsof deformationhavebeenused.In ordertoprovidea context for thebiomechanicalsimulationamongsttheotherimageanalysisandacquisitiontasksthatmustoccurintraoperatively, a timelinefor theseactionsis shown in Figure6.
Ultimately theability to usetheseintraoperative imageanalysismethodsreliesuponthembeingsufficiently robustto provideaccurateresultsfor typicalclinical cases,andcritically, to besufficientlyfastto provide feedbackto thesurgeonat a ratethatcanbepracticalto useduringneurosurgery. Wecollectedperformanceresultson threedifferentarchitecturesin orderto assessthe absoluteperfor-manceandthescalingbehavior of ourparallelimplementation.
Figure7 shows the timing resultsfor a biomechanicalsimulationof brain deformationon theDeepFlow cluster. Thesimulateddeformationdeterminedby solvingthis setof equationsis shownin Figure4 andFigure5. This resultdemonstratesthat a biomechanicalsimulationof volumetricbraindeformationcanbecarriedout in lessthantenseconds.
The slow scalingof the implementationis attributed to imbalancein the matrix assemblyand
8
(a) A singleslice from thefirst 3D intra-operativeMRI scan.
(b) Thecorrespondingslicein thesecond3D MRI scan.
(c) Thematchedsliceof thefirst volumeaftersimulationof thebraindeformation.
(d) Magnitudeof the differencebetweenthesimulateddeformationandthe corre-spondingslice of the secondscanshow-ing the closenessof the alignmentof thebrain.
Figure4: Two dimensionalslicesthroughthree-dimensionaldata,showing thematchof thesimulateddeformationof the initial brainonto the actualdeformedbrain. Thequality of the matchis signifi-cantlybetterthancanbeobtainedthroughrigid registrationalone.Theclosenessof thematchof thesimulateddeformationto theactualdeformationcanbejudgedby theverysmallintensitydifferencesat the boundaryof the simulateddeformedbrain andthe air gap insidethe skull of the target im-age.SomesmallintensitydifferencesareexpectedbecauseintrinsicMR scannerintensityvariabilitycausesasmallvariationin theobservedvoxel intensitiesfrom scanto scan.A smallmisregistrationofthelateralventricleson thesideoppositethesurgical resectioncanbeobserved.Thisoccursbecauseour biomechanicalmodel treatsthe brain asa homogeneousmaterial,but the cerebralfalx (a stiffmembranebetweenthetwo hemispheres)andthecerebrospinalfluid insidethelateralventriclesarenot well approximatedby this homogeneousmodel. This misregistrationis not particularlyrelevantfor asurgical resectionon theoppositesideof thebrain.
9
Figure5: Visualizationof deformationof first intraoperativescanvolumeontosecondintraoperativescanvolume.The3D surfacerenderingindicatesthesimulateddeformationof thefirst scanandthecross-sectionalslicesshow theMRI of thesecondscan.Theskin appearsbright in theMRI, andthelargedarkregionbetweentheskinandthebrainsurfaceis partof thedeformationdueto thesurgery.The closematchof the simulationand the actualbrain deformationare clear. The color codingindicatesthemagnitudeof thedeformationateverypointon thesurfaceof thedeformedvolumeandthebluearrows indicatethemagnitudeanddirectionof thedeformationshowing theinitial andfinalpositionof pointson thesurfaceundergoingdeformation.
10
Timeline of image processing for image guided neurosurgery
ActionPreoperative segmentation
Intraoperative MRI
Rigid registration
Tissue classification
Surface displacement
Biomechanical simulation
Visualization
Surgical progress
TimeDuring surgery:Before surgery:
Figure6: A timelineof typical intraoperative imageacquisitionandanalysistasks.
0 2 4 6 8 10 12 14 1610
0
101
102
Number of CPUs
Tim
e (s
)
Assemble system of equations Solve system of equations Sum of initialization, assembly and solve times.
Figure7: Timing resultsfor assembling,solving,andthesumof initialization,assemblingandsolvingtime for a systemof 77511equationssimulatingthe biomechanicaldeformationof the brain on aclusterof 16CompaqAlpha21164A533MHzCPU-basedworkstationsnetworkedwith FastEthernet.
11
0 2 4 6 8 10 12 14 16 18 2010
0
101
102
Number of CPUs
Tim
e (s
)
Assemble system of equationsSolve system of equations
0 2 4 6 8 10 12 14 16 18 2010
0
101
102
Number of CPUs
Tim
e (s
)
Assemble system of equationsSolve system of equations
(a) Sun MicrosystemsUltra HPC 6000with 20250MHzCPUs
(b) Two Sun Microsystems Ultra 80serverseachwith 4 450MHz CPUsnet-workedwith FastEthernet.
Figure8: Timing resultsfor assemblingandsolving the systemof 77511equationssimulatingthebiomechanicaldeformationof thebrain.
matrix solve processes.Our parallel decompositionfor the matrix assemblyis basedon sendingapproximatelyequalnumbersof meshnodesto eachCPU.However, in ourunstructuredgrid differentmeshnodescanhavedifferentconnectivity, andhencerequirea differentamountof work in ordertointerpolatevaluesto the meshnodes. A secondload imbalanceaffects the scalingof the solvingtime. Givenaninitially balancedsetof equations,thesurfacedisplacementsareappliedasboundaryconditions,substitutingknown valuesfor equationsin the original system,reducingthe numberofunknownsthatmustbesolvedfor. This hastheeffectof creatingsomeimbalance,asthedistributionof surfacedisplacementsis notequalacrossCPUs.
For intraoperativeuse,theabsolutetimerequiredfor initialization,assemblyandsolvingisaccept-able. Time requiredfor initialization canbeoverlappedwith earlierimageprocessing.For displayof thesimulateddeformationweneedto resamplea datasetaccordingto thecomputeddeformation,whichrequiresapproximately0.5seconds.Ourcurrentimplementationat this timesimplywritesthedeformationto disk which imposesa fixedI/O cost(not includedin the timing results)thatwill notbenecessaryin practicalapplicationduringsurgery.
Figure8 shows timing resultsfor assemblingandsolvingthesystemof 77511equationssimulat-ing thebiomechanicaldeformationof thebrainon a SunMicrosystemsUltra HPC6000symmetricmulti-processorwith 20250MHzCPUsandonapairof SunMicrosystemsUltra 80serverseachwith4 450MHzCPUsnetworkedwith FastEthernet.Theseresultsindicatescalingperformancesimilar tothatobtainedon theDeepFlow cluster, despitethedifferencesin architectures.
In the futurean improvedbiomechanicalmodelcouldaim to bettermodeldifferentstructuresinthe brain. This may necessitatea higherresolutionmesh,andhencea larger numberof equationsto solve. Figure9 shows timing resultsfor assemblingandsolvinga systemof 253308equationstosimulatethe biomechanicaldeformationof the brain on a SunMicrosystemsUltra HPC 6000with20250MHzCPUs.Thetiming resultsindicatethatwecanassembleandsolveasystemof equations2.5 timeslarger thanthatnecessaryto obtainexcellentresultswith our currentmodelin a clinicallycompatibletime frame.
12
0 2 4 6 8 10 12 14 16 18 2010
0
101
102
Number of CPUs
Tim
e (s
)
Assemble system of equationsSolve system of equations
Figure9: Timing resultsfor assemblingandsolving a systemof 253308equationssimulatingthebiomechanicaldeformationof the brain on a Sun MicrosystemsEnterpriseServer 6000 with 20250MHzCPUs.
4 Discussion and Conclusion
Our early experiencewith two neurosurgery casesindicatesthat our intraoperative biomechanicalsimulationof braindeformationalgorithmis a robustandreliablemethodfor capturingthechangesin brainshapethatoccurduringneurosurgery. Theregistrationalgorithmrequiresnouserinteractionandtheparallelimplementationis sufficiently fastto beusedintraoperatively.
Intraoperativeregistrationaddssignificantlyto thevalueof intraoperativeimaging.It providesforquantitative monitoringof therapy applicationincludingtheability to comparequantitatively with apreoperatively determinedtreatmentplanandenablespreoperativedatato bealignedwith thecurrentconfigurationof the brain of the patientin order to allow 3D interactive visualizationof the fusedmulti-modalitydata.
Thecontribution of this work is the evaluationof a parallelimplementationof analgorithmforintraoperative registrationby biomechanicalsimulationof braindeformationandthedemonstrationthathigh fidelity simulationsof braindeformationarenot toocomputationallyexpensive for intraop-erativeuse.High performancecomputingis akey enablingtechnologythatallowsthebiomechanicalsimulationto be executedquickly enoughfor the nonrigid registrationalgorithmto be practicalinclinical useduringneurosurgery. Ourparallelimplementationshowsagoodspeedupwith increasingCPUcountandawallclock timecompatiblewith therequirementsof neurosurgical intervention.Wehave evaluatedour algorithmwith two neurosurgery caseswith promisingresults. Furtherclinicalvalidationwith largernumbersof caseswill benecessaryto determineif our algorithmsucceedsinimproving intraoperative navigationandintraoperative therapy delivery andhenceimprovestherapyoutcomes.
13
In thefuture,theparallelimplementationdescribedherecouldbeimprovedby addressingtheloadimbalancesin orderto achieve betterscalingperformance.A tetrahedralmeshwith a moreregularconnectivity patternwould allow betterscalingin thematrix assemblyprocess.Theparalleldecom-positionof thesystemof equationsto solvecouldbemodifiedto accountfor thedistributionof knowndisplacementsin orderto improvethescalingof thesolver. Improvedregistrationcouldresultfrom amoresophisticatedmodelof thematerialpropertiesof thebrain(suchasmoreaccuratemodellingofthecerebralfalx andthelateralventricles).Thecreationof anintraoperativesegmentationcapabilityableto identify thesestructureswouldbenecessaryto enablesucha modelto beappliedroutinelyinsurgery.
Acknowledgements: This investigationwas supportedby NIH NCRR P41 RR13218,NIH P01CA67165andNIH R01RR11747.Accessto theDeepFlow parallelcluster(http://www.meca.ucl.ac.be/mema/deepflow), fundedthroughtheARC97/02-210projectof theCom-munauteFrancaisedeBelgique,waskindly grantedto usby RolandKeunings.
References
[1] F. Jolesz, “Image-guidedProceduresand the OperatingRoom of the Future,” Radiology,vol. 204,pp.601–612,May 1997.
[2] A. Hagemann,K. Rohr, H. Stiel,U. Spetzger, andJ.Gilsbach,“Biomechanicalmodelingof thehumanheadfor physicallybased,non-rigidimageregistration,” IEEETransactionsOnMedicalImaging, vol. 18,no.10,pp.875–884,1999.
[3] O. SkrinjarandJ.Duncan,“Real time 3D brainshift compensation,” in IPMI’99, pp.641–649,1999.
[4] M. Miga, K. Paulsen,J. Lemery, A. Hartov, andD. Roberts,“In vivo quantificationof a ho-mogeneousbrain deformationmodel for updatingpreoperative imagesduring surgery,” IEEETransactionsOnMedicalImaging, vol. 47,pp.266–273,February1999.
[5] D. Hill, C. Maurer, R. Maciunas,J. Barwise,J. Fitzpatrick,andM. Wang,“Measurementofintraoperative brainsurfacedeformationundera craniotomy,” Neurosurgery, vol. 43, pp. 514–526,1998.
[6] N. Hata,Rigidanddeformablemedicalimageregistrationfor image-guidedsurgery. PhDthesis,Universityof Tokyo, 1998.
[7] M. Bro-Nielsen,“SurgerySimulationUsingFastFiniteElements,” in Visualizationin Biomedi-cal Computing, pp.529–534,1996.
[8] W. Schroeder, K. Martin, andB. Lorensen,TheVisualizationToolkit: An Object-OrientedAp-proach to 3D Graphics. PrenticeHall PTR,New Jersey, 1996.
[9] B. Geiger, “Threedimensionalmodelingof humanorgansandits applicationto diagnosisandsurgicalplanning,” Tech.Rep.2105,INRIA, 1993.
[10] M. Ferrant,S.K. Warfield,C. R. G. Guttmann,R. V. Mulkern,F. A. Jolesz,andR. Kikinis, “3DImageMatchingUsing a Finite ElementBasedElasticDeformationModel,” in MICCAI 99:
14
SecondInternationalConferenceon Medical Image Computingand Computer-AssistedInter-vention(C.TaylorandA. Colchester, eds.),pp.202–209,Springer-Verlag,Heidelberg,Germany,1999.
[11] P. M. Black, T. Moriarty, E. Alexander, P. Stieg, E. J. Woodard,P. L. Gleason,C. H. Martin,R. Kikinis, R. B. Schwartz,andF. A. Jolesz,“The DevelopmentandImplementationof Intra-operative MRI andits Neurosurgical Applications,” Neurosurgery, vol. 41, pp. 831–842,April1997.
[12] D. Gering,A. Nabavi, R. Kikinis, W. Grimson,N. Hata,P. Everett,F. Jolesz,andW. Wells,“AnIntegratedVisualizationSystemfor Surgical PlanningandGuidanceusing ImageFusionandInterventionalImaging,” in MICCAI 99: Proceedingsof theSecondInternationalConferenceonMedicalImage ComputingandComputerAssistedIntervention, pp.809–819,SpringerVerlag,1999.
[13] S.Warfield,J. Dengler, J.Zaers,C. R. Guttmann,W. M. Wells III, G. J.Ettinger, J. Hiller, andR. Kikinis, “Automaticidentificationof Grey MatterStructuresfrom MRI to Improve theSeg-mentationof White MatterLesions,” Journal of Image GuidedSurgery, vol. 1, no. 6, pp. 326–338,1995.
[14] S.K. Warfield,M. Kaus,F. A. Jolesz,andR. Kikinis, “AdaptiveTemplateModeratedSpatiallyVarying StatisticalClassification,” in MICCAI 98: First InternationalConferenceon MedicalImage ComputingandComputer-AssistedIntervention, pp. 231–238,Springer-Verlag,Heidel-berg, Germany, October11–131998.
[15] M. R. Kaus,S.K. Warfield,A. Nabavi, E.Chatzidakis,P. M. Black,F. A. Jolesz,andR. Kikinis,“Segmentationof MRI of meningiomasandlow gradegliomas,” in MICCAI99: SecondInterna-tional ConferenceonMedicalImageComputingandComputer-AssistedIntervention(C.TaylorandA. Colchester, eds.),pp.1–10,Springer-Verlag,Heidelberg, Germany, 1999.
[16] S.K. Warfield,M. Kaus,F. A. Jolesz,andR. Kikinis, “Adaptive,TemplateModerated,SpatiallyVaryingStatisticalClassification,” MedicalImageAnalysis, vol. 4, no.1, pp.43–55,2000.
[17] S.K. Warfield,F. A. Jolesz,andR. Kikinis, “Real-TimeImageSegmentationfor Image-GuidedSurgery,” in Supercomputing1998, pp.1114:1–14,November1998.
[18] R. Kikinis, M. E. Shenton,G. Gerig,J. Martin, M. Anderson,D. Metcalf, C. R. G. Guttmann,R. W. McCarley, W. E. Lorenson,H. Cline, andF. Jolesz,“Routine Quantitative AnalysisofBrainandCerebrospinalFluidSpaceswith MR Imaging,” Journalof MagneticResonanceImag-ing, vol. 2, pp.619–629,1992.
[19] I. Ragnemalm,“The Euclideandistancetransformin arbitrarydimensions,” PatternRecognitionLetters, vol. 14,pp.883–888,1993.
[20] W. M. Wells, P. Viola, H. Atsumi,S.Nakajima,andR. Kikinis, “Multi-modal volumeregistra-tion by maximizationof mutualinformation,” MedicalImageAnalysis, vol. 1, pp.35–51,March1996.
[21] R. O. DudaandP. E. Hart,PatternClassificationandSceneAnalysis. JohnWiley & Sons,Inc.,1973.
15
[22] J. Denglerand M. Schmidt, “The Dynamic Pyramid– A Model for Motion Analysis withControlledContinuity,” InternationalJournalof PatternRecognitionandArtificial Intelligence,vol. 2, no.2, pp.275–286,1988.
[23] S. K. Warfield,A. Robatino,J. Dengler, F. A. Jolesz,andR. Kikinis, “NonlinearRegistrationand TemplateDriven Segmentation,” in Brain Warping (A. W. Toga,ed.), ch. 4, pp. 67–84,AcademicPress,SanDiego,USA, 1999.
[24] D. V. Iosifescu,M. E. Shenton,S. K. Warfield,R. Kikinis, J. Dengler, F. A. Jolesz,andR. W.McCarley, “An AutomatedRegistrationAlgorithm for MeasuringMRI SubcorticalBrainStruc-tures,” NeuroImage, vol. 6, pp.12–25,1997.
[25] M. Ferrant,O. Cuisenaire,andB. Macq,“Multi-Object Segmentationof BrainStructuresin 3DMRI Using a ComputerizedAtlas,” in SPIEMedical Imaging ’99, vol. 3661-2,pp. 986–995,1999.
[26] O. C. Zienkiewicz andR. L. Taylor, TheFinite ElementMethod:BasicFormulationandLinearProblems. McGraw Hill BookCo.,New York, 4thed.,1994.
[27] S.Balay, W. D. Gropp,L. C. McInnes,andB. F. Smith,“Efficientmanagementof parallelisminobjectorientednumericalsoftwarelibraries,” in ModernSoftwareToolsin ScientificComputing(E. Arge,A. M. Bruaset,andH. P. Langtangen,eds.),pp.163–202,BirkhauserPress,1997.
[28] S.Balay, W. D. Gropp,L. C. McInnes,andB. F. Smith,“PETSc2.0usersmanual,” Tech.Rep.ANL-95/11- Revision2.0.28,ArgonneNationalLaboratory, 2000.
16
