Model-Based Organ Segmentation: Recent Methods

Model-Based Organ Segmentation: Recent MethodsJiun-Hung ChenGeneral Exam Paper200912Problem StatementLearn how to segment new, unseen CT images from a set of training CT images with ground truth organs marked.

Goal: Minimize the training errors while generalizing to the new CT images2Problem Organ: The Liver3

4Why Difficult? (Shape Variations)

45Why Difficult? (Similar Appearances)

56Why Difficult? (Appearance Changes)

67Why Difficult? (Position Changes)

78Active Shape Model Based FrameworkTraining CT Volumes3D point correspondenceLearning Statistical Shape Models Known point correspondencesLearning Organ DetectionLearning Boundary Intensity ModelBoundary Refinement Organ DetectionA Testing CT VolumeShape Model InitializationLearned Statistical Shape ModelLearned Boundary Intensity Model

Organ DetectorDetected Bounding BoxInitial ShapeFinal ShapeTraining phaseTesting phase

Input

8Active Shape Models: Training9Shapes are modeled in a training phase using a set of CT volumes whose ground truth segmentations are given.

There are 4 steps to the training phase.

1. Find 3D point correspondences on training meshes.2. Learn a statistical 3D shape model of the shapes.3. Learn a boundary intensity model for each vertex.4. Learn an organ detector that finds bounding boxes.10P1P2P3P1P2P3P2P1P3P1P2P3P1P2P3P2P1P3InputOutput

10

PSpin images for point P3-D facesCorresponding points on head meshes plus theirnumeric (spin image) signatures from the work ofSalvador Ruiz Correa. 12P1P2P3P1P2P3P2P1P3xyzInputOutput

1213Prob.Intensity profilesInputOutput

Learned Boundary Intensity ModelIntensity profilesP1P2P3P1P2P3P2P1P3

1314x,y,zrxryrzsxsyszx,y,zrxryrzsxsyszx,y,zrxryrzsxsyszInputOutput

Given a bounding box and the CT slices inside it, a classifier learns to decide if everything inside the box is liver or not.livernot liver14Active Shape Models: Testing15 There are 3 steps to the testing phase

organ detection: use the learned organ detector to detect the organ in the testing volume and return a bounding box2. shape model initialization: initialize the learned statistical model based on the detected bounding boxboundary refinement: use the learned boundary intensity model to estimate the refinement to the model for this shape

16Learned Statistical Shape Model

Test OrganBoundary Refinementx,y,zrxryrzsxsyszFind Bounding BoxInitialize Shape ModelOrgan Detector16Methods for Point Correspondences17Principal Component Analysis (PCA)

PCA takes in the points of each shape in the training set. It produces a set of basis vectors (the components).

Each shape can then be represented as a linear combinationof these components.

The optimal K projection axes bk, k = 1 to K are theeigenvectors of the covariance matrix of the training set of points corresponding to the K largest eigenvalues.

x = x + ckbk where x is the mean shape~ k=1KIntuitive Meaning of Principal Components18eigenvector corresponding to highesteigenvalueeigenvector corresponding to secondeigenvalue3 eigen-imagesmeanimage

trainingimageslinearapproxi-mationsEigenimages for Face Recognition203D Point Correspondence(MDL)Goal: Find 3D Point CorrespondenceIdea: Minimize MDL-based objective function Evaluate the quality of the correspondence

The ks are the eigenvalues from PCA.How: Gradient descent Manipulate correspondences by parameterization and re-parameterization.

Davies et al. [IEEE TMI02]

2021Statistical Shape Models Principal Component Analysis (PCA)Kernel PCABoundary Intensity ModelsGaussian distributionAdaBoosted histogram classifiersHeuristics

Cootes et al. [IEEE PAMI 01], Li [ICCV05], Kainmuller et al. [MICCAI07]Cootes et al. [IEEE PAMI 01], Twining et al. [BMVC01]2122Organ Detection (MSL)Goal: Find the bounding boxThe parameter space is 9D.3D positions, 3D scales and 3D orientations. IdeaUniform and exhaustive search is unnecessaryHow: decompose the problem into three stepsposition estimation, position-scale estimation and finally position-scale-orientation estimation. Zheng et al. [ICCV07]

2223Two ASM-based SystemsStatistical shape modelsPCA 43 CT volumes Boundary intensity modelHeuristicsLiver detectionLungs detection and DICOM infoPerformanceRanked first in a recent liver segmentation competition. 10 testing volumes1.1mm (the average symmetric surface distance)15 minutes.Statistical Shape modelsPCA, hierarchical shape pyramids75 volumesBoundary intensity modelA boundary classifierLiver detectionMSL (marginal space learning)Performance5 fold cross validation1.59 mm (the average symmetric surface distance)1.38 mm (the median) 12 seconds.

Kainmuller et al. [MICCAI07]Ling et al. [CVPR08]

Statistical Shape ModelsBoundary Intensity ModelLiver DetectionPerformance2324Experimental SettingDatasets: 4 types of organs (livers, left kidneys, right kidneys, spleens)15-20 subjects Leave-one-out cross validationMeasure the reconstruction errorMetrics: Euclidean and Hausdorff distance2425MDL-2DPCA

MDL-based objective function

Idea: Generalize the objective function to 2DPCA spaceReplace eigenvalues from PCA with from 2DPCAHow: Gradient descentComparisons: original MDL vs. MDL-2DPCA

* Chen and Shapiro [EMBC09]

2526Results (3D Point Correspondences)

# eigenvectorsReconstruction errorLiversLeft KidneysRight KidneysSpleensOriginal MDLMDL-2DPCA2627Tensor-based SSMIdea: Tensor-based dimension reduction methods2DPCAParafac model Tucker decompositionComparisons: PCA vs. Tensor-based dimension reduction* Chen and Shapiro [to appear in EMBC09]

2728

Results (Statistical Shape Models)# eigenvectorsReconstruction errorLiversLeft KidneysRight KidneysSpleensParafac2DPCAPCATucker

2829Organ Detection(Boosting Approach)Idea: Classify whether an image block contains an organ of interestHow:Partition slices into non-overlapping 32x32 blocksGlobal features: gray-tone histogram of the image slice and its slice indexLocal features: the position of a block, the mean and variance of its intensity values, and its intensity histogram. 20,000 SVM linear classifiers + AdaboostingComparisons: Manual vs. Adaboosting

2930Results (Organ Detection)Positive (predicted)Negative (predicted)Positive (actual)96.23%3.77%Negative (actual)4.57%95.43%Positive (predicted)Negative (predicted)Positive (actual)91.23%8.77%Negative (actual)6.57%93.43%Livers (Training)Livers (Testing)3031

Sub. 1Sub. 2True Positive : Green,False Positive: BlueFalse Negative: Red,True Negative: Cyan3132Graph Cuts Based Boundary RefinementIdea: Adding hard constraints to min s-t cutsMin s-t cuts with side constraintsNP-hard in general casesApproximation algorithm: standard rounding algorithmComparisons: with constraints vs. without constraintsChen and Shapiro [ICPR08]

3233Results (Boundary Refinement)

withoutwithInitial ContourSlice 1Slice 233