Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department...
-
Upload
nichole-goffe -
Category
Documents
-
view
219 -
download
2
Transcript of Towards Robust Medical Image Segmentation Shaoting Zhang CBIM Center Computer Science Department...
Towards Robust Medical Image Segmentation
Shaoting ZhangCBIM Center
Computer Science DepartmentRutgers University
IntroductionBackground
• Segmentation (finding 2D/3D region-of-interest) is a fundamental problem and bottleneck in many areas.
• We focus on learning-based deformable models with shape priors (2D contour or 3D mesh).
Chest X-ray
Lung CADLiver in whole-
body CT (PET-CT)
Rat brain structures in MR Microscopy
2
IntroductionBackground
• End-to-end, automatic, accurate, efficient.
• Robustness– Handle weak or misleading
appearance cues.– Handle diseased cases (e.g.,
with tumor/cancer).– Leverage shape priors to
improve the robustness (Active Shape Model, T. Cootes, CVIU’95; 3D ASM for cardiac segmentation, Y. Zheng, TMI’08)
Segmentation system
3
Liver shape variations[Springer images]
IntroductionResearch Void
• Limitations of existing shape prior methods:– Assume Gaussian errors → Sensitive to outliers– Assume unimodal distribution of shapes → Cannot
handle large shape variations, e.g., multimodal– Only keep major variation → Lose local shape detail
4
IntroductionResearch Void
• Handling gross errors or outliers. – RANSAC + ASM [M. Rogers, ECCV’02]– Robust Point Matching [J. Nahed, MICCAI’06]
• Handling multimodal distribution of shapes.– Mixture of Gaussians [T. F. Cootes, IVC’97]– Manifold learning for shape prior [Etyngier, ICCV’07]– Patient-specific shape [Y. Zhu, TMI’10]
• Preserving local shape details.– Sparse PCA [K. Sjostrand, TMI’07]– Hierarchical ASM [D. Shen, TMI’03]
5
Need to solve all three
challenges simultaneously
in practice
MethodsSegmentation framework
6
Landmark Detectors
Offline Learning
Boundary Detectors
Runtime Segmentation
New Image
Model Initialize
Iterative Deformation and Shape Refinement
Final Results
Shape Priors via Sparse Shape Representation
Image Data and Manually Labeled Ground Truths
Critical anatomical landmarks.
MethodsSegmentation framework
• Initialization: Landmark detectors + shape prior– Automatic, fast (AdaBoosting/PBT/random forest + 3D Haar).
• Deformation: Boundary detectors + shape prior– Extend the landmark detectors to locate the sub-surfaces.
Critical anatomical landmarks.
7
Lung Heart
Rib
Colon
Abdomen
7
MethodsShape prior using sparse shape representation
• Our shape prior is based on two observations:– An input shape can be approximately represented by a
sparse linear combination of training shapes.– The given shape information may contain gross errors, but
such errors are often sparse....
≈
8
MethodsShape prior using sparse shape representation
• Formulation:–
• Sparse linear combination:–
...≈
...≈
Dense xSparse xAligned shape data matrix DWeight x
Input y
Global transformationparameter
Number of nonzero elements
Global transformationoperator
9
MethodsShape prior using sparse shape representation
• Non-Gaussian errors:–
10
MethodsShape prior using sparse shape representation
• Why it works?– Robust: Explicitly modeling “e” with L0 norm constraint.
Thus it can detect gross (sparse) errors, i.e., non-Gaussian– General: No assumption of any parametric distribution
model (e.g., a unimodal distribution assumption in ASM). Thus it can model large shape variations.
– Lossless: It uses all training shapes. Thus it is able to recover detail information even if the detail is not statistically significant in training data.
11
MethodsOptimization
• Convex relaxation [Candes, Tao, T-IT’06]:
• Tuning parameters (λ1 and λ2) are not difficult to choose, compared to k1 and k2. – λ1 x║ ║1, λ1 controls the sparsity of x. – λ2 e║ ║1, λ2 controls the sparsity of e.
• One group of parameters work well for all images.• Efficient convex optimization: fast proximal gradient
methods (e.g., FISTA)12Shaoting Zhang
MethodsScalability
13Shaoting Zhang
... ≈... ...
Dictionary size = 64 #Coefficients = 1000 #Input samples = 1000
1. To efficiently handle many training samples - dictionary learning:
2. To efficiently handle thousands of vertices - mesh partitioning:
Applications – Part I 2D lung localization in X-ray
(Lung computer-aided diagnosis system, Siemens)• Handling gross errors
Detection PA ASM RASM NN TPS Sparse1 Sparse2
Procrustes analysisActive Shape ModelRobust ASMNearest NeighborsThin-plate-splineWithout modeling “e”
Proposed method
14
Applications – Part I2D lung localization in X-ray
• Multimodal shape distribution
Detection PA ASM/RASM NN TPS Sparse1 Sparse2
15
Applications – Part I2D lung localization in X-ray
• Recover local detail information
Detection PA ASM/RASM NN TPS Sparse1 Sparse2
16
Applications – Part I2D lung localization in X-ray
• Sparse shape components
• ASM modes:
17
≈
0.5760
+ +
0.2156 0.0982
Applications – Part I2D lung localization in X-ray
• Mean values and standard deviations. ~1,000 cases.
Left lung
Right lung
1)PA, 2)ASM, 3)RASM, 4)NN, 5)TPS, 6)Sparse1, 7)Sparse2
µσ
18
Applications – Part II3D liver segmentation in low-dose CT
19
• Use case: Improved interpretation of PET-CT, with Siemens– PET: Measure functional process (metabolic activity), oncology– CT: Provide anatomical information, co-registered with PET– Issues: High variations of activities across different organs– Solution: Segment organs in CT for organ-specific interpret.– Challenge: Low dose results in low contrast & fuzzy boundary
Applications – Part II3D liver segmentation in low-dose CT
Procrustesanalysis Sparse shape Ground truth
Initialization
Deformation
20
Same landmarks + different shape priors
Same deformation module
Applications – Part II3D liver segmentation in low-dose CT
Procrustesanalysis
Sparse shape
21
Applications – Part II3D liver segmentation in low-dose CT
Procrustesanalysis
Sparse shape
22
Applications – Part II3D liver segmentation in low-dose CT
Procrustesanalysis
Sparse shape
23
Applications – Part II3D liver segmentation in low-dose CT
Procrustesanalysis
Sparse shape
24
Applications – Part II3D liver segmentation in low-dose CT
ASM-type [Zhan’09] Sparse shape Ground truth
• Shape refinement during segmentation
25
Applications – Part II3D liver segmentation in low-dose CT
• Quantitative results: surface distances. ~80 cases
Mean value and standard deviation (voxel)
26
Applications – Part IIISegmentation of Multiple Brain Structures in MRI
Rat brain structures Drug addiction and alcoholism
(with Brookhaven National Lab)
Human brain, 34 structures Alzheimer's disease
(with GE Global Research)
27
Summary of Robust Segmentation• Robustly handle abnormal cases, such as diseased cases
(liver tumor). Critical to healthcare applications such as computational diagnosis systems.
• Patent with Siemens. Used in several clinical applications. Key contribution for our awarded NSF-MRI grant (’12-’16).
• Relevant publications:– First author papers
• MICCAI 2012, 2011 (MICCAI Young Scientist Award Finalist)• CVPR 2011• Two journal papers in Medical Image Analysis
– Second author paper• Medical Physics 2013 (with my co-mentored student, G. Wang)• ISBI 2013, oral (with my co-mentored student, Z. Yan)28