Texture Analysis and SegmentationAnalysis and Segmentation...
Transcript of Texture Analysis and SegmentationAnalysis and Segmentation...
Texture Analysis and SegmentationTexture Analysis and Segmentation using Modulation Models
Department of Mathematics, UCLAI P i S iImage Processing Seminar
Iasonas KokkinosDepartment of Statistics, UCLA
Joint work with Georgios Evangelopoulos and Petros Maragos,
Department of ECE, National Technical University of Athens, Greece
Presentation Outline
Amplitude Modulation- Frequency Modulation (AM-FM) models
2-D AM-FM Model2-D AM-FM ModelEnergy Separation Algorithm, Regularized DemodulationDominant Component Analysis (DCA)
Filtering and modelling
Model-based interpretation of Gabor filtering Model based interpretation of Gabor filtering Alternative models for edge and smooth signalsTexture / edge / smooth classification via model comparison
Applications to Segmentation
Variational Image Segmentation using AM-FM featuresWeighted Curve Evolution for cue combination
1-D AM-FM Models
AM FM AM-FM
Applications: Telecommunications, Speech Analysis ...
2-D AM-FM models
Monocomponent AM-FM signal
Multicomponent AM-FM signals
= +
AM-FM models for Natural Images
Man-made structures
Results of natural processes
AM-FM Demodulation: Energy Separation Algorithm
Given, recover s.t.Assume bandpass modulating signalsTeager-Kaiser Energy Operator:
Energy Separation Algorithm:
Compared with Hilbert transform: locality
Refs. 1-D: Maragos, Quattieri & Kaiser, IEEE TSP ‘92, 2-D: Maragos & Bovik, JOSA ‘95
Natural Image Demodulation
Problems:Natural images do not satisfy ESA assumptionsNatural images do not satisfy ESA assumptionsDecomposition into AM-FM components: ill-posed problemEffects of noise and approximations of derivatives
G b filt i l tiGabor filtering solution:Break signal into simple components by Gabor filtering
ertic
al F
requ
ency
Fourier transform
isocurves
Horizontal Frequency
Verisocurves
Demodulate individual outputsUse derivative-of-Gabor filters to avoid differentiation
Channelized & Dominant Component AnalysisHavlicek & Bovik IEEE TIP ’00Havlicek & Bovik, IEEE TIP 00
DCA:
DCA reconstruction of textured signals
Presentation Outline
Amplitude Modulation- Frequency Modulation (AM-FM) models
2-D AM-FM Model2-D AM-FM ModelEnergy Separation Algorithm, Regularized DemodulationDominant Component Analysis (DCA)
Filtering and Modelling
Model-based interpretation of Gabor filtering Model based interpretation of Gabor filtering Alternative models for edge and smooth signalsTexture / edge / smooth classification via model comparison
Applications to Segmentation
Variational Image Segmentation using AM-FM featuresWeighted Curve Evolution for cue combination
Motivation: deciding when to trust texture features
Input Image DCA Features
Model-based approachDetermine where the model fits the image wellWell = better than alternatives: Bayesian approach
`Special treatment’ for textured regions:F lk Shi & M lik N li d C t f S t tiFowlkes, Shi & Malik, Normalized Cuts for Segmentation Meyer, Vese, Osher, U+V decompositionGuo, Wu, Zhu, Texture + Sketch for reconstruction
Bayesian approach
Synthesis model for each class
Adopt probabilistic error model I t t t t t b ti lik lih d i lIntegrate out parameters to express observation likelihood given class
Derive class posterior using Bayes’ rule
Model 1 D profile along principal orientation:
Texture Model: sinusoidModel 1-D profile along principal orientation:
Rewrite as expansion on linear basis:
Typical Matched filtering: Project signal on sine/cosine basis (convolution with sine/cosine filters)
Gabor filtering: Filters have falloff (local analysis)
Probabilistic formulation of locality
Leave distant data for a background model
b ti t i tobservation at point xmodel-based predictionprobability that observationi d t f d d lis due to foreground model
Lower bound of likelihood
Likelihood for independent errors
White Gaussian noise: weighted least squares
Gabor filtering as a weighted projection on a linear basis
Rewrite lower bound in matrix form
1Texture model components
Weighted least squares estimate
0.5
1DCEvenOddCertainty
Weighted least squares estimate
−30 −20 −10 0 10 20 30−0.5
0
For diagonal : parameters obtained by Gabor/Gaussian responses at
Relation between Amplitude and bound
Alternative HypothesesCast edge detection in same setting:
Phase congruency model for edges & lines:
Rewrite as expansion on basis:
Edge model components
0.4
0.6
0.8
1
1.2Edge model components
DCEvenOddCertainty
−30 −20 −10 0 10 20 30−0.4
−0.2
0
0.2
Iterate previous stepsConnection with Energy-based edge detection - QFPs
Morrone & Owens ‘87, Perona & Malik ’90,
Smooth signal:
Structure captured by the Edge and Texture models
Input Edge Reconstruction Texture Reconstruction
Texture/Edge/Smooth discrimination in 2D imagesFor each scale/orientation combination use all three models
Use Gabor/Edge/Gaussian filters to estimate model parameters
Quantify gain of Edge/Texture hypothesis vs Smooth hypothesisQuantify gain of Edge/Texture hypothesis vs. Smooth hypothesis
Normalize for scale invariance: per-pixel gain
Compute class posteriors
Text/Edge/Smooth Hypothesis ClassificationIntensity Texture Amplitude Edge Amplitude
Posterior Probabilities
Prob(Smooth) Prob(Texture) Prob(Edge)
Texture vs. Edge discriminationIntensity Prob(Texture) Prob(Edge)( ) ( g )
Presentation Outline
Amplitude Modulation- Frequency Modulation (AM-FM) models
2-D AM-FM Model2-D AM-FM ModelEnergy Separation Algorithm, Regularized DemodulationDominant Component Analysis (DCA)
Probabilistic Aspects
Model-based interpretation of Gabor filtering Model based interpretation of Gabor filtering Alternative models for edge and smooth signalsTexture / edge / smooth classification via model comparison
Applications to Segmentation
Variational Image Segmentation using AM-FM featuresW i ht d C E l ti f bi tiWeighted Curve Evolution for cue combination
Variational Image Segmentation
Mumford & Shah ’89 Zhu & Yuille, ’96: Region Competition Functional
Level Set framework:Chan & Vese, Scale-Space ’99, Yezzi, Chai & Willsky, ICCV ’99Yezzi, Chai & Willsky, ICCV 99Paragios & Deriche, ICCV ’99, ECCV ’00
Combination with Geodesic Active Contours (Paragios & Deriche):
Features for Variational Texture Segmentation
Filterbank-based methodsZhu & Yuille, PAMI ‘96: Small filterbank, few results on textureZhu & Yuille, PAMI 96: Small filterbank, few results on textureParagios & Deriche, IJCV ‘02: SupervisedSagiv, Sochen et al. , ‘02. Sandbert Chan & Vese et al, ‘02 : Feature selection
HistogramsKim, Fisher & Willsky, ICIP `01: Nonparametric estimate of intensity Tu & Zhu PAMI ’02: Histograms of intensity + model calibrationTu & Zhu, PAMI 02: Histograms of intensity + model calibration
Low dimensional descriptorsZ H li k A t & P tti hi ICIP ‘01 M d l ti f t l t iZray, Havlicek, Acton & Pattichis, ICIP ‘01: Modulation features + clusteringVese & Osher, JSC ’02, features from decompositionRousson, Brox & Deriche, CVPR ‘03: Anisotropic diffusion + structure tensor.
Modulation features via Dominant Component Analysis
DCA
Variational Segmentation with Modulation Features
3-dimensional feature vectorAmplitude function: ContrastMagnitude of frequency vector: ScaleMagnitude of frequency vector: ScaleAngle of frequency vector: Orientation
Smooth, low-dimensional descriptorGaussian distribution for von-Mises forGaussian distribution for , von Mises for
Initialize segmentation randomly and iterate: Estimate region parameters using current segmentationEstimate region parameters using current segmentation Modify segmentation by curve evolution
Cue Combination TaskIntensity Prob(Smooth)
Texture Features P b(T t )Texture Features Prob(Texture)
Edge Strength Prob(Edge)Edge Strength Prob(Edge)
Classifier Combination Approach
Treat probabilistic balloon force of RC as log-odds of two-class classifier
Decide about pixel label by comparing feature likelihoods Consider separate classifiers based on texture/intensity/edge cues
`Supra –Bayesian’ classifier combination, a.k.a. `stacking’Treat classifier outputs themselves as random variables
Ideally, Consider joint distribution of vector of classifier log-odds.For independent classifiers s.t. decision is given by
Weighted Curve Evolution
Last slide summary: give higher weight to log-odds of better classifier
Adaptation to curve evolution: set weights equal to class posteriorsWeighted curve evolution:Weighted curve evolution:
Compare to Geodesic Active RegionsCompare to Geodesic Active Regions
Geodesic Active Regions Weighted Curve Evolution
Segmentation Result ComparisonsIn
put
)D
CA
(pla
in)
Dox
et.
al.
Bro
WC
ED
CA
+
Quantitative EvaluationBerkeley Benchmark: 100 hand-segmented images (test-set)Bidirectional Consistency Error
At each pixel: normalized set difference of machine- and user- regions
Make symmetric, take minimum over users, and averagey g
Precision-Recall
Berkeley Dataset Segmentations
Conclusions & Future Work
AM FM models: naturally suited for modelling oscillationsEfficient and reliable parameter estimationL di i l d i tLow-dimensional descriptors
Model-based interpretation of feature extractionGabor filteringEnergy-based feature detection
Cue Combination for Curve Evolution
Future workAM FM models: synthesis, PDE methods (G. Evangelopoulos)I t t ith th t tIntegrate with other structures
Crosses, junctions, blobs, ridgesUse segmentation to drive object detection
U t l t i t tUse segments as elementary image structures Construct segment-based object representations
Synthetic signal reconstruction Constant Edge Texture
nt 0.7
0.8
0.9
1Weighted MSE: 0.401
0.7
0.8
0.9
1Weighted MSE: 0.308
0.7
0.8
0.9
1Weighted MSE: 0.388
Con
sta
50 60 70 80 90 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
50 60 70 80 90 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
50 60 70 80 90 1000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Edg
e
50 60 70 80 90 100
0.4
0.5
0.6
0.7
0.8
0.9
1Weighted MSE: 1.893
0.5
0.6
0.7
0.8
0.9
1Weighted MSE: 0.138
0.4
0.5
0.6
0.7
0.8
0.9
1Weighted MSE: 1.507
Ee
50 60 70 80 90 1000
0.1
0.2
0.3
0.4
50 60 70 80 90 1000
0.1
0.2
0.3
0.4
0.5
50 60 70 80 90 1000
0.1
0.2
0.3
0.4
1Weighted MSE: 1.216
1Weighted MSE: 1.665
1Weighted MSE: 0.327
Text
ure
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
50 60 70 80 90 1000
0.1
0.2
50 60 70 80 90 1000
0.1
0.2
50 60 70 80 90 1000
0.1
0.2