CS654: Digital Image Analysis Lecture 30: Clustering based Segmentation Slides are adapted from:...
-
Upload
garry-white -
Category
Documents
-
view
215 -
download
0
Transcript of CS654: Digital Image Analysis Lecture 30: Clustering based Segmentation Slides are adapted from:...
CS654: Digital Image Analysis
Lecture 30: Clustering based Segmentation
Slides are adapted from: http://www.wisdom.weizmann.ac.il/~vision/
Recap of Lecture 26
โข Thresholding
โข Otsuโs method
โข Region based segmentation
โข Region growing, split-merge, quad-tree
โข Clustering, K-means
Outline of Lecture 27
โข Overview about Mean Shift Segmentation
โข Mean Shift algorithmโข Kernel density estimation โข Kernel
โข Mean shift segmentation
โข Applications of Mean Shift โข Clusteringโข Filtering
Segmentation methods โ overview
โข Edge based
โข Thresholding
โข K-means clusteringโข The k-means algorithm is an algorithm to cluster n objects
based on attributes into k partitions, k < n
โข Mean-shift algorithm
โข Normalized cuts
K-Means pros and cons
โข Prosโข Simple and fastโข Easy to implement
โข Consโข Need to choose Kโข Sensitive to outliers
Mean-shift โ motivation and intuitive description
โข Given a distribution of points, mean shift is a procedure for finding the densest region.
โข Example for simple 2D case
1. Start from arbitrary point in the distribution
2. Region of interest is a circle centered in this point
3. On each iteration find the center of the mass for the region of
interest
4. Move the circle to the center of the mass
5. Continue the iterations until convergence
Intuitive Description
Distribution of identical billiard balls
Region ofinterest
Center ofmass
Mean Shiftvector
Objective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
Intuitive Description
Distribution of identical billiard balls Objective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
Intuitive Description
Distribution of identical billiard balls Objective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
Intuitive Description
Distribution of identical billiard balls Objective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
Intuitive Description
Distribution of identical billiard balls Objective : Find the densest region
Region ofinterest
Center ofmass
Mean Shiftvector
Intuitive Description
Distribution of identical billiard balls Objective : Find the densest region
Region ofinterest
Center ofmass
Intuitive Description
Distribution of identical billiard balls Objective : Find the densest region
Mean-shift โ algorithm formal definition
โข The Basic Mean Shift Algorithm is formulated according to the following paper:
D. Comaniciu, P. Meer: Mean Shift Analysis and Applications, IEEE Int. Conf. Computer Vision (ICCV'99), Kerkyra , Greece , 1197-1203, 1999
Mean-shift โ algorithm formal definition
Given:
โข A set of n points in the d-dimensional space
Model:
โข Assume non-parametric statistical model,
โข There is a probability density function (PDF) associated with the set of points,
โข Without any assumptions on its parameters.
Goal:
โข For any given point find closest local mode of the density function.
PDF in feature space
โข Color spaceโข Scale spaceโข Actually any feature space you can conceiveโข โฆ
What is Mean Shift ?
Non-parametricDensity Estimation
Non-parametricDensity GRADIENT Estimation
(Mean Shift)
Data
Discrete PDF Representation
PDF Analysis
A tool for:Finding modes in a set of data samples, manifesting an underlying probability density function (PDF) in RN
Non-Parametric Density Estimation
Assumption : The data points are sampled from an underlying PDF
Assumed Underlying PDF Real Data Samples
Data point density implies PDF value !
Non-Parametric Density Estimation
Assumed Underlying PDF Real Data Samples
Assumed Underlying PDF Real Data Samples
Non-Parametric Density Estimation
Parametric Density Estimation
Assumption : The data points are sampled from an underlying PDF
2
2
( )
2
i
PDF( ) = i
iic e
x-ฮผ
x
Estimate
Assumed Underlying PDF Real Data Samples
Kernel Density Estimation
1
1( ) ( )
n
ii
P Kn
x x - xA function of some finite number of data pointsx1โฆxn
DataIn practice one uses the forms:
1
( ) ( )d
ii
K c k x
x or ( )K ckx x
Same function on each dimension Function of vector length only
Parzen Windows - Function Forms
Profile
Various Kernels
Examples:
โข Epanechnikov Kernel
โข Uniform Kernel
โข Normal Kernel
21 1
( ) 0 otherwise
E
cK
x xx
1( )
0 otherwiseU
cK
xx
21( ) exp
2NK c
x x
Data
1
1( ) ( )
n
ii
P Kn
x x - xA function of some finite number of data pointsx1โฆxn
Kernel Density Estimation
Gradient
1
1 ( ) ( )
n
ii
P Kn
x x - x Give up estimating the PDF !Estimate ONLY the gradient
2
( ) iiK ck
h
x - xx - xUsing the
Kernel form:
Size of window
๐ป ๐ (๐ )= 1๐โ
๐=1
๐
๐ป {๐๐(โ๐โ๐ ๐
h โ2
)}
Kernel Density Gradient Estimation
๐ป ๐ (๐ )= 1๐โ
๐=1
๐
๐ป {๐๐(โ๐โ๐ ๐
h โ2
)}โ๐ป๐ (๐ )=๐
๐โ๐=1
๐
๐ โฒ(โ๐โ ๐๐
h โ2
) .2.(๐โ ๐๐ )h2
โ๐ป๐ (๐ )= 2๐๐ h2 โ
๐=1
๐
( ๐โ๐ ๐ )๐ โฒ (โ๐โ๐ ๐
h โ2
)Let,
๐ป ๐ (๐ )= 2๐๐h2โ
๐=1
๐
(๐ ๐โ ๐ )๐(โ๐โ๐ ๐
h โ2
)
Kernel Density Gradient Estimation
๐ป ๐ (๐ )= 2๐๐h2โ
๐=1
๐
(๐ ๐โ ๐ )๐(โ๐โ๐ ๐
h โ2
)โ๐ป๐ (๐ )= 2๐
๐ h2 [โ๐=1
๐
๐๐๐ (โ๐โ ๐ ๐
h โ2
)โโ๐=1
๐
๐๐ (โ๐โ ๐ ๐
h โ2
)]
โ๐ป๐ (๐ )= 2๐๐ h2 โ
๐=1
๐
๐(โ๐โ๐ ๐
h โ2
) [โ๐=1
๐
๐ ๐๐ (โ๐ โ๐ ๐
h โ2
)โ๐=1
๐
๐(โ๐โ๐ ๐
h โ2
)โ๐ ]
โ๐ป๐ (๐ )= 2๐๐ h2 โ
๐=1
๐
๐(โ๐โ๐ ๐
h โ2
) [โ๐=1
๐
๐ ๐๐ (โ๐ โ๐ ๐
h โ2
)โ๐=1
๐
๐(โ๐โ๐ ๐
h โ2
)โ๐ ]
Yet another Kernel density estimation !
Simple Mean Shift procedure:โข Compute mean shift vector
โขTranslate the Kernel window by m(x)
2
1
2
1
( )
ni
ii
ni
i
gh
gh
x - xx
m x xx - x
Computing the Mean Shift
Mean Shift Procedure
At density maxima
โ๐ป๐ (๐ )= 2๐๐ h2 โ
๐=1
๐
๐(โ๐โ๐ ๐
h โ2
) [โ๐=1
๐
๐ ๐๐ (โ๐ โ๐ ๐
h โ2
)โ๐=1
๐
๐(โ๐โ๐ ๐
h โ2
)โ๐ ]=0
mean shift vector โ must be 0 at optimum
โข Automatic convergence speed โ the mean shift vector size depends on the gradient itself.
โข Near maxima, the steps are small and refined
โข Convergence is guaranteed for infinitesimal steps only infinitely convergent, (therefore set a lower bound)
โข For Uniform Kernel ( ), convergence is achieved in a finite number of steps
โข Normal Kernel ( ) exhibits a smooth trajectory, but is slower than Uniform Kernel ( ).
AdaptiveGradient Ascent
Mean Shift Properties
Tessellate the space with windows Run the procedure in parallel
Real Modality analysis
Attraction basin
โข Capture theorem
โข Attraction basin:
โข The region for which all trajectories lead to the same mode
โข Cluster: all data points in the attraction basin of a mode
Clustering: Real Example
L*u*v space representation
( )s r
s rs r
K C k kh h
x xx
Clustering: Real Example
Not all trajectoriesin the attraction basinreach the same mode
2D (L*u) space representation
Final clusters
Mean shift clustering
The mean shift algorithm seeks modes of the given set of points
1. Choose kernel and bandwidth
2. For each point:a) Center a window on that pointb) Compute the mean of the data in the search windowc) Center the search window at the new mean locationd) Repeat (b,c) until convergence
3. Assign points that lead to nearby modes to the same cluster
Segmentation by Mean Shift
1. Find features (color, gradients, texture, etc)
2. Set kernel size for features Kr and position Ks
3. Initialize windows at individual pixel locations
4. Perform mean shift for each window until convergence
5. Merge windows that are within width of Kr and Ks
Feature space : Joint domain = spatial coordinates + color space
( )s r
s rs r
K C k kh h
x xx
http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
Mean shift segmentation results
http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
Mean shift segmentation results
Application of Mean Shift: Filtering
โข Image filtering is a process by which we can enhance, modify or multilate images.
โข Filtering reduces the influence from noise to mode detection.
โข Mean shift filtering can work with binary, gray scale, RGB and arbitrary multichanel images.
โข Filtering is the first step of mean shift segmentation process.
โข A second step is the clustering of filtered data point
Mean Shift Filtering
โข Let the d-dimensional input image be the filtered image
Algorithm
1. Initialize , and
2. Compute till convergence
3. Assign
Spatial location will have range component of the point of convergence
Examples
Examples
Comparison with Gaussian
Mean-shift: other issues
โข Speedupsโข Uniform kernel (much faster but not as good)
โข Binning or hierarchical methods
โข Approximate nearest neighbor search
โข Methods to adapt kernel size depending on data density
Mean shift pros and cons
โข Prosโข Good general-practice segmentationโข Finds variable number of regionsโข Robust to outliers
โข Consโข Have to choose kernel size in advanceโข Original algorithm doesnโt deal well with high dimensions
โข When to use itโข Over-segmentatoinโข Multiple segmentationsโข Other tracking and clustering applications
Watershed algorithm
Watershed segmentation
Image Gradient Watershed boundaries
Meyerโs watershed segmentation
1. Choose local minima as region seeds
2. Add neighbors to priority queue, sorted by value
3. Take top priority pixel from queue1. If all labeled neighbors have same label, assign to pixel2. Add all non-marked neighbors
4. Repeat step 3 until finished
Matlab: seg = watershed(bnd_im)
Simple trickโข Use median filter to
reduce number of regions
Watershed pros and cons
โข Prosโข Fast (< 1 sec for 512x512 image)โข Among best methods for hierarchical segmentation
โข Consโข Only as good as the soft boundariesโข Not easy to get variety of regions for multiple segmentationsโข No top-down information
โข Usageโข Preferred algorithm for hierarchical segmentation
Thank youNext Lecture: Graph based approach