EECS 274 Computer Vision Segmentation by Clustering II.
-
Upload
stephen-scott -
Category
Documents
-
view
223 -
download
0
Transcript of EECS 274 Computer Vision Segmentation by Clustering II.
![Page 1: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/1.jpg)
EECS 274 Computer Vision
Segmentation by Clustering II
![Page 2: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/2.jpg)
Segmentation and Grouping
• Watershed• Mean shift filtering and segmentation
• Reading: FP Chapter 14, S Chapter 5
![Page 3: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/3.jpg)
Watershed Segmentation Visualize an image in 3D: spatial coordinates and gray
levels. In such a topographic interpretation, there are 3 types of
points Points belonging to a regional minimum (basin) Points at which a drop of water would fall to a single minimum.
(The catchment basin or watershed of that minimum.) Points at which a drop of water would be equally likely to fall to
more than one minimum. (The divide lines or watershed lines.)
Watershed lines
![Page 4: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/4.jpg)
Watershed Segmentation The objective is to find watershed lines (via thresholding) The idea is simple:
Suppose that a hole is punched in each regional minimum and that the entire topography is flooded from below by letting water rise through the holes at a uniform rate.
When rising water in distinct catchment basins is about the merge, a dam is built to prevent merging. These dam boundaries correspond to the watershed lines.
![Page 5: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/5.jpg)
Watershed Segmentation
![Page 6: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/6.jpg)
• Start with all pixels with the lowest possible value– These form the basis for initial watersheds
• For each intensity level k:– For each group of pixels of intensity k
• If adjacent to exactly one existing region, add these pixels to that region• Else if adjacent to more than one existing regions, mark as boundary• Else start a new region
– Implemented with breadth-first search
Watershed Segmentation
![Page 7: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/7.jpg)
Watershed Segmentation
Watershed algorithm might be used on the gradient image instead of the original image.
![Page 8: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/8.jpg)
Watershed Segmentation
Due to noise and other local irregularities of the gradient, over segmentation might occur.
![Page 9: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/9.jpg)
Watershed Segmentation
A solution is to limit the number of regional minima. Use markers to specify the only allowed regional minima.
![Page 10: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/10.jpg)
Watershed Segmentation
A solution is to limit the number of regional minima. Use markers to specify the only allowed regional minima. (For example, gray-level values might be used as a marker.)
Often leads to over-segmentationUseful for interactive segmentation
![Page 11: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/11.jpg)
Mean Shift
• Non-parametric model• Efficiently finding peaks in
the high-dimensional data distribution without computing the complete function explicitly
• See– K. Fukunaga and L. Hostetler,
“The estimation of the gradient of a density function with applications in pattern recognition”, IEEE Trans. Information Theory, 1975
– Y. Cheng, “Mean shift, mode seeking, and clustering”, PAMI 1995
![Page 12: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/12.jpg)
Non-parametric density function• Smooth the data by convolving it with a
fixed kernel of width h
• Known as kernel density estimation or Parzen window technique
• Instead of using brute force approach, mean shift uses a variant of multiple restart gradient descent technique
)(
)()(2
2
rk
h
xxkxxKxf
i i
ii
is the kernel function or Parzen window
![Page 13: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/13.jpg)
1D mean shift
Starting at some initial guess for a local maximum, yk, which can beRandom input data pint xi, mean shift computes the gradient of density estimate f(x) at yk and takes uphill step
![Page 14: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/14.jpg)
Mean shift algorithm
• Compute the gradient of the density estimate f(x) at yk
• Rewrite the gradient function
• Mean shift is the difference between the weighted mean of neighbors xi around x and the current value of x
)(')(where
)()()()(2
2
rkrg
h
xxgxxxxGxxxf
i
ii
iii
mean shiftxxxG
xxGxxm
xmxxGxf
ii
iii
ii
thecalled is)(
)()(where
)()()(
![Page 15: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/15.jpg)
Modes
• The mode yk at iteration k is replaced by locally weighted mean
• Comaniciu and Meer show that this algorithm converges to a local maximum of f(x) under some reasonably weak condition on kernel k(r)
• Two kind of kernels:
• The corresponding derivative kernel g(r) are a unit ball and another is Gaussian, respectively
iik
iiki
kkk xyG
xyGxymyy
)(
)()(1
)exp()(
)1,0max()(
21 rrk
rrk
N
E
Epanechnikov kernel
Gaussian kernel
![Page 16: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/16.jpg)
Finding modes
• Simplest way– start a separate mean shift mode
estimate y at every input point xi
– and iterate for a fixed number of steps until the magnitude is below a threshold
• Faster way – randomly subsample the input points xi
– and find the nearest mode sample evolution path to the remaining points
![Page 17: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/17.jpg)
Mean shift segmentation
• Better results can be obtained with a combination of kernels, i.e., joint domain of color, r, and location, s
• Clustering of {xi} is achieved by placing a seed yi at each xi in n-d feature space (e.g., n=5)
2
2
2
2
)(s
s
r
rj
h
xk
h
xkxK
Regions containing less than M pixels are discarded
![Page 18: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/18.jpg)
Mean shift filtering and segmentation
![Page 19: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/19.jpg)
Mean Shift Segmentation
http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
![Page 20: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/20.jpg)
Mean Shift AlgorithmMean Shift Algorithm
1. Choose a search window size.2. Choose the initial location of the search window.3. Compute the mean location (centroid of the data) in the search window.4. Center the search window at the mean location computed in Step 3.5. Repeat Steps 3 and 4 until convergence.
The mean shift algorithm seeks the “mode” or point of highest density of a data distribution:
![Page 21: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/21.jpg)
Mean Shift Segmentation Algorithm1. Convert the image into tokens (via color, gradients, texture measures etc).2. Choose initial search window locations uniformly in the data.3. Compute the mean shift window location for each initial position.4. Merge windows that end up on the same “peak” or mode.5. The data these merged windows traversed are clustered together.
*Image From: Dorin Comaniciu and Peter Meer, Distribution Free Decomposition of Multivariate Data, Pattern Analysis & Applications (1999)2:22–30
Mean Shift Segmentation
![Page 22: EECS 274 Computer Vision Segmentation by Clustering II.](https://reader036.fdocuments.us/reader036/viewer/2022062421/56649f515503460f94c74149/html5/thumbnails/22.jpg)
Mean Shift Segmentation
http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html