2IL90: Graph Drawing Introduction Fall 2006. Graphs Vertices Edges.
Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for...
-
Upload
ernest-green -
Category
Documents
-
view
230 -
download
0
Transcript of Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for...
![Page 1: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/1.jpg)
Graph-based Graph-based SegmentationSegmentation
![Page 2: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/2.jpg)
Main IdeasMain Ideas
Convert image into a graphConvert image into a graph Vertices for the pixelsVertices for the pixels Edges between the pixelsEdges between the pixels Additional vertices and edges to encode other Additional vertices and edges to encode other
constraintsconstraints
Manipulate the graph to segment the Manipulate the graph to segment the imageimage
![Page 3: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/3.jpg)
PapersPapers
Interactive graph cuts for optimal boundary Interactive graph cuts for optimal boundary & region segmentation of& region segmentation ofobjects in N-D images objects in N-D images Boykov and JollyBoykov and Jolly Minimize an energy functionMinimize an energy function
Efficient Graph-based SegmentationEfficient Graph-based Segmentation Felzenszwalb and HuttenlocherFelzenszwalb and Huttenlocher Cluster the vertices based on edge weightCluster the vertices based on edge weight
![Page 4: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/4.jpg)
Boykov and JollyBoykov and Jolly
Binary image segmentationBinary image segmentation Classify pixels as object or backgroundClassify pixels as object or background Their contribution is adding interactivityTheir contribution is adding interactivity
Minimise an energy functionMinimise an energy function E(A) = B(A) + E(A) = B(A) + λλR(A)R(A)
A: Segmentation (assign each pixels to object A: Segmentation (assign each pixels to object or background)or background)B(A): The cost of all the edges between object B(A): The cost of all the edges between object pixels and background pixelspixels and background pixelsR(A): The cost of deciding a pixel to be object or R(A): The cost of deciding a pixel to be object or backgroundbackground
![Page 5: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/5.jpg)
Creating the GraphCreating the Graph
Each pixel has a corresponding vertexEach pixel has a corresponding vertex
Additionally, a source (“object”) and a sink Additionally, a source (“object”) and a sink (“background”)(“background”)
Each pixel vertex has an edge to its Each pixel vertex has an edge to its neighbours (e.g. 8 adjacent neighbours in neighbours (e.g. 8 adjacent neighbours in 2D), an edge to the source, an edge to 2D), an edge to the source, an edge to the sinkthe sink
![Page 6: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/6.jpg)
Edge Weights between pixelsEdge Weights between pixels
Weight of edges between pixel vertices Weight of edges between pixel vertices are determined by the B() functionare determined by the B() function
Low score when boundary is likely to pass Low score when boundary is likely to pass between the verticesbetween the vertices
high score when vertices are probably part high score when vertices are probably part of the same elementof the same element
E.g. the difference in pixel intensities, the E.g. the difference in pixel intensities, the gradientgradient
![Page 7: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/7.jpg)
Edges to Source/SinkEdges to Source/Sink
If pixel is known to be an object, use a high If pixel is known to be an object, use a high weight (K) to the source, zero weight to the sinkweight (K) to the source, zero weight to the sink K is chosen so that it will never be cutK is chosen so that it will never be cut
Conversely, if pixel is background, use weight K Conversely, if pixel is background, use weight K to the sink, zero weight to the sourceto the sink, zero weight to the source
Otherwise, weigh edges to source and sink Otherwise, weigh edges to source and sink appropriately using the R() functionappropriately using the R() function Note that the edge to the source is the “likelihood” for Note that the edge to the source is the “likelihood” for
the pixel being the the pixel being the backgroundbackground – – we break this edge we break this edge when the pixel is assigned to the backgroundwhen the pixel is assigned to the background
![Page 8: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/8.jpg)
ApplicationsApplications
Handles arbitrary number of dimensionsHandles arbitrary number of dimensions
Finds global minimum energyFinds global minimum energy
Needs “good” user input to work effectivelyNeeds “good” user input to work effectively
Need intelligent functionsNeed intelligent functions
Need to select Need to select λλ
![Page 9: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/9.jpg)
Felzenszwalb and HuttenlocherFelzenszwalb and Huttenlocher
Download the program from the webpage:Download the program from the webpage: http://people.cs.uchicago.edu/~pff/segment/http://people.cs.uchicago.edu/~pff/segment/
Minimal documentationMinimal documentation Short README fileShort README file PaperPaper
![Page 10: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/10.jpg)
The programThe program
Comes as a tar.gz archive and .zip archiveComes as a tar.gz archive and .zip archive
ProcessProcess Extract archiveExtract archive ‘‘make’ (Makefile supplied)make’ (Makefile supplied)
Program consists ofProgram consists of A .cpp “wrapper” file (only calls the functions)A .cpp “wrapper” file (only calls the functions) Actual algorithm functions are in .h filesActual algorithm functions are in .h files Basic portable C++ codeBasic portable C++ code
![Page 11: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/11.jpg)
Program TestingProgram Testing
Built on Mac OS X, Linux, Windows cygwinBuilt on Mac OS X, Linux, Windows cygwin Gcc toolchain, but any C++ compiler should workGcc toolchain, but any C++ compiler should work Supplied basic MakefileSupplied basic Makefile
Results were basically the same between Results were basically the same between platformsplatforms Colors are chosen randomlyColors are chosen randomly
Results obtained are not the same as posted on Results obtained are not the same as posted on the websitethe website Image files on the website may be modified Image files on the website may be modified
(scaled/compressed/downsampled)(scaled/compressed/downsampled)
![Page 12: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/12.jpg)
AlgorithmAlgorithm
Create a graphCreate a graph Each vertex corresponds to a vertexEach vertex corresponds to a vertex
Edges are between “neighbouring” verticesEdges are between “neighbouring” vertices Choose a small neighbourhood to reduce Choose a small neighbourhood to reduce
computation time (otherwise we have a complete computation time (otherwise we have a complete graph)graph)
Weight on the edge is the 5D distance between the Weight on the edge is the 5D distance between the points (for a 2D image)points (for a 2D image)
5D vector = x position, y position and 3 color components5D vector = x position, y position and 3 color components
![Page 13: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/13.jpg)
ParametersParameters
σσ: Use this value and do Gaussian : Use this value and do Gaussian smoothing (preprocessing the image to smoothing (preprocessing the image to reduce noise)reduce noise)
k: threshold value for doing the clusteringk: threshold value for doing the clustering
min: “hack” parametermin: “hack” parameter the smallest cluster size must contain at least the smallest cluster size must contain at least
this many vertices – clusters that are too this many vertices – clusters that are too small will be merged with other clusters until small will be merged with other clusters until sufficiently largesufficiently large
![Page 14: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/14.jpg)
ClusteringClustering
Put each vertex in a component Put each vertex in a component
Sort edges by weightSort edges by weight
Take each edge in turnTake each edge in turn If the edge is between vertices in two different If the edge is between vertices in two different
components A and B, we can merge if the edge components A and B, we can merge if the edge weight is lower enough than the threshold weight is lower enough than the threshold
Threshold is the minimum of the following value, Threshold is the minimum of the following value, computed on A and Bcomputed on A and B
(Lowest weight edge in minimum spanning tree of the (Lowest weight edge in minimum spanning tree of the component) + (k / size of component)component) + (k / size of component)
![Page 15: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/15.jpg)
NotesNotes
Low Low edge weights between vertices that edge weights between vertices that are likely to be in the same clusterare likely to be in the same clusterAs a cluster gets larger, it becomes As a cluster gets larger, it becomes harder to add vertices to itharder to add vertices to itHeuristic – not really minimising a Heuristic – not really minimising a particular energy functionparticular energy function More similar to “region growing”More similar to “region growing”
User has select “good” parameters to get User has select “good” parameters to get good resultsgood results
![Page 16: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/16.jpg)
Effect of Effect of σσ
Increased smoothing results in removal of Increased smoothing results in removal of noisenoise
Can cause “bleeding” – the algorithm has Can cause “bleeding” – the algorithm has difficulty separating background from the difficulty separating background from the object if the boundaries are too smoothobject if the boundaries are too smooth
![Page 17: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/17.jpg)
Reference ImagesReference Images
![Page 18: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/18.jpg)
Increasing Increasing σσ
Clouds are Clouds are recognised as recognised as one objectone object
Palm tree Palm tree gets confused gets confused with oceanwith ocean
![Page 19: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/19.jpg)
GrainGrain
Increasing Increasing σσ introduces more blurring introduces more blurring (reduces the edge weight between pixels)(reduces the edge weight between pixels)
![Page 20: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/20.jpg)
Vertebrae MRIVertebrae MRI
Gets rid of noise (bottom left, right hand side), Gets rid of noise (bottom left, right hand side), but purple vertebrate piece bleeds outbut purple vertebrate piece bleeds out
![Page 21: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/21.jpg)
Increasing ThresholdIncreasing Threshold
Clusters Clusters more more aggresivelyaggresively
Palm tree is Palm tree is confused confused with ocean with ocean and cloudsand clouds
![Page 22: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/22.jpg)
GrainGrain
Non-grain pixels are almost all clustered Non-grain pixels are almost all clustered togethertogetherMeasure of how “similar” all the pixels of an Measure of how “similar” all the pixels of an object should beobject should be
![Page 23: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/23.jpg)
MRIMRI
Vertebrae and region next to vertebrae are Vertebrae and region next to vertebrae are very similar shade so are easily confusedvery similar shade so are easily confused
![Page 24: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/24.jpg)
Increasing ‘min’ valueIncreasing ‘min’ value
Segmentation Segmentation is the same, is the same, but small but small components components are merged are merged with with neighboring neighboring onesones
![Page 25: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/25.jpg)
GrainGrain
Easy to control change, gets rid of small Easy to control change, gets rid of small artifactsartifacts
![Page 26: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/26.jpg)
MRIMRI
Not Not much much effect if effect if regions regions are are already already largelarge
![Page 27: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/27.jpg)
Parameter TweakingParameter Tweaking
Need to manually Need to manually tune parameters to tune parameters to get a good imageget a good image
Image of MRI after Image of MRI after selectingselecting σσ = 0.6 = 0.6 k = 200k = 200 min = 60min = 60
![Page 28: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/28.jpg)
PerformancePerformance
Need to tune parameters by handNeed to tune parameters by handVery fastVery fast Program usually takes a couple seconds on Program usually takes a couple seconds on
the test images providedthe test images provided
Only takes ppm images in RAW data Only takes ppm images in RAW data formatformatTheoretically, algorithm generalises to Theoretically, algorithm generalises to arbitrary number of dimensions and arbitrary number of dimensions and arbitrary number of features per pixelarbitrary number of features per pixel
![Page 29: Graph-based Segmentation. Main Ideas Convert image into a graph Vertices for the pixels Vertices for the pixels Edges between the pixels Edges between.](https://reader030.fdocuments.us/reader030/viewer/2022032611/56649e795503460f94b7975f/html5/thumbnails/29.jpg)
Command-line toolsCommand-line toolsWorking with images involves opening up the results Working with images involves opening up the results in an image viewer – can get messyin an image viewer – can get messy