Texture-Based Segmentation - Inria...Motivation Method Results Future Work Texture-Based...
Transcript of Texture-Based Segmentation - Inria...Motivation Method Results Future Work Texture-Based...
MotivationMethodResults
Future Work
Texture-Based SegmentationInternship at the project team PULSAR
A. Schnaars1
1University of MagdeburgSupervisor: Guillaume Charpiat PhD
Pulsar - INRIA Sophia-Antipolis
INRIA Sophia-Antipolis, 15th march 2010
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Outline
1 Motivation
2 MethodCreating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
3 Results
4 Future Work
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Definition
No unique definition of what a texture is:
Human Vision System[3]Information of texture to distinguish between things whereedge of objects in the enviroment are not defined by clearboundariesLarge number of symbols or simple shapes -> individualobjectsV1 neurons in the primary visual cortex: Texture elementorientation, size, contrast and color
!Consisting of repetition or quasi repetition of some fundamentalimage elements
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
State of the Art & Goal
There is no “optimal” texture featureStructrual, statistical, model-based approachSIFT and SURF most widly usedSparsly application in REAL-time processing like video-analysis
GoalCombination of several informations (scale, boundaries, ...)Fast classificationRobust classifierBasis for object segmentation
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Outline
1 Motivation
2 MethodCreating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
3 Results
4 Future Work
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Ground Truth
Problem: Mostly segmentation of objects but not of singletexture regions and no annotations informationDatabase: Texture patterns from free texture libraries !17di!erent texture classes, each pattern in 3 di!erent resolutions
Brick ...
Fabric ...Grounds ...
Hair ...
Plants ...Skin ...
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Texture Descriptor
Requirements:Fast processingScale invarianceIntroduction of color informationIllumination invariance
Integrative method: Combination of color and texture informationCIE L*a*b* color spaceUnser Sum- and Di!erence-Histogram Features
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Unser Sum- and Di!erence-Histogram FeaturesTexture Descriptor
Approximation for the two-dimensional Haralick texturefeatures on the co-occurrence matrixFrequencies of sums, respectively di!erences of pixel colorlevels with a certain displacement (dx , dy ) within a region ofinterest D
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Unser Sum- and Di!erence-Histogram FeaturesTexture Descriptor
hcs (i) =Card {(x1, y1) ! Dc |gx1,y1 +gx2,y2 | = i} , i ! [0; 2(G c "1)] ,
hcd (j) =Card {(x1, y1) ! Dc |gx1,y1 "gx2,y2 | = j} ,
j ! ["G c +1; G c "1] , c ! L#a #b#h3s/d (i) ={hL
s/d (i), has/d (i), hb
s/d (i)}
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Unser Sum- and Di!erence-Histogram FeaturesTexture Descriptor
Features
Sum/ di!erence mean: fcj = 1N !i i hc
s/d (i)
Sum/ di!erence contrast: fcj = 1N !i (i "µ)2 hc
s/d (i)
Sum/ di!erence angular second momentum: fi = !i
!h3s/d (i)
"2
Sum/ di!erence entropy: fj = !i "h3s/d (i)log(h3
s/d (i))
Plus color values of each color channel
Regions of interest: 5x5, 7x7, 9x98 directions x displacement 1, 2, 4
$1000 Features
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Feature Selection and Reduction
Normalisation via a sigmodial function to keep outliers
y % =y "µstd
y %% =1" e"y %
1+ e"y %
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
PCAFeature Selection and Reduction
Assumption: Correlated featuresPCA: Uncorrelating data by mapping the data into a mostsignificant subspaceGoal: Feature selection through analysis of the most importantprinciple components!Features with same loadings have the same character
Feature selection (as defined in [2])kMeans on the reduced principle componentsChoosing feature as representants whose loadings have thesmallest distance to a cluster’s center
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Interim ResultsClustering of the Reduced Texture Descriptor
Problems at:BoundariesCoarse texturesSubtle color shifts (small value di!erences)
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Textures Classes Training
Assumption: Each class could be described by a mixture ofgaussiansProblem: Big amount of data!Online learning of the parameters
PTexture(x) =!k
p(x |µk ,!k)"k
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Outline
1 Motivation
2 MethodCreating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
3 Results
4 Future Work
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Ground Truth
Berkeley Segmentation Dataset [1]: Hand-labeled segmentationsfrom 30 human subjects, public benchmark contains 300 imagesand its segmentations
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Edge Descriptor
For each channel of the CIE L*a*b* color space:Sobel Operator
First and second derivativesBlocksizes 1, 2, 3, 5, 7Magnitude and orientation
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Edge Descriptor
Autocorrelation Matrix
A(x) =!x ,y
w(x ,y)
#I 2x (x ,y) Ix Iy (x ,y)
Ix Iy (x ,y) I 2y (x ,y)
$
Block sizes of 3, 9, 23The biggest Eigenvalue
Plus color values$70 Features
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Feature Selection and Reduction
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Edge Classifier Training
Assumption: Edges can be very noisyClassifier that seperates boundaries clearly! SVMIssue: Big amount of data!Online learning of the parameters : Neural Network
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Interim Results
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Interim Results
Results are moderate compared to the e!ortEventually overfit
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Creating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
Outline
1 Motivation
2 MethodCreating a Texture ClassifierCreating an Edge ClassifierGlobal Optimization
3 Results
4 Future Work
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Combination of all learned informations in a (minimised) costfunction:
Graph Cut
E (p) = !p!I
L(p)+# !p,q!I
V (L(p),L(q))
L(p) =" log(PTexture(p))
V (L(p),L(q)) ="log(min(PEdge(q),PEdge(p)))
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Extension
Measurement of the interactions between di!erent textureclassesLocation of some texture
Frequency of certain neighboring textures in certain directions
Problems:Annotation of the data, e.g. Berkeley tool only segmentsOptimisation tool (GraphCut): No di!erent interactionsbetween two classes in di!erent directions
A.Schnaars Texture-Based Segmentation
MotivationMethodResults
Future Work
Further Applications
Object recognitionLearning shape parameters of certain textures in certain objectsLearning locations of objectsProbability to find an object in an area based on its locationand texture components
A.Schnaars Texture-Based Segmentation
Appendix
Thank you for your attention!
Questions?
A.Schnaars Texture-Based Segmentation
Appendix For Further Reading
For Further Reading I
Pablo Arbelaez, Charless Fowlkes, and David Martin.The berkeley segmentation dataset and benchmark, 2007.
Yijuan Lu, Ira Cohen, Xiang Sean Zhou, and Qi Tian.Feature selection using principal feature analysis.In MULTIMEDIA ’07: Proceedings of the 15th internationalconference on Multimedia, pages 301–304, New York, NY,USA, 2007. ACM.
Colin Ware.Visual Thinking for Design.Morgan Kaufmann, July 2008.
A.Schnaars Texture-Based Segmentation