A Database of Human Segmented Natural Images and Two Applications David Martin, Charless Fowlkes,...
-
date post
20-Dec-2015 -
Category
Documents
-
view
217 -
download
2
Transcript of A Database of Human Segmented Natural Images and Two Applications David Martin, Charless Fowlkes,...
A Database of Human Segmented Natural Images
and Two Applications
David Martin, Charless Fowlkes, Doron Tal, Jitendra Malik
UC Berkeley{dmartin,fowlkes,doron,malik}@eecs.berkeley.edu
David Martin - UC Berkeley - ICCV 2001 2
Motivation
• Berkeley Segmentation Dataset Groundtruth for image segmentation of natural images
• App#1: A segmentation benchmark• App#2: Ecological statistics
David Martin - UC Berkeley - ICCV 2001 3
Benchmark Example for Recognition
MNIST handwritten digit dataset [LeCun, AT&T]http://www.research.att.com/~yann/exdb/mnist/index.html
METHOD ERROR (%)Boosted LeNet-4, [distortions] 0.7Virtual SVM deg 9 poly [distortions] 0.8LeNet-5, [distortions] 0.8LeNet-5, [huge distortions] 0.85LeNet-5, [no distortions] 0.95Reduced Set SVM deg 5 polynomial 1K-NN, Tangent Distance, 16x16 1.1SVM deg 4 polynomial 1.1LeNet-4 1.1LeNet-4 with K-NN instead of last layer 1.1LeNet-4 with local learning instead of ll 1.12-layer NN, 300 HU, [deskewing] 1.6LeNet-1 [with 16x16 input] 1.7K-nearest-neighbors, Euclidean, deskewed 2.4
Training set, test set, evaluation methodology, algorithm ranking
David Martin - UC Berkeley - ICCV 2001 4
The Image Dataset
• 1000 Corel images– Photographs of outdoor scenes– Texture is common– Large variety of subject matter– 481 x 321 x 24b
David Martin - UC Berkeley - ICCV 2001 5
David Martin - UC Berkeley - ICCV 2001 6
David Martin - UC Berkeley - ICCV 2001 7
David Martin - UC Berkeley - ICCV 2001 8
David Martin - UC Berkeley - ICCV 2001 9
Establishing Groundtruth• Def: Segmentation
= Partition of image pixels into exclusive sets
• Manual segmentation by human subjects– Custom Java tool to facilitate task
• Currently: 1000 images, 5500 segmentations, 20 subjects
• Naïve subjects (UCB undergrads) given simple, non-technical instructions
David Martin - UC Berkeley - ICCV 2001 10
Directions to Image Segmentors
• You will be presented a photographic image• Divide the image into some number of
segments, where the segments represent “things” or “parts of things” in the scene
• The number of segments is up to you, as it depends on the image. Something between 2 and 30 is likely to be appropriate.
• It is important that all of the segments have approximately equal importance.
David Martin - UC Berkeley - ICCV 2001 11
David Martin - UC Berkeley - ICCV 2001 12
David Martin - UC Berkeley - ICCV 2001 13
• The segmentations are not identical.
• But are they consistent??
David Martin - UC Berkeley - ICCV 2001 14
Perceptual organization
forms a hierarchyimage
background left bird right bird
grass bush
headeye
beakfar
body headeye
beak
body
Each subject picks a slice through this hierarchy.
David Martin - UC Berkeley - ICCV 2001 15
Quantifying inconsistency
S1 S2
How much is S1 a refinement of S2 at pixel ?
),(
),(\),(),(
1
2121
i
ii
pSR
pSRpSRSSLRE
David Martin - UC Berkeley - ICCV 2001 16
Segmentation Error Measure
• One-way Local Refinement Error:
i
ii pSSLREpSSLREn
SSSE ),,(),,,(min1
),( 122121
• Segmentation Error allows refinement in either direction at each pixel:
),(
),(\),(),(
1
2121
i
ii
pSR
pSRpSRSSLRE
David Martin - UC Berkeley - ICCV 2001 17
Human segmentations are consistent
SE (Color Human Segmentations)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Segmentation Error (SE)
Same Image
Different Images
Distribution of segmentation error over the dataset.
David Martin - UC Berkeley - ICCV 2001 18
Color Gray InvNeg
David Martin - UC Berkeley - ICCV 2001 19
InvNeg
David Martin - UC Berkeley - ICCV 2001 20
Color Gray InvNeg
David Martin - UC Berkeley - ICCV 2001 21
Gray vs. Color vs. InvNeg Segmentations
SE (gray, gray) = 0.047SE (gray, color) = 0.047
Color may affect attention, but doesn’t seem to affect perceptual organization
SE (gray, gray) = 0.047SE (gray, invneg) = 0.059
InvNeg interferes with high-level cues
(2500 gray, 2500 color,200 invneg segmentations)
David Martin - UC Berkeley - ICCV 2001 22
Benchmark Methodology
• Separate training and test datasets with no images in common
• Generate computer segmentation(s) of each image in test set– Determine error of each computer
segmentation using SE measure– Algorithm scored by mean SE
• Example: – SE (human, human) = 0.05– SE (NCuts, human) = 0.22– SE (different images) = 0.30
David Martin - UC Berkeley - ICCV 2001 23
Ecological Statistics of Image Segmentations
• Validating and quantifying Gestalt grouping factors [Brunswik 1953]
• Priors on region properties
• Recent work on natural image statistics:– Filter outputs [Ruderman 1994, Olshausen & Field 1996,
Yuille et. al. 1999]– Object sizes [Alvarez, Gousseau, Morel 1999]– Shape [Zhu 1999] – Contours [August & Zucker 2000, Geisler et al. 2001]
David Martin - UC Berkeley - ICCV 2001 24
Relative power of cues
• Pairwise grouping cues– Proximity– Luminance similarity– Color similarity– Intervening contour– Texture similarity
David Martin - UC Berkeley - ICCV 2001 25
P (Same Segment | Proximity)
David Martin - UC Berkeley - ICCV 2001 26
P (Same Segment | Luminance)
David Martin - UC Berkeley - ICCV 2001 27
Bayes Risk for Proximity Cue
David Martin - UC Berkeley - ICCV 2001 28
Bayes Risk for Various Cues Conditioned on Proximity
David Martin - UC Berkeley - ICCV 2001 29
Mutual Information for Various Cues Conditioned on Proximity
David Martin - UC Berkeley - ICCV 2001 30
Priors on Region Properties
• Area• Convexity
David Martin - UC Berkeley - ICCV 2001 31
Empirical Distribution of Region Area
y = Kx-
= 0.913
Compare with Alvarez, Gousseau, Morel 1999.
David Martin - UC Berkeley - ICCV 2001 32
Empirical Distribution of Region Convexity
David Martin - UC Berkeley - ICCV 2001 33
Conclusion
• Large new database of segmentations of natural images by humans
• A segmentation benchmark• Ecological statistics
– Relative power of grouping cues– Priors on region properties
http://www.cs.berkeley.edu/~dmartin/segbench