Object Recognizing. Recognition -- topics Features Classifiers Example ‘winning’ system.
Review: Intro to recognition Recognition tasks Machine learning approach: training, testing,...
-
Upload
donald-mathews -
Category
Documents
-
view
217 -
download
0
Transcript of Review: Intro to recognition Recognition tasks Machine learning approach: training, testing,...
Review: Intro to recognition
• Recognition tasks• Machine learning approach: training, testing,
generalization• Example classifiers
• Nearest neighbor• Linear classifiers
Image features
• Spatial support:
Pixel or local patch Segmentation region
Bounding box Whole image
Image features
• We will focus mainly on global image features for whole-image classification tasks• GIST descriptors• Bags of features• Spatial pyramids
GIST descriptors• Oliva & Torralba (2001)
http://people.csail.mit.edu/torralba/code/spatialenvelope/
Bags of features
Origin 1: Texture recognition
• Texture is characterized by the repetition of basic elements or textons
• For stochastic textures, it is the identity of the textons, not their spatial arrangement, that matters
Julesz, 1981; Cula & Dana, 2001; Leung & Malik 2001; Mori, Belongie & Malik, 2001; Schmid 2001; Varma & Zisserman, 2002, 2003; Lazebnik, Schmid & Ponce, 2003
Origin 1: Texture recognition
Universal texton dictionary
histogram
Julesz, 1981; Cula & Dana, 2001; Leung & Malik 2001; Mori, Belongie & Malik, 2001; Schmid 2001; Varma & Zisserman, 2002, 2003; Lazebnik, Schmid & Ponce, 2003
Origin 2: Bag-of-words models
• Orderless document representation: frequencies of words from a dictionary Salton & McGill (1983)
Origin 2: Bag-of-words models
US Presidential Speeches Tag Cloudhttp://chir.ag/projects/preztags/
• Orderless document representation: frequencies of words from a dictionary Salton & McGill (1983)
Origin 2: Bag-of-words models
US Presidential Speeches Tag Cloudhttp://chir.ag/projects/preztags/
• Orderless document representation: frequencies of words from a dictionary Salton & McGill (1983)
Origin 2: Bag-of-words models
US Presidential Speeches Tag Cloudhttp://chir.ag/projects/preztags/
• Orderless document representation: frequencies of words from a dictionary Salton & McGill (1983)
1. Extract local features
2. Learn “visual vocabulary”
3. Quantize local features using visual vocabulary
4. Represent images by frequencies of “visual words”
Bag-of-features steps
1. Local feature extraction
• Regular grid or interest regions
Normalize patch
Detect patches
Compute descriptor
Slide credit: Josef Sivic
1. Local feature extraction
…
1. Local feature extraction
Slide credit: Josef Sivic
2. Learning the visual vocabulary
…
Slide credit: Josef Sivic
2. Learning the visual vocabulary
Clustering
…
Slide credit: Josef Sivic
2. Learning the visual vocabulary
Clustering
…
Slide credit: Josef Sivic
Visual vocabulary
Review: K-means clustering• Want to minimize sum of squared Euclidean
distances between features xi and their nearest cluster centers mk
Algorithm:• Randomly initialize K cluster centers• Iterate until convergence:
• Assign each feature to the nearest center• Recompute each cluster center as the mean of all features
assigned to it
k
ki
kiMXDcluster
clusterinpoint
2)(),( mx
Example codebook
…
Source: B. Leibe
Appearance codebook
Another codebook
Appearance codebook…
Source: B. Leibe
1. Extract local features
2. Learn “visual vocabulary”
3. Quantize local features using visual vocabulary
4. Represent images by frequencies of “visual words”
Bag-of-features steps
Visual vocabularies: Details
• How to choose vocabulary size?• Too small: visual words not representative of all patches• Too large: quantization artifacts, overfitting• Right size is application-dependent
• Improving efficiency of quantization• Vocabulary trees (Nister and Stewenius, 2005)
• Improving vocabulary quality• Discriminative/supervised training of codebooks• Sparse coding, non-exclusive assignment to codewords
• More discriminative bag-of-words representations• Fisher Vectors (Perronnin et al., 2007), VLAD (Jegou et al., 2010)
• Incorporating spatial information
Bags of features for action recognition
Juan Carlos Niebles, Hongcheng Wang and Li Fei-Fei, Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words, IJCV 2008.
Space-time interest points
Bags of features for action recognition
Juan Carlos Niebles, Hongcheng Wang and Li Fei-Fei, Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words, IJCV 2008.
Spatial pyramids
level 0
Lazebnik, Schmid & Ponce (CVPR 2006)
Spatial pyramids
level 0 level 1
Lazebnik, Schmid & Ponce (CVPR 2006)
Spatial pyramids
level 0 level 1 level 2
Lazebnik, Schmid & Ponce (CVPR 2006)
Results: Scene category dataset
Multi-class classification results(100 training images per class)
Results: Caltech101 dataset
Multi-class classification results (30 training images per class)