Locality-constrained Linear Coding for Image Classification
Presenter : Han-Mu Park
2
Introduction Coding methods Proposed method Experimental results Conclusion References
Contents
Locality-constrained Linear Coding for Image Classification, CVPR 2010
3
Bag-of-Words (BoW) model– An image is represented as a collection of visual words.– Generally, to represent the collection, histogram of
words form is used.
Introduction
Locality-constrained Linear Coding for Image Classification, CVPR 2010
4
General Spatial Pyramid Matching frame-works– Feature extraction
• SIFT• HOG• etc
– Coding• Vector Quantization• Sparse coding• etc
– Pooling• Max pooling• Sum pooling
Introduction
Locality-constrained Linear Coding for Image Classification, CVPR 2010
Spatial Pyramid Match-ing framework [J.Wang2010]
5
Spatial Pyramid Match-ing framework [J.Wang2010]
General Spatial Pyramid Matching frame-works
Introduction
Locality-constrained Linear Coding for Image Classification, CVPR 2010
6
Coding methods– Vector quantization (VQ)– Sparce coding (SC)– Locality-constrained Linear Coding (LLC)
Coding methods
Locality-constrained Linear Coding for Image Classification, CVPR 2010
[J.Wang2010]
7
Vector quantization (VQ)– Hard quantization method
– A set of -dimensional local descriptors •
– Codebook with entries
– Objective function
• Where is the set of codes for X
Coding methods
Locality-constrained Linear Coding for Image Classification, CVPR 2010
8
Sparse coding (SC)– Soft quantization method– Relaxed the cardinality constraint
– Objective function
– The roles of sparsity regularization term• Because the codebook is usually over-com-
plete , it is necessary to ensure that the under-determined system has a unique solution.
• Sparsity allows the learned representation to capture salient patterns of local descriptors.
• The sparse coding can achieve much less quantization error than VQ.
Coding methods
Locality-constrained Linear Coding for Image Classification, CVPR 2010
9
Locality-constrained Linear Coding (LLC)– Replaced the sparsity regularization term
with new constraint.
– Objective function
• : the element-wise multiplication
Where
Proposed method
Locality-constrained Linear Coding for Image Classification, CVPR 2010
10
Properties of LLC1. Better reconstruction
• Because LLC represents each descriptor by using multiple weighted bases (codewords), it has less reconstruction error than VQ.
2. Local smooth sparsity• Because the regularization term of in SC is not smooth, therefore,
SC loses correlations between codes.
3. Analytical solution• The solution of LLC can be derived analytically by
Where
Proposed method
Locality-constrained Linear Coding for Image Classification, CVPR 2010
11
Approximated LLC for fast encoding– The LLC selects the local bases for each descriptor to
form a local coordinate system.– To speedup the encoding process, authors used nearest
neighbors of as the local bases , and solve a much smaller linear system to get the codes
– The reduced computation complexity• , where
Proposed method
Locality-constrained Linear Coding for Image Classification, CVPR 2010
12
Codebook optimization– To improve the accuracy, authors trained the codebook
to optimize for LLC codes.– The optimal codebook can be obtained by
– This can be solved by using Coordinate Descent method.– However, because the number of training descriptors is
usually very large, the huge memory space is needed to solve that problem.
Proposed method
Locality-constrained Linear Coding for Image Classification, CVPR 2010
13
Incremental codebook optimiza-tion
– First, initialize by using K-means clustering.
– Then loop through all the training descriptors to update incrementally.
– In each iteration, we take in a single (or a small set of) examples , and solve original objective function to obtain the corresponding LLC codes.
Proposed method
Locality-constrained Linear Coding for Image Classification, CVPR 2010
[J.Wang2010]
14
Incremental codebook optimiza-tion
– Then select bases whose corre-sponding weights are larger than predefined threshold, and refit without the locality constraint.
– The obtained code is used to update the basis in a gradient descent fash-ion.
Proposed method
Locality-constrained Linear Coding for Image Classification, CVPR 2010
[J.Wang2010]
15
Performance of codebook
Experimental results
Locality-constrained Linear Coding for Image Classification, CVPR 2010
16
Performance under different neighbors
Experimental results
Locality-constrained Linear Coding for Image Classification, CVPR 2010
17
Results using Pascal VOC 2007
Experimental results
Locality-constrained Linear Coding for Image Classification, CVPR 2010
18
Contribution– In this paper, the Locality-constrained Linear Coding
(LCC) method is proposed• Better reconstruction• Local smooth sparsity• Analytical solution
– For speedup, K-nearest neighbors algorithm is used.– To optimize the accuracy, incremental codebook learning
is proposed for LCC.
Conclusion
Locality-constrained Linear Coding for Image Classification, CVPR 2010
19
[1] J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, Y. Gong, “Locality-constrained Linear Coding for Image Classification,” CVPR 2010.
References
Locality-constrained Linear Coding for Image Classification, CVPR 2010
20
Thank you!
Top Related