Sparse Dictionary-based Representation and Recognition of...
Transcript of Sparse Dictionary-based Representation and Recognition of...
![Page 1: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/1.jpg)
Sparse Dictionary-based
Representation and Recognition
of Action Attributes
Qiang Qiu, Zhuolin Jiang, Rama Chellappa
Center for Automation Research,
Institute for Advanced Computer Studies
University of Maryland, College Park
[email protected], {zhuolin, rama}@umiacs.umd.edu
![Page 2: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/2.jpg)
Action Feature Representation
2
Shape
Motion
HOG
![Page 3: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/3.jpg)
Action Sparse Representation
3
=
0 0 0.64 0.53 -0.40 0.35 0 0 0
0.43 0.63 0 0 -0.33 0 -0.36 0 0
= 0.43 × + 0.63 × - 0.33 × - 0.36 ×
= 0.64× + 0.53× - 0.40 × +0.35 ×
Action Dictionary
Sparse code
![Page 4: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/4.jpg)
K-SVD
4
=
Y
y2 d1 d2 d3 …
0 0 0.64 0.53 -0.40 0.35 0 0 0
0.43 0.63 0 0 -0.33 0 -0.36 0 0
x1 x2
y1
D
X
K-SVD [1]
Input: signals Y, dictionary size, sparisty T
Output: dictionary D, sparse codes X
arg min |Y- DX|2 s.t. i , |xi|0 ≤ T D,X
Input signals Dictionary
Sparse codes
[1] M. Aharon and M. Elad and A. Bruckstein, K-SVD: An Algorithm for Designing
Overcomplete Dictionries for Sparse Representation, IEEE Trans. on Signal Process, 2006
![Page 5: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/5.jpg)
5
Compact Discriminative and
Dictionary.
Learn a
Objective
![Page 6: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/6.jpg)
Probabilistic Model for Sparse
Representation
A Gaussian Process
Dictionary Class Distribution
6
![Page 7: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/7.jpg)
7
= 0.43 0.63 0 0 -0.33 0 -0.36 0 0 0 0 0 0
0 0 0.64 0.53 -0.40 0.35 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 -0.28 0.698 0.37 0.25 0
0 0 0 0 -0.42 0 0 0 0 0.42 0.47 0 0.32
y2 y1 y4 y3
l1 l1 l2 l2
d1 d2 d3 …
x1 x2 x3 x4
xd1
l1 l1 l2 l2
More Views of Sparse Representation
![Page 8: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/8.jpg)
8
y2 y1 y4 y3
l1 l1 l2 l2
xd1 0.43 0 0 0
x1 x2 x3 x4
0.63 0 0 0
0 0.64 0 0
0 0.53 0 0
-0.33 -0.40 0 -0.42
… …
xd2
xd3
xd4
xd5
l1 l1 l2 l2
d1
d2
d3
d4
d5
A Gaussian Process
• Covariance function entry: K(i,j) = cov(xdi, xdj)
• P(Xd*|XD*) is a Gaussian with a closed-form conditional variance
A Gaussian Process
![Page 9: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/9.jpg)
9
y2 y1 y4 y3
l1 l1 l2 l2
xd1 0.43 0 0 0
x1 x2 x3 x4
0.63 0 0 0
0 0.64 0 0
0 0.53 0 0
-0.33 -0.40 0 -0.42
… …
xd2
xd3
xd4
xd5
l1 l1 l2 l2
d1
d2
d3
d4
d5
Dictionary Class Distribution
• P(L|di), L [1, M]
• aggregate |xdi| based on class labels to obtain a M sized vector
• P(L=l1|d5) = (0.33+0.40)/(0.33+0.40+0.42) = 0.6348
• P(L=l2|d5) = (0+0.42)/(0.33+0.40+0.42) = 0.37
Dictionary Class Distribution
![Page 10: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/10.jpg)
Dictionary Learning Approaches
Maximization of Joint Entropy (ME)
Maximization of Mutual Information (MMI)
Unsupervised Learning (MMI-1)
Supervised Learning (MMI-2)
10
![Page 11: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/11.jpg)
11
Maximization of Joint Entropy (ME)
- Initialize dictionary using k-SVD
Do =
- Start with D* = - Untill |D*|=k, iteratively choose d* from Do\D*,
d* = arg max H(d|D*)
Where
-A good approximation to ME criteria
arg max H(D)
d
D
ME dictionary
![Page 12: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/12.jpg)
12
Maximization of Mutual Information
for Unsupervised Learning (MMI-1)
- Initialize dictionary using k-SVD
Do =
- Start with D* = - Untill |D*|=k, iteratively choose d* from Do\D*,
d* = arg max H(d|D*) - H(d|Do\(D* d))
-A near-optimal approximation to MMI criteria
arg max I(D; Do\D)
Within (1-1/e) of the optimum
d
D
MMI dictionary
![Page 13: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/13.jpg)
13
Dictionary Class Distribution
• P(L|di), L [1, M]
• aggregate|xdi|based on class labels to obtain a M sized vector
• P(l1|d5) = (0.33+0.40)/(0.33+0.40+0.42) = 0.6348
• P(l2|d5) = (0+0.42)/(0.33+0.40+0.42) = 0.37
• P(Ld) = P(L|d) • P(LD) = P(L|D) , where
y2 y1 y4 y3
l1 l1 l2 l2
xd1 0.43 0 0 0
x1 x2 x3 x4
0.63 0 0 0
0 0.64 0 0
0 0.53 0 0
-0.33 -0.40 0 -0.42
… …
xd2
xd3
xd4
xd5
l1 l1 l2 l2
d1
d2
d3
d4
d5
Revisit
![Page 14: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/14.jpg)
14
Maximization of Mutual Information
for Supervised Learning (MMI-2)
- Initialize dictionary using k-SVD
Do =
- Start with D* = - Untill |D*|=k, iteratively choose d* from Do\D*,
d d* = arg max [H(d|D*) - H(d|Do\(D* d))]
+ λ[H(Ld|LD*) – H(Ld|LDo\(D* d))]
- MMI-1 is a special case of MMI-2 with λ=0.
![Page 15: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/15.jpg)
Other learning methods
K-means
Liu-shah [1]
15
where
[1] J. Liu and M. Shah, Learning Human Actions via Information Maximization, CVPR 2008
![Page 16: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/16.jpg)
Purity and Compactness
16
![Page 17: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/17.jpg)
Representation Consistency
17
![Page 18: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/18.jpg)
Keck gesture dataset
18
![Page 19: Sparse Dictionary-based Representation and Recognition of ...zhuolin/Publications/ICCV2011-MMI-slide.pdfSparse Dictionary-based Representation and Recognition of Action Attributes](https://reader033.fdocuments.us/reader033/viewer/2022051608/603e444a32b02159a875a324/html5/thumbnails/19.jpg)
Recognition Accuracy
19
The recognition accuracy using initial dictionary Do: (a) 0.23 (b) 0.42 (c) 0.71