SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1....
Transcript of SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1....
![Page 1: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/1.jpg)
SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL)
FOR HUMAN ACTION RECOGNITION
1J.H. Ma, 1P.C. Yuen, 1W.W. Zou, 2J.H. Lai1Hong Kong Baptist University2Sun Yat-sen University
ICCV workshop on Machine Learning for Vision-based Motion Analysis (MLVMA09)
1
![Page 2: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/2.jpg)
OUTLINE
Introduction ProblemsSupervised Neighborhood Topology Learning (SNTL)Experiments and AnalysisConclusion
2
![Page 3: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/3.jpg)
INTRODUCTION
3
![Page 4: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/4.jpg)
INTRODUCTIONApplications of Human Action Recognition
intelligent video surveillanceperceptual interface content-based video retrievaletc
Existing Methods for Human Action RecognitionFlow based [AA Efros et al., ICCV 2003]Template based [L. Gorelick et al., PAMI 2007]Interest points based [J. Niebles et al., IJCV 2008] Trajectory based [R. Messing et al., ICCV, 2009] Manifold learning-based [Wang et al., TIP, 2007]Etc
Manifold learning-based methods (LPP and SLPP)achieved great success
4
![Page 5: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/5.jpg)
LPP AND SLPP
5
3
1
2
wij
Framework of LPP and SLPP1. Constructing the adjacency graph
ε-neighborhood, KNN, or supervised2. Choosing the weights
Simple-minded or heat kernel3. Eigenmaps
Solve the optimization problem projection matrix
![Page 6: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/6.jpg)
PROBLEMS
6
![Page 7: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/7.jpg)
CHARACTERISTIC OF HUMAN ACTIONS
similar pose in two different actions
dissimilar pose in the same action
- = two actions
one action
7
![Page 8: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/8.jpg)
PROBLEMS IN LPP AND SLPP
adjacency graphs play an important role in LPPand SLPP
in adjacency graph construction:LPP only considers the local informationSLPP only considers the class information
LPP SLPP
8
![Page 9: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/9.jpg)
PROPOSED METHOD
9
![Page 10: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/10.jpg)
Reasonable adjacency graphLPP SLPP
10
![Page 11: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/11.jpg)
THE TOPOLOGY IN LPP AND SLPPthe adjacency graph corresponding to a topologyData points with different topologies are considered to be different manifolds.denote: LPP τ1 , and SLPP τ2.
τ1 : topology induced by Euclidean, consider Euclidean distance data close together are in the same neighborhoodτ2 : topology induced by label information, consider class distance data point from same action are in the same neighborhood
integrate these two together? restrict the τ1 onτ2 or τ2 on τ1
11
},,,{ 21 SSS∅two class problem τ2 =
![Page 12: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/12.jpg)
A NEW TOPOLOGYproposed method: define a new topology τ3, so that make use of class label information and the local informationa subset is open in τ3 iff it is the intersection of action data set Si with an open set in the Euclidean space.
LPP SLPP
SNTLτ1
link node
τ2
link node
τ3
link node
SNTL
![Page 13: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/13.jpg)
ADVANTAGE OF NEW TOPOLOGYKeep the local information
temporal informationmore suitable for manifold learning
Keep the label informationrecognition perspectivepreserve the discriminative features
make use of our new topology, and adopt the framework of LPP, we propose Supervised Neighborhood Topology Learning (SNTL)
SNTL
τ3
link node
13
![Page 14: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/14.jpg)
ALGORITHMIC PROCEDURE
Proposed Neighborhood Topology Manifold Learning:
1. Computing KNN parameter ki for class i. Choosing a percentage parameter a, 0<a<1, let ki=aNi, where Ni denotes the sample number of class i.
2. Putting edges on the graph. An edge will be put between nodes t and s, if xt and xs belong to the same class i, and xt is among the kinearest neighbors of xs or xs is among the ki nearest neighbors of xt.
3. Eigenmaps. Optimize the cost function by EVD.
14
![Page 15: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/15.jpg)
EXPERIMENTS
15
![Page 16: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/16.jpg)
EXPERIMENT SETTING
Database person num action num training testingWeizmann 9 10 8 persons 1 person
leave-one-out ruleexample frames of different actions
Classifier: nearest neighbor framework with median Hausdorff distance
( , ) ( , ) ( , ),
( , ) median(min( ( ) ( ) ))
i j i j j i
i j i jlk
d A A S A A S A A
S A A A k A l
= +
= −
b e n d j a c k p j u m pj u m p ru n
s i d e w a l ks k i p w a v e1 w a v e2
( )
16
![Page 17: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/17.jpg)
COMPARATIVE EXPERIMENTS 1 2D embeddings of training data bySNTL, LPP and SLPP
2D embeddings of testing data by SNTL, and SLPP
SNTL LPP SLPP
SNTL SLPP
17
![Page 18: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/18.jpg)
COMPARATIVE EXPERIMENTS 2
Recognition ratemethod LPP SLPP SNTL
Recognition rate (%) 92.22 84.84 95.56
Optimal dimension 70 108 42
18
![Page 19: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/19.jpg)
CONCLUSION
19
![Page 20: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/20.jpg)
CONCLUSIONSPropose a new supervised manifold learning method,
namely supervised neighborhood topology learning (SNTL)for recognition perspective
Advantages preserves discriminative featurespreserves temporal information of each action contained in local structure
Disadvantages Do not take full advantage of temporal informationParameter is empirically determined 20
![Page 21: SUPERVISED NEIGHBOURHOOD TOPOLOGY LEARNING (SNTL) …andyjhma/publications/iccv workshop.pdf · 1. Computing KNN parameter k i for class i. Choosing a percentage parameter a, 0](https://reader033.fdocuments.us/reader033/viewer/2022051919/600b8524fed5ac03b31de3ef/html5/thumbnails/21.jpg)
Q & A ?21