Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang
description
Transcript of Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang
![Page 1: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/1.jpg)
1Intelligent Database Systems Lab
國立雲林科技大學National Yunlin University of Science and Technology
Recognizing Partially Occluded, Expression Variant Faces from Single Training Image per
Person with SOM and soft kNN Ensemble
Advisor : Dr. HsuPresenter : Jia-Hao Yang
Author :X Tan, S Chen, ZH Zhou, F Zhang
![Page 2: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/2.jpg)
2Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Motivation Object Architecture Introduction The propose method Experiments Conclusion Opinion
Guide
![Page 3: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/3.jpg)
3Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Motivation In many real-world applications only one
training image per person is available. The test images may be partially occluded or
may vary in expressions.
![Page 4: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/4.jpg)
4Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Object
This paper using the SOM to learn the subspace that represented each individual.
And then it uses a soft k nearest neighbor (soft k-NN) ensemble method to identify the unlabelled subjects.
![Page 5: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/5.jpg)
5Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Architecture Although template-based methods have
become one of the main techniques, a large training data set is not always possible in many real world tasks.
Beside above problem, there exist other problems, such as occlusion and expression.
![Page 6: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/6.jpg)
6Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Architecture (cont.) This paper extends Martinez’s work using SO
M and soft kNN and then it achieves high performance.
The procedure is as follows:─ Localization─ The use of SOM
The Single SOM-face Strategy The Multiple SOM-face Strategy
─ Identification
![Page 7: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/7.jpg)
7Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Architecture (cont.) Finally, this paper have conducted various
experiments to verify the performance of the proposed method.
![Page 8: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/8.jpg)
8Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Introduction Face Recognition Technology (FRT) has a vari
ety of potential applications in many aspect.
![Page 9: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/9.jpg)
9Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Introduction (cont.) However, the general face recognition problem
is still unsolved due to its inherent complexity.
To overcome this problem is to Search one or more face subspaces of the face to lower the influence of the variations.
![Page 10: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/10.jpg)
10Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Introduction (cont.) Most template-based FRT assume that
multiple images per person are available for training.
But a large training data set is not always possible in many real world tasks.
![Page 11: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/11.jpg)
11Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.The Proposed Method A. Localizing the face image:
─ the original image is divided into M(=l/d) sub-blocks with equal size, where l and d are the dimensionalities of the whole image and each sub-block.
ImageLocalization Images
![Page 12: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/12.jpg)
12Intelligent Database Systems Lab
N.Y.U.S.T.
I. M. The Proposed Method (cont.) B. The use of SOM
─ The SOM is chosen for several reasons as follows: It is efficient and suitable for high dimensional process Its algorithm is more robust to initialization than any other The trained SOM map are similar to input sub-blocks.
SOMProjection
ImageLocalization
Soft kNNEnsembleDecision Images Results
![Page 13: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/13.jpg)
13Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.The Proposed Method (cont.)─ The Single SOM-face Strategy
Step1: according to: Partition all the sub-blocks into Voronoi regions
Setp2: average : Setp3: Smooth :
─ The multiple SOM-face Strategy new image be presented to the system, denoted as
Then a separate small SOM map for the face will be trained using the above SOM algorithm.
![Page 14: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/14.jpg)
14Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.The Proposed Method (cont.) C. Identification
Given C classes, to decide which class the test face x belongs to, we first divide the test face into M sub-blocks.
and then project those sub-blocks onto the trained SOM maps.
Arranging it in increasing order :
normalization :
Finally, the label can be obtained :
![Page 15: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/15.jpg)
15Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments On the AR database (variations in Facial
Expressions)─ the neutral expressions images of the 100 individuals
were used for training, while the smile, anger and scream images were used for testing.
![Page 16: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/16.jpg)
16Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.)
![Page 17: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/17.jpg)
17Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.)
![Page 18: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/18.jpg)
18Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.) On the AR database (variations in partially occluded)
─ Simulated occlusion The number of the training data is same, while the smiling,
angry and screaming images with simulated partial occlusions were used for testing.
![Page 19: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/19.jpg)
19Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
![Page 20: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/20.jpg)
20Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.) We can find that half face occlusion does not h
arm the performance except the occlusion of upper face (see Fig.8b).
Because the lower half, included the mouth and cheeks, which can be easily affected by most facial expression variation.
![Page 21: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/21.jpg)
21Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.)─ Real occlusion
the neutral expression images of the 100 individuals were used for training, while the occluded images were used for testing.
![Page 22: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/22.jpg)
22Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.) It is interesting to note that the occlusion of the
eyes area led to better recognition results because the scarf occluded each face irregularly.
![Page 23: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/23.jpg)
23Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.) To simulate the occlusion, we randomly localized a s
quare of size pxp (5<p<50) pixels in each of the four testing image.
![Page 24: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/24.jpg)
24Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.) On the FERET database
─ Experiment 1 the performance of the two SOM-face based algorithms on the
subset was evaluated and was compared with other two method’s.
![Page 25: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/25.jpg)
25Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.)─ Experiment 2
choosing an appreciate k-value for the soft k-NN classifier.
![Page 26: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/26.jpg)
26Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.)─ Experiment 3
The effect of different sub-block sizes is studied.
![Page 27: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/27.jpg)
27Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.)─ Experiment 4
To investigate the incremental learning capability of the MSOM strategy, experiment was conducted using different gallery sizes.
![Page 28: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/28.jpg)
28Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Experiments (cont.)─ Experiment 5
we repeated one of the simulated occlusion experiments done on the AR dataset .
![Page 29: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/29.jpg)
29Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Conclusion This paper introduce the “SOM-face” to address the problem o
f face recognition with one training image per person and has several advantages over some of the previous methods.
It attributes these advantages to the seamless connection between the three parts of the method.
SOMImage
![Page 30: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/30.jpg)
30Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Conclusion (cont.) But the proposed method assumes that
occluded is known in advance. This paper shows that this paradigm works
well in the scenario of face recognition with one training image per person.
![Page 31: Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang](https://reader031.fdocuments.us/reader031/viewer/2022020118/568149ed550346895db71c87/html5/thumbnails/31.jpg)
31Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.Opinion Advantage