General Tensor Discriminant Analysis and Gabor Features for Gait Recognition

by D. Tao, X. Li, and J. Maybank, TPAMI 2007

Presented by Iulian Pruteanu

Duke University Machine Learning Group

Friday, June 8th, 2007

Outline

1. Introduction

2. Gabor representation

3. Linear discriminant analysis

4. General tensor discriminant analysis

5. Results (from the paper)

6. ISA vs. Gabor features

7. Results on video analysis

8. Conclusions

1/20

1. Introduction

1. The under sample problem (USP): the dimensionality of the feature space is much higher than the number of training samples.

2. General tensor discriminant analysis as a preprocessing step for LDA has some benefits compared with PCA or simple LDA: the USP is reduced and the discriminative information in the training tensors is preserved.

3. Gabor functions are used as a preprocessing step for feature extraction in image representation.

4. The LDA is used for classification combined with a dissimilarity measure: the distance between the gallery sequence and the probe sequence.

2/20

2. Gabor representation

1. A Gabor function is the product of an elliptical Gaussian envelope and a complex plane wave

where is the variable in a spatial domain and is the frequency vector which determines the scale and direction of Gabor functions

3/20

]exp[expexp),( 222,

2

2

22

xk

xkk

Ψ ids yx

),( yxx k

)exp( ds ik k

8

dd

The real part of Gabor functions (5 scales, 8 directions)

2. Gabor representation (contd.)

4/20

3. Linear discriminant analysis

• given a number of training samples in known classes, where is the

class number, and is the sample ID in the class with , the aim

of LDA is to find a projection of the , which is optimal for separating the different classes in

a low dimensional space.

• we define two scatter matrices

between-class

within-class

• the projection is chosen such as

Nn

jciij

i 11}{x c ici1 j thi inj1

ijx

c

i

Tiiib mmmmn

nS

1

))((1

c

i

n

j

Tiijiijw

i

mxmxn

S1 1

))((1

c

iinn

1

],...,,[ 21 cuuuU

UU

UUU

U wT

bT

S

Smax arg

5/20

3. Linear discriminant analysis (contd.)

• if , LDA reduces to the Fisher linear discriminant and the solution

corresponds to the largest eigenvalue of the following equation:

vvSSS bwb 2

112

1

kbk vS 2

1

U )( mxy Tk U

2c

6/20

• the general tensor discriminant analysis allows us to chose the optimal reduction in the

feature space. The projection matrix has a number of columns calculated in order to get the

best performance.

• if we want to extract features, we estimate as , where are the largest

eigenvalues of .

))()max(( arg UUUUUUU

wT

bT

I

SST

tuning parameter

)( x NN*NN *U

c

i

n

j

Tiijiij

TTii

Tc

ii

I

i

T

mxmxmmmmn1 11

))))((()))(((max( arg UUUUUUU

*N

*N

ii

1

*N

ii 1}{ *N vvSSS bwb 2

112

1

4. General tensor discriminant analysis

7/20

Alternating projection optimization procedure for GTDA

Step 1:

Step 2:

Convergence:

where is the number of classes (in our case ) and is the current step; indicates the feature

dimension which is minimized.

The tuning parameter and the dimension of the output tensors are determined automatically.

c

i

TTtli

Ttlii

tl n

1

111 )))((())(( UMMUMMB

c

i

n

j

TTtliij

Ttliij

tl

i

1 1

111 )))((())(( UMXUMXW

UWBUUU

* )-(max arg 11 tl

tl

Ttl 11- t

ltl WB

M

l

Ttl

tlt

1

1)(-)err( IUU

M

llijij

1

*UXY

t1c l

4. General tensor discriminant analysis (contd.)

8/20

c

The experiments are carried out upon the USF HumanID outdoor gait (1,870 sequences from 122 subjects).

For algorithm training, the database provides a gallery that has all the 122 subjects, collected at a separate moment in time.

For testing they use the dissimilarity measure: the distance between the gallery sequence and the probe sequence.

5. Results (from the paper)

9/20

6. ISA vs. Gabor features

10/20

Gabor functions:

• we use Gabor functions with five different scales and eight different orientations, making a

total of forty Gabor functions.

GTDA + Gabor features:

The original Gabor features : dim = 80 x 60 x 5 x 8.

The GaborSD features : dim = 80 x 60.

The GTDA + GaborSD features : dim = 10 x 6.

GTDA + ISA features:

The original ISA features : dim = 40.

The GTDA + ISA features : dim = 32.

GTDA + Gabor features

(dim = 60)

-18.297-20.385-21.35-15.396

-16.075-17.781-15.106-14.284

-14.639-23.026-21.827-16.018

-22.398-19.538-15.932-14.582

-16.21-19.403-21.045-14.492

-19.012-18.079-16.114-15.213

-17.185-19.895-20.142-14.515

-17.048-17.989-16.072-15.713

-15.436-22.631-22.757-17.581

-23.692-21.007-16.355-15.576

-18.854-19.124-22.338-15.842

-17.312-18.397-18.863-14.269

-21.037-16.981-18.595-15.291

-18.395-15.551-19.235-16.279

-16.886-20.084-18.795-18.305

-20.537-17.269-18.287-18.183

-18.421-16.783-18.611-17.085

-16.333-15.907-20.053-19.982

GTDA + ISA features

(dim = 32)

ISA features

(dim = 40)

11/20

-18.994-17.112-16.206-15.877

-19.283-16.827-17.617-51.873

-19.109-20.881-23.313-16.116

-18.707-29.834-51.993-15.001

-51.381-39.12-21.172-15.002

-21.019-41.978-18.928-17.139

-18.964-18.804-18.335-14.329

-20.206-17.368-18.191-53.709

-20.393-19.088-20.643-17.796

-18.324-30.429-53.921-17.248

-49.808-38.117-22.641-16.644

-20.528-43.842-19.987-18.579

-23.741-18.839-19.161-18.222

-22.069-19.735-17.077-48.593

-22.162-23.741-26.839-17.161

-21.222-35.069-46.318-18.162

-46.662-38.246-25.983-18.003

-24.267-37.299-21.011-20.318

GTDA + Gabor features

GTDA + ISA features

ISA features

12/20

Conclusion

1. Gabor functions and general tensor discriminant analysis have been introduced for visual information processing and recognition.

2. Tensor gait is also introduced to represent the Gabor features.

3. To further take the feature selection into account, the size of tensor gait is reduced by the GTDA

4. Gabor features and ISA are compared in abnormal event detection.

13/20