Shape and Dynamics in Human Movement Analysis

Ashok Veeraraghavan

Outline Motivation What do we want to do? Shape Shape based methods for recognition Dynamics based methods for

recognition Results Current Work

Motivation

Human Perception Shape or Dynamics (or is it Both??)

Laurel and Hardy

Who is this ? ? ?

Introduction Psychophysics work indicates that

dynamics is important for recognition in humans.

Johansson: Light Display Moving dots Murray(1964) : 24 gait components Cutting :Familiarity;Static Vs Dynamic Kozlowski: dynamics speed,

bounciness, rhythm. Cutting : Dynamic Invariant Gender Discrimination

Prior Work

Image Correlation. Silhoutte Based Nearest

Neighbour. Dynamic Time Warping Hidden Markov Model Model parts of human body and

extract gait signature.(eg., Thigh)

Most gait recognition algorithms are shape based !

Relative importance of shape and dynamics

Definition of Shape “Shape is all the geometric

information that remains when location, scale and rotational effects are filtered out from the object”.

Kendall’s Statistical Shape Theory used for the characterization of shape.

Pre-shape accounts for location and scale invariance alone.

Pre-Shape

k landmark points (complex vector)

Translational Invariance: Subtract mean

Scale Invariance : Normalize the scale

1, 1 1 T

c k k k

CXZ where C I

kC X

Feature Extraction Silhoutte

Landmarks

Centered Landmarks

Pre-shape vector

Distance between shapes

Shape lies on a spherical manifold. Shape distance must incorporate

the non-Euclidean nature of the shape space.

1)Full Procrustes distance.2)Partial Procrustes distance.3)Procrustes distance.

Full Procrustes Distance

Procrustes Fit

Full Procrustes Distance=Minimum Procrustes Fit.

( , ) ( )1 .jkd Y X se a jb

, , ,( , ) inf ( , ).F

s a bd Y X d Y X

Other shape distances

Partial procrustes distance

Procrustes distance (ρ): distance on the Great circle.

( )( , ) inf .P SO m

d X Y

( , ) sin ,

( , ) 2 sin( ) .2

F

P

d X Y

d X Y

Tangent Space Linearization of spherical shape

space around a particular pole. The Procrustes mean shape is

usually chosen as the pole. If the shapes in the data are very

close to each other then Euclidean distance in tangent space approximates shape distances.

Shape based methods for Recognition

Stance Correlation.

Dynamic time warping in shape space.

Hidden Markov Model in shape space.

Stance Correlation Exemplars for 6

stances for each individual.

The correlation between exemplars is used as the matching criterion.

Performance comparable to Baseline.

Dynamic time warping in shape space .

Enforce end-point constraint. Obtain best warping path. Cumulative error is computed

using the shape distances described.

Performance is better than baseline.

Hidden Markov Model in shape-space Exemplars are regarded as states. HMM built for each person in the

gallery. Identity established by maximizing

the probability that the observation came from the model in the gallery.

Performance is better than baseline and comparable to DTW.

Dynamical Models

Stance based AR model.

Linear Dynamical System

Stance based AR model Video sequence is clustered into 3

distinct stances. Each frame is identified as belonging to one of these three stances.

Parameters of an AR model learnt for each stance.

Model parameters used for recognition.

Performance is below baseline.

Linear Dynamical System(ARMA)

Parameters (A,C) of a dynamical system learnt for each individual.

Distance between models used as score for recognition.

( ) ( )

( ) ( ) ( ) ; ( ) (0 , )

( 1) ( ) ( ) ; ( ) (0 , )

L e t t b e t h e o b s e r v a t i o n p r e s h a p e

t C x t w t w t N R

x t A x t v t v t N Q

Results on USF database

Gallery 71 people. Probe varies from Gallery in view,

shoe and surface. Average CMS curves shown.

Sample Sequences

Comparison of various methods on the USF database.

Comparison of various methods on the USF database.

Results on the CMU database

Gallery consists of 25 people.

3 different activities studied: Slow walk, Fast walk and walk with ball.

Recognition performed within and across activities.

Percentage of Recognition using Stance correlation.

Slow Walk

Fast Walk

Ball

Slow Walk

100 80 48

Fast Walk

84 100 48

Ball 68 48 92

Similarity Matrix using Linear Dynamical system(ARMA)

Percentage of Recognition using Linear Dynamical system

Slow Walk

Fast Walk Ball

Slow Walk

100 72 64

Fast Walk 75 100 60

Ball 70 50 100

Conclusions Shape is more important for recognition

than dynamics. Shape also provides for speed change invariance.

Dynamics can help to improve performance of shape based methods.

Activity Recognition: Dynamics plays a important role.

Dynamical models like ARMA can perform recognition across activities.

Current Work

Experiments on Manually Segmented Gait data. An attempt to isolate the effects of kinematics. ( so far “kinematics” referred to the dynamics of the representation feature (shape)).

Building a kinematic model for gait. This model will be used as a prior for human body tracking.

Manually Segmented Data

Extract Joint Angles from manually segmented data.

Each gait sequence is now a sequence of Joint angles.

Dynamic Time Warping on Joint Angles.

Effect of Joint angles alone on HMM/DTW recognition

performance

Impose the extracted joint angles on a 3-d Volumetric Model (Thanks to Aravind Sundaresan) and artificially generate a gait sequence.

Each sequence corresponds to same body shape but different style of walking.

HMM and DTW for recognition on these silhouttes.

Recognition Performance of Shape normalized silhoutte data

Gait Modeling

Model the trajectories of the joint angles over the gait sequence.

Exploring Vector Autoregressive model (Order atleast 2).

1

pn i nn ii

A e

MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM

Model Based Tracking

Dynamical Model of gait will serve as a guide during 3-d model based tracking of humans.

Hope to use image cues such as edges, silhouettes etc. as observations for tracking.

Thank You!!