Wave to Me: User Identification Using Body Lengths and Natural Gestures

Eiji HayashiManuel Maas

Jason Hong

Human-Computer Interaction InstituteCarnegie Mellon University

Slick user identification

Slick user identificationwith reasonable security

Gesture

EachDifferent Gesture for user

AllSame Gesture for user

11

97% accurate

SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study

SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study

Body-basedUser Identification

Overview

Body-basedUser Identification

Registration

Body-basedUser Identification

Registration

Identification

Body-basedUser Identification

Registration

Identification

User IDReject

Kinect

Kinect

Kinect

JointPositions

KinectFeatureExtraction

JointPositions

KinectFeatureExtraction

Physiological

17 body segmentlengths

KinectFeatureExtraction

Physiological

17 body segmentlengths

Behavioral

26 movementproperties

KinectFeatureExtraction

43 Features

KinectFeatureExtraction

Feature Vector

KinectFeatureExtraction

SVM

Feature Vector

KinectFeatureExtraction

SVM Pre-RecordedData

Feature Vector

KinectFeatureExtraction

SVM Pre-RecordedData

User ID + Confidence

KinectFeatureExtraction

SVM

Threshold

Pre-RecordedData

User ID + Confidence

KinectFeatureExtraction

SVM

Threshold

Pre-RecordedData

User ID or Reject

Errors

False Acceptance Rate (FAR)Accept others as a registered user

False Rejection Rate (FRR)Reject a registered user as others

Errors

False Acceptance Rate (FAR)Accept others as a registered user

False Rejection Rate (FRR)Reject a registered user as others

Equal Error Rate (EER)FAR = FRR = EER

Errors

False Acceptance Rate (FAR)Accept others as a registered user

False Rejection Rate (FRR)Reject a registered user as others

Equal Error Rate (EER)FAR = FRR = EER

Accuracy = 1 – 2 x EER

Assumption

There are 7 registered users in our system

Assumption

There are 7 registered users in our system

Make comparison among studies easy

Assumption

There are 7 registered users in our system

Make comparison among studies easy

Be reasonable for home use

Assumption

There are 7 registered users in our system

Choose 10,000 combination of 7 participants

Calculate EER over them

SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study

Gestures

Hand Waving

Come-Over

One Hand Raised

Making a Phone Call

Data Collection

Hand WavingCome-OverOne Hand RaisedPhone Call

Gesture

Data Collection

Hand WavingCome-OverOne Hand RaisedPhone Call

StandingSitting

Gesture Posture

Data Collection

Hand WavingCome-OverOne Hand RaisedPhone Call

StandingSitting

1st Day3 days later

Gesture Posture Session

Data Collection

Hand WavingCome-OverOne Hand RaisedPhone Call

StandingSitting

1st Day3 days later

Gesture Posture Session

10

Data Collection

Hand WavingCome-OverOne Hand RaisedPhone Call

StandingSitting

1st Day3 days later

Gesture Posture Session

10

160 / participants

Participants

36 participants

14 males / 22 females

19 – 64 years old

168cm (SD=10.2)

78.0 kg (SD=22.0)

Gestures

Hand Waving

Come-Over

One Hand Raised

Making a Phone Call

Using Either Gesture or Lengths

Same day & posture

3 days later

Different Posture

Same day & posture

3 days later

Different Posture

Gesture Body Lengths

EER [%]

Using Either Gesture or Lengths

Same day & posture

3 days later

Different Posture

Same day & posture

3 days later

Different Posture

Gesture Body Lengths

EER [%]

2.1%

0.5%

Using Either Gesture or Lengths

Same day & posture

3 days later

Different Posture

Same day & posture

3 days later

Different Posture

Gesture Body Lengths

EER [%]

11.8%

19.8%

Using Either Gesture or Lengths

Same day & posture

3 days later

Different Posture

Same day & posture

3 days later

Different Posture

Gesture Body Lengths

EER [%]

10.0%

41.5%

Using Both Gesture and Lengths

3 days later

3 days later

Gesture Body Lengths

EER [%]

Both

3 days later (Standing)

3 days later (Sitting)

4.3%

6.2%

SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study

Data Collection

Hand WavingStandingSitting

1st Day3 days later1 week later2 weeks later

Gesture Posture Session

10

80 / participants

Participants

27 participants

20 males / 7 females

19 – 62 years old

173cm (SD=9.8)

75.1 kg (SD=21.1)

Long term StabilityE

ER

[%]

Days

Long term StabilityE

ER

[%]

Days

Sitting

Standing

Long term StabilityE

ER

[%]

Days

Stable after the 3rd session

Training with 2 sessionsE

ER

[%]

Days

Training with 2 sessionsE

ER

[%]

Days

Sitting

Standing

Training with 2 sessionsE

ER

[%]

Days

EER < 1%

SystemLab Study 1 (Basic Evals)Lab study 2 (Long-term Eval)Field Study

Does it Work at Homes?

• Collected data at participants’ living rooms• Placed a Kinect on a TV• Asked participant to behave as usual

– Stand where you feel reasonable– Sit as you normally do in a living room

Participants

12 participants (5 house hold)

5 males / 7 females

18 – 42 years old

159.5cm (SD=11.4)

56.9 kg (SD=6.9)

It Worked!

EER [%]

Standing

Sitting

Standing

Sitting

Lab Study 2 Field Study

Implication

Recognizing a gesture

Recognizing a gesture AND a user’s identity

• Natural gestures + body lengths• 2 seconds of hand waving gesture

Conclusion

Gesture

Body Lengths

Proposed Scheme

EER [%]

Wave to Me: User Identification Using Body Lengths and Natural Gestures

Eiji Hayashiehayashi@cs.cmu.eduwww.cs.cmu.edu/~ehayashi/

Human-Computer Interaction Institute

Carnegie Mellon University

Backup slides

# of Registered UsersE

ER

[%]

# of Registered Users

# of Registered UsersE

ER

[%]

# of Registered Users

2.8%

2.3%

N=25

Open Questions

Getting worse constantly?

Training with two sessions?

Data Collection

Hand WavingStandingSitting

1st Session2nd Session3rd Session

Gesture Posture Session

10

60 / participants

Yet Another Open Question

Does it actually work at home?