1/96

Gait Recognition Using WiFi Signals

Wei Wang Alex X. Liu Muhammad ShahzadNanjing University Michigan State University North Carolina State University

Nanjing University

2/96

3/96

Gait Based Human Authentication & Identification

§ Human Authentication– Problem Statement: Are you Trump or not?

– Why need gait based human authentication?• Multi-factor human authentication

§ Human Identification– Problem Statement: Which one of the following people are

you: Trump or Hillary?

– Why need gait based human identification?• Customization

4/96

Gait Based Human Authentication & Identification

§ Why possible?– Individually unique gait patterns

§ What have been done?– Video camera

• Cannot work in dark• Privacy concern

– Wearble sensors• Inconvenient

– Floor sensors• Expensive

§ What do we propose?– WiFi signals

5/96

WiFiU: Gait Based Human Authentication & Recognition Using WiFi Signals

§ Using commercial off the shelf devices§ Woks in dark

§ Privacy friendly

NetGEARJR6100

LenovoThinkpad X200

6/96

Key Insight§ Human body reflects wireless signals

§ Chanel State Information (CSI) has huge amount of useful information about environmental changes– 2,500 samples per second– Complex valued samples with 8 bit accuracy

– On 30 subcarriers for each antenna pair

§ So, commercial WiFi devices can act as Doppler Radars to measure human activities

Signal analysis

WiFi router

Laptop

WiFi signal reflection

WiFi signal

7/96

System Architecture

Data collection and pre-processing

Feature extraction

- Gait cycle time- Torso speed- Footstep size- Leg speed- Spectrogram signature

1. CSI data collection

2. PCA denoising

3. Spectrogram generation

Identification

1. Model generation

2. Prediction

Training data collection

SVM basedprediction

8/96

How to deal with noisy signals?§ Signals from commercial WiFi

devices are very noisy

§ Traditional filter based denoisingapproaches not work

§ We proposed a Principal Component Analysis based approach in our MobiCom 15 paper

11 11.5 12 12.560

65

70

75

CS

I

Time (seconds)

Original

11 11.5 12 12.5

65

70

75

CS

I

Time (seconds)

Low-pass filter

11 11.5 12 12.5−10

−5

0

5

10

Time (seconds)

CS

I

PCA

9/96

PCA Based Noise Reduction§ Changes in CSI streams caused by human movement are correlated

– Theoretically proved– Experimentally validated

10/96

How to extract gait information?§ Challenge: signal reflections of different body parts are

mixed together in the wave form.

§ Insight: different body parts move at different speeds– Signal reflections of different parts have different frequencies.– CSI fluctuations of different frequencies are separable in the

frequency domain.

§ How: convert waveforms into time-frequency domain– Use Short-Time Fourier Transform (STFT) to convert each slice

of waveforms to a spectrogram.

– Spectrogram 3 dimensions: time, frequency, and FFT amplitude– Window size: tradeoff between frequency and time resolutions

• Larger: higher frequency resolution, low time resolution• Smaller: low frequency solution, high time resolution

– Our choice: FFT size=1024 samples, window size=32 samples• Frequency resolution=2.44Hz, time resolution=12.8ms

11/96

Spectrogram Enhancement§ We apply spectrogram enhancement techniques to

further reduce noise.

§ Insight: CSI spectrograms give similar information as the expensive Doppler radars

12/96

Feature Extraction§ Gait cycle time

– Obtain upper contour of the torso reflection.

– Use autocorrelation of upper contours to estimate gait cycle time.

§ Torso and leg speeds– Use percentile method for

Doppler radars

0.5 1 1.5 2 2.5 3−400

−200

0

200

400

τ (seconds)

Au

toco

rrela

tio

n v

alu

e

Autocorrelation of torso contour

Peaks for autocorrelation

L/2L/2 L/2

0 1 2 3 4 5 60

0.5

1

1.5

2

Time (Seconds)

Sp

ee

d (

m/s

)

50% percentile

95% percentile

Torso contour

13/96

Feature Extraction§ Spectrogram signatures

– Motivation: gait cycle time and torso speeds are not enough

– Idea: use the distribution of reflected energy on predetermined frequency points as signatures

– Why: energy distributions give an overview on how different body parts move at a given stage of walking

§ Extraction method:– Divide each walking cycle into 2 phases by torso contour curve.

– Further divide each half-gait-cycle into 2 stages of leg swinging.

– For each stage, calculate the normalized energy on 40 frequency points.

14/96

Training and Identification§ Use SVM to generate model

– Feature vector size = 170

– Sample size = 40

§ Two applications– Single user authentication

– Multiple users identification

15/96

Data Collection§ Devices

– NetGear WiFi Router

– ThinkPad Laptop

(with Intel 5300 nic)

§ Dataset– 50 users

– 50 walking samples for each user

– Walking for 5.5 meters

§ Metrics– Distance

– Effectiveness

– Robustness

– Efficiency

Table

Table

Walking route (5.5m)

7.7 m

6.5 m

1.6 m

sender

receiver

16/96

Distance§ WiFiU can detect a walking human subject at a range

as long as 14 meters with an accuracy of 92%.

17/96

Accuracy§ Human authentication

– Average FAR=8.05%

– Average FRR=9.54%

§ Human identification– Top-1 accuracy=92.31%

– Top-2 accuracy=97.58%

– Top-3 accuracy=98.86%

18/96

Robustness§ Evolution of human gait does not significantly affect

WiFiU accuracy– Collected data over 4 months

§ WiFiU is also robust to small changes in environments such as chair movements.

19/96

Efficiency§ WiFiU can run in real time on laptops and desktops.

– Less than 1 second.

20/96

Limitations§ Single person walking

– Multiple users are OK as long as other remain roughly static.

§ Predefined walking path– Doppler spectrograms are sensitive to walking directions.

§ Limited accuracy– Not enough for single-factor authentication.

21/96

Conclusions§ We demonstrate the feasibility of using WiFi signals

from COTS devices for gait based human authentication and identification.

§ We propose signal processing techniques for converting raw noisy WiFi signals to high-fidelity spectrograms.

§ We propose methods to further convert spectrograms to gait features.

22/96