Activity, Audio, Indoor/Outdoor classification using cell phones Hong Lu, Xiao Zheng Emiliano...
-
Upload
todd-carroll -
Category
Documents
-
view
216 -
download
2
Transcript of Activity, Audio, Indoor/Outdoor classification using cell phones Hong Lu, Xiao Zheng Emiliano...
Activity, Audio, Indoor/Outdoor classification using cell phones
Hong Lu, Xiao Zheng Emiliano Miluzzo, Nicholas Lane
CS 185 Final Project presentation
Motivation
• In our lab we are working on a project where we use sensors to determine the sensing presence of people, i.e.:– Activity– Audio analysis– Location (Indoor/Outdoor)
• We are interested in using off-the-shelf devices like cell phones (with the camera, microphone, and accelerometer)
Requirements
• The activity and audio inference must be done on the phone (to improve scalability and limit the amount of bytes sent over the air)
• Downside: limited hardware (computation resources) and software (programmability) capabilities
Experimental methodology
• Over a period of one week we collected 250 audio samples of conversations and 250 audio samples w/out conversation
• 8 (>) people involved in different conversation settings
• The phone used to collect the samples was placed at different places (inside/outside pocket) and at different distances from the audio source
• The samples w/out conversation are taken indoors (office environment), outdoor in a quite location, and outdoor on the sidewalk (to catch the sound of cars, etc.)
Sound feature vector
• We focus on the 250 – 750 Hz portion of the spectrum
• We focus on the – Mean: because we see that the power differs
in the 2 cases in this portion of the spectrum– Std Dev: we also see that the power
oscillates more in the conversation case than in the no conversation case
ProblemDetecting when the cell phone is indoor or outdoors. Use the limited sensing capabilities of the phone.
Why try to do this?
1.Health Sensing Simple less accurate UV and pollution exposure with value in the potential for wide spread sampling
2.Context-dependant sensing queriesIndoor/outdoor classification is one useful criteria of describing conditions when (and when not) sense.
3.It seemed interesting to try
Intuition of an approach
• Sensor values will exhibit different characteristics depending on if the phone is indoors or outdoors.
• A key problem == Limited Sensors Available: – Different sensors would have made this simpler– Available: WiFi, GPS, Sound, Camera, GPRS etc.
• We tried feature vectors formed by using: – GPS + Radio Patterns (from WiFi)
• Various features tried that were based on:– WiFi AP Counting
• (assumption: lower density outdoors vs indoors)
– Sum of WiFi AP Neighborhood RSS • (assumption: lower signal strengths seen outdoors)
– GPS Satellite Signal Lock • (assumption: signal lock, which is effected by LOS issues, will occur
less often indoors than outside)
Hope from weak discriminators?
KEY: GREEN == OUTDOORS BLUE == INDOORS
Data Set: 1 user, 2,000 data points collected; 1 sample per minute. Sampling WiFi and GPS based measurements.
Groping for a modelTechnique Used: Logistic regression
Tested many variants of the inputs to find effective feature vectors.Windowed Variables – [RSS Sum, AP Count, GPS Flag]Variance of Windowed Variables – [RSS Sum, AP Count, GPS Flag]
Multiple variants of model inputs failed to provide sufficient discriminative power.
ANOVA test showed little reduction in deviance.We favored a simpler model over the complex one.
Checked that normal distribution existed in feature vectors
The Modelclassifier_label_param ~ ap_count + gps +
rss_sum
(Intercept) ap_count gps rss_sum -0.23107255 2.49738687 0.46060383 -0.03293664
Significance of the co-efficients? (Are they really different from zero?)
ap_count < 2e-16 ***gps 0.00358 ** rss_sum < 2e-16 ***
Empirical performance
dubious hypothesis results not impressive but interesting focused on other parts.
Mini-conclusion
Outdoor Indoor
Outdoor 1636 94
Indoor 259 468
Accuracy85.6%
Data collection
• 3-axis accelerometer embedded in the Nokia Sport Phone (in the future many phones will have an accelerometer)
• Sampling rate = 37Hz
• The phone can be clipped on the belt or carried inside the pocket
• Data labeled by the user
The raw data
0 2000 4000 6000 8000 10000 12000
-300
-200
-100
0
100
200
Time
Standing
0 2000 4000 6000 8000 10000 12000 14000 16000
-2000
-1000
0
1000
Time
Walking
0 1000 2000 3000 4000 5000 6000-4000
-2000
0
2000
Time
Running
X
YZ
X-axis – BLUE
Y-axis – GREEN
Z-axis – RED
From the 3 axis of the accelerometer
Feature Extraction
• Simply the mean and standard deviation
of each axis
• Window size = 128 data points
• Lightweight for running on the phone
PCA
• Use centerized PCA to get principal component projection matrix:
• Each column contains the coefficients for one principal component. The columns are sorted by decreasing component variance.
-0.0150 -0.0862 0.0762 -0.0316 -0.0801 0.9895 -0.0086 0.3733 0.4937 -0.7741 0.1314 -0.0197 0.0898 0.6502 -0.7286 -0.1438 0.0665 0.1149 -0.8425 -0.2966 -0.3070 -0.3274 0.0082 -0.0248 -0.4889 0.5694 0.3288 0.4243 -0.3854 -0.0008 -0.2067 0.1352 0.1310 0.3031 0.9074 0.0817
The 2 dimensional example
• Use the first 2 components, leave out the others.
-600 -500 -400 -300 -200 -100 020
30
40
50
60
70
80
standing
walkingrunning
Conclusion
• We wanted to do more than we could do
• Collecting the data took long time
• Programming the phones can be challenging
• The phone’s hardware and software platform limit the applicability of off-the-shelf data mining techniques
Future work
• Increase the number of activities we can detect (including sitting, cycling, climbing stairs, etc)
• Augment the audio classifier to be able to do speaker recognition, party detection, type of noise (car, wind, etc)
• Use the cell phone’s camera and light detector to make the outdoor/indoor classifier more robust