KineticSense: Wearable Context Sensing from Kinetic …mahbub/PDF... · [PerCom2015] Pervasive...
Transcript of KineticSense: Wearable Context Sensing from Kinetic …mahbub/PDF... · [PerCom2015] Pervasive...
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Never Stand Still Faculty of Engineering Computer Science and Engineering
Click to edit Present’s Name
Never Stand Still Faculty of Engineering Computer Science and Engineering
KineticSense: Wearable Context Sensing
from Kinetic Energy Harvesting
Mahbub Hassan [email protected]
School of Computer Science and Engineering University of New South Wales, Sydney, Australia
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Health Care Problems in Australia
• 1993 to 2013, # of hospital beds dropped from 30 to 17 (per 1000 people of 65+)
• 2009 to 2013, private hospital admissions for subacute and non-acute services grew by 14% a year
• 2003 to 2013, health care cost grew from 68 B to 172 B (14% of GDP)
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Automating Health Care with Wearable Context Sensing
Health care provider
Patient@Home (No hospital admission)
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Growing Popularity of Wearable Devices
• Continuous activity and context monitoring with cloud analytics • Step count, calorie burning, sleep analysis, …
Sensor Sampling
Tx Sensor Data to Cloud
Cloud Analytic
1 2 3
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Battery Recharging Bottleneck • Too many sensors to sample – accelerometers,
gyroscopes, magnetometers, microphones, GPS, temperature, humidity, pulse rate, …
• Batteries last for few hours (intense use) to few weeks at most
• Battery recharging is inconvenient at best and problematic in many cases, such as elderly patient monitoring
• Battery recharging is a major roadblock for pervasive deployment of current wearable devices
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Why Not Use Human Energy Instead of Batteries? • A human accumulates and expends a tremendous amount
of energy each day
1 doughnut = 1.3 Mega Joule
Sprinting expends 1.6KW
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Powering wearable devices from Kinetic Energy Harvesting convert motion energy to electrical energy
Piezoelectric Energy Harvester
Sensors
CPU
Radio
Harvested Power A few to few hundred µW could be harvested in a wrist-worn device
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Overview of My Talk ① A new context sensing framework for human-powered
wearables à detect human contexts directly from harvested energy patterns à significantly more power efficient than existing sensor-based sensing paradigm
② Experimental evidence of successful context sensing with real energy harvesters and real human subjects (Contexts: activity, steps, calorie, voice, mobility, location, identification)
③ Limitations of current prototype and future directions
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
The Essence of KineticSense
KEH Sensor Array (acc,gyro,mag,mic,…)
Context Detection
Power Samples
KEH Context Detection
Samples
KineticSense
Conventional Approach
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
EXPERIMENTING WITH REAL ENERGY HARVESTERS AND HUMANS
Can we infer human activity from kinetic energy harvesting?
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Piezoelectric Energy Harvester (vibration-based)
Mide.com
AC Rectifier
Recharge battery/
Capacitor
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
KEH Data Logger (samples AC voltage values)
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
VEH Data Logger
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Exp #1 [IC2015] Human activity recognition – KEH Data Collection
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
KEH Time Series
Walking
Running
Standing
accelerometer VEH
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Does KEH contain information for activity recognition In
form
atio
n G
ain
Features
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Activity Recognition Accuracy – 5 activities (W, R, S, SU, SD)
Classifier Activity Recognition Accuracy (%)
Accelerometer-based
KEH-based
Hand Waist Hand Waist
K-nearest neighbour 95.01 98.70 81.13 87.01
Decision Trees 87.91 91.02 79.74 79.86
Multilayer Perceptron
88.25 96.39 78.29 85.52 Low
er th
an a
ccel
erom
eter
, bu
t not
too
bad
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Exp #2 [iThings2015] Step counting
4 su
bjec
ts, s
trai
ght a
s w
ell
as tu
rnin
g pa
ths,
pea
k de
tect
ion
algo
rithm
, 570
st
eps,
96%
acc
urac
y
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Exp #3 [BODYNET2015]
Calorie burning estimation • 10 subjects, 4 male, 6 female: waist mounted • Anthropometric data: Age (26-35 years, mean = 29, s.d.
= 3.06), Weight (58-91 Kg, mean = 69.3 s.d. = 10.21), Height (154-185 cm, mean = 168.5, s.d. = 9.98)
• Two activities: walking and running • Linear regression model to estimate calorie burning from
energy harvesting samples and anthropometric data • Leave-one-out cross validation (1 for testing, 9 for
training)
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Calorie estimation results - running
Close to accelerometer
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Exp #4 [PerCom-WIP2016]
Mobility Monitoring
Train Car Bus Walking
Running
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
2-layer classification
Peak analysis
Mean analysis
Summary of Trace Data
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Results
93% 77% 88%
Bus and Car are confused, but not train
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Exp #5 [WOWMOM2016]
Hotword detection
3cm
Quiet Room Conditions
Hotword: “OK Google” Non-hotwards: “Good morning”, “how are you”, “fine, thank you” 8 subjects: 4 m, 4 f 60 instances (30 hotword 30 non-hotword) per subject
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Speaker orientation
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Results
Flat Vertical
Speaker Independent
78% 62%
Speaker Dependent
85% 73%
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Exp #6 [NDSS2017]
User Identification (via gait recognition)
Electromagnetic Energy Harvester (EEH)
We have experimented with both piezoelectric as well as electromagnetic energy harvesters
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
PEH and EEH Signals from Different Subjects (Walking)
2
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
User Identification (gait recognition) Accuracy
20 subjects
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Exp #7 [PerCom2017]
Location Context (based on short-range acoustic communication)
If the energy harvester can recognize known sound vibrations, then a wearable can detect its location from a sound beacon
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Impact of Transmitted Sound on Energy Harvester
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Communication Performance
5bps at 80 cm for BER < 1% (laptop to wearable)
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Natural Immunity Against Noise Even loud music didn’t affect the communication performance
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Conclusion
• Positive: A vast array of useful contexts can be detected using KEH as a sensor à power saved can be used for collecting more physiological data (improved health care)
• Negative: Power generated is small and detection accuracy not good for fine-grained contexts
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Future Directions
① Advanced machine learning to improve context detection accuracy
② Hybrid energy harvesters à increase both power and information content Ø Multi-axial PEH, multi-modal KEH, hybrid with PEH+TEG
③ Fancy wireless communications, e.g., pulse-based energy rate transfer à reduce transmission power consumption [TOSN2017]
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
2-axial PEH prototype in progress
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
References 1. [PerCom2015] Pervasive Self-powered Human Activity Recognition without the Accelerometer, PerCom 2015
2. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services, IEEE Internet Computing, 9(5), 2015 3. [iThings2015] Step detection from power generation pattern in energy-harvesting wearable devices, IEEE iThings 2015
4. [BODYNET2015] Estimating Calorie Expenditure from Output Voltage of Piezoelectric Energy Harvester - an Experimental Feasibility Study, BODYNETS 2015
5. [PerCom_WIP2016] Transportation Mode Detection using Kinetic Energy Harvesting Wearables, PerCom Work-in-Progress, 2016
6. [WOWMOM2016] Feasibility and Accuracy of Hotword Detection using Vibration Energy Harvester, WOWMOM 2016.
7. [NDSS2017] KEH-Gait: Towards a Mobile Healthcare User Authentication System by Kinetic Energy Harvesting, NDSS 2017.
8. [PerCom2017] VEH-COM: Demodulating Vibration Energy Harvesting for Short-Range Communications, PerCom 2017
9. [TOSN2017] SEMON: Sensorless Event Monitoring in Self-powered Wireless Nanosensor Networks, ACM Tran. On Sensor Networks, Accepted 05 March 2017
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
Questions?
Mahbub Hassan Missouri University of Science and Technology, 10 March 2017
96% power saving
Power Measurements (KEH vs accelerometer) TI Sensor Tag