Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf ·...
Transcript of Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf ·...
![Page 1: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/1.jpg)
Gabriel: Towards
Wearable Cognitive Assistance
Kiryong Ha, Zhuo Chen, Wenlu Hu, Wolfgang Richter,
Padmanabhan Pillai, and Mahadev Satyanarayanan
![Page 2: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/2.jpg)
Faces Daily Routine
Cognitive Decline
20,000,000+ Americans affected
Ron traumatic brain injury
Alzheimer’s disease
mild cognitive impairment survivors of stroke
One-month delay in nursing home
admissions saves $1,000,000,000+/year
10/24/2014 2
Text
![Page 3: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/3.jpg)
Can Wearable Technology Help?
10/24/2014 3
Continuously capture, interpret, give guidance and
System Architecture
![Page 4: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/4.jpg)
10/24/2014 4
“Please stop and check traffic”
“Barack is saying hello to you”
“Your dog wants to go out for a walk”
![Page 5: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/5.jpg)
Why Today? Advances in 3 Independent Arenas
10/24/2014 5
Algorithms
Offloading Technology
Wearable Hardware
![Page 6: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/6.jpg)
Challenges – Architecture
1. Crisp Interactive Response
2. Graceful Degradation of Services
3. Coarse-grain Parallelism
10/24/2014 6
![Page 7: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/7.jpg)
C1. Crisp Interactive Response
Goal: Latency of infrastructure = tens of millisecond
10/24/2014 7
Face exists?
<700ms
Whose face?
~500ms
Voice Phrases?
~400ms
Humans are fast and sensitive
Satellite Phone?
ANNOYING!
![Page 8: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/8.jpg)
S1. Crisp Interactive Response
Choice 1: standalone apps
Choice 2: offload to cloud
Choice 3: offload to cloudlets
10/24/2014 8
Distant Resources in Cloud
Nearby Cloudlet
Wi-Fi WAN
LAN
~74ms
<10ms
Glass Device
![Page 9: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/9.jpg)
Exp. – Cloudlet Shortens Latency
10/24/2014 9
0
0.25
0.5
0.75
1
0 200 400 600
Response Time (ms)
Cloudlet
Cloud
Cloudlet vs. Cloud (Augmented Reality)
Offloading vs. Standalone (OCR)
Metric Standalone With Offload
Per-image speed (s) 10.49 1.28
Per-image energy(J) 12.84 1.14
Cloudlet shortens
response time
Offloading saves
latency and energy
![Page 10: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/10.jpg)
C2. Graceful Degradation of Services
10/24/2014 10
What if offloading impossible?
Distant Resources in Cloud
Wi-Fi WAN
LAN
Situation 1: No cloudlet
Situation 2: No network
? ?
Nearby Cloudlet
Goal: still work during failures – with performance drop
Glass Device
![Page 11: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/11.jpg)
S2. Graceful Degradation of Services
10/24/2014 11
Use fallback resources
No cloudlet – offload to cloud
No network – offload to carry-on device
Application-specific fidelity vs. Crispness & battery life
Distant Resources in Cloud
Glass Device
Nearby Cloudlet
Wi-Fi WAN
LAN
? ?
Wi-Fi or Bluetooth
On-body Offload Device
![Page 12: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/12.jpg)
C3. Coarse-grain Parallelism
• Programming languages are different
• Runtime systems are different (different OSes, closed-
source, etc.)
10/24/2014 12
Don’t reinvent the wheel
Goal: reuse existing work, but…
Face recognition
Object detection
Activity inference
OCR ……
![Page 13: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/13.jpg)
S3. Coarse-grain Parallelism
10/24/2014 13
VM Ensemble and PubSub Backbone
Core Algorithm
Logic
Virtual Machine
Dataset
Proxy
Config
![Page 14: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/14.jpg)
Glass Device
S3. Coarse-grain Parallelism
10/24/2014 14
Cognitive Engine 1 e.g. Face recognition
Cognitive VMs
Video/Acc/
sensor streams
Device
Comm
UPnP
PubSub
Control VM
Cloudlet VM boundary
Sensor flows
Cognitive flows
User Guidance VM
VM Ensemble and PubSub Backbone
Cognitive Engine 2
Cognitive Engine 3
Cognitive Engine n
...
Wi-Fi
![Page 15: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/15.jpg)
S3. Coarse-grain Parallelism
10/24/2014 15
Cognitive Engine 1 e.g. Face recognition
Cognitive VMs
Device
Comm
UPnP
PubSub Video/Acc/
sensor streams
Control VM
Cloudlet
User Guidance VM
Cognitive Engine 2
Cognitive Engine 3
Cognitive Engine n
...
Glass Device
Wi-Fi
Context Inference Wi-Fi
VM boundary
Sensor flows
Cognitive flows
VM Ensemble and PubSub Backbone
![Page 16: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/16.jpg)
Exp. – Gabriel Overhead
10/24/2014 16
0
0.25
0.5
0.75
1
0 20 40 60 80 100
Response Time (ms)
Gabriel
Ideal
Echos every image
Dummy Cognitive Engine
No VM, echos from host
Gabriel Latency: ~50ms
Gabriel Overhead: ~3ms
![Page 17: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/17.jpg)
10/24/2014 17
GDK Preview
Ice pack to cool down Glass
TCP Connection
Speech Guidance
Back-end Server
Prototype
![Page 18: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/18.jpg)
10/24/2014 18
Cloudlet: 4 advanced desktop machines Running OpenStack – Virtualized Cloud Computing Platform
Back-end Server
Prototype
![Page 19: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/19.jpg)
10/24/2014 19
Face Recognition
Object Recognition (1. MOPED 2. STF )
OCR (1. Tesseract 2. VeryPDF )
Motion Classifier
Augmented Reality
Activity Detection
Commercial Product
Based on Accelerometer
Prototype Cognitive Engines
![Page 20: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/20.jpg)
Exp. – Full System Performance
10/24/2014 20
Gabriel architecture allows easy upgrade.
Cognitive Engines are slower
Cognitive Engine FPS Response time (ms)
Glass Life 1% 10% 50% 90% 99%
Face Recognition 4.4 196 389 659 929 1175
~1 hour
Object (MOPED) 1.6 877 962 1207 1647 2118
Object (STF) 0.4 4202 4371 4609 5055 5684
OCR (Open) 14.4 29 41 87 147 511
OCR (Comm) 2.3 394 435 522 653 1021
Motion Classifier 14.0 126 152 199 260 649
Augmented Reality 14.1 48 72 126 192 498
![Page 21: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/21.jpg)
Exp. – Full System Performance
10/24/2014 21
Gabriel uses two-level token-based flow control
Cognitive Engines require different FPS
Cognitive Engine FPS Response time (ms)
Glass Life 1% 10% 50% 90% 99%
Face Recognition 4.4 196 389 659 929 1175
~1 hour
Object (MOPED) 1.6 877 962 1207 1647 2118
Object (STF) 0.4 4202 4371 4609 5055 5684
OCR (Open) 14.4 29 41 87 147 511
OCR (Comm) 2.3 394 435 522 653 1021
Motion Classifier 14.0 126 152 199 260 649
Augmented Reality 14.1 48 72 126 192 498
![Page 22: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/22.jpg)
10/24/2014 22
drop
M A X ••
•
drop
drop
• •
•
Glass Device
Control VM
Cognitive VM
Cognitive VM
sensor
sensor data counts
yes no
a > b?
sum
a b
Processed Items Count
Drop Count
tokens
![Page 23: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/23.jpg)
More in the Paper
1. Token-based flow control improves response
time a lot
2. Gabriel supports multi-VM parallelism
3. Tradeoff between fidelity reduction and crisp
user interaction
10/24/2014 23
![Page 24: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/24.jpg)
10/24/2014 24
Conclusion & Future Work
Algorithms
Offloading Technology
Wearable Hardware
Gabriel: low-latency, flexible architecture
Speed improvement
needed
Cloudlets are helpful,
need good biz. model
Longer battery, better
thermal dissipation
![Page 25: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/25.jpg)
10/24/2014 25
Cognitive Assistance Wearable Towards
Gabriel:
Contact: Kiryong Ha ([email protected]) Zhuo Chen([email protected])
![Page 26: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/26.jpg)
Backup
10/24/2014 26
![Page 27: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/27.jpg)
S3. Coarse-grain Parallelism
10/24/2014 27
Cognitive Engine 1 e.g. Face recognition
Cognitive VMs
Device
Comm
UPnP
PubSub Video/Acc/
sensor streams
Control VM
Cloudlet
User Guidance VM
Cognitive Engine 2
Cognitive Engine 3
Cognitive Engine n
...
Glass Device
Wi-Fi
Context Inference
VM boundary
Sensor flows
Cognitive flows
VM Ensemble and PubSub Backbone
![Page 28: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/28.jpg)
10/24/2014 28
Prototype Mitigating Queueing Delay
Queues hurt latency:
Node A
Node B
Two-level token-based flow control
![Page 29: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/29.jpg)
10/24/2014 29
Queues hurt latency:
Node A
Node B
Two-level token-based flow control
Throughput vs. Latency
= 1 0 ACK
Prototype Mitigating Queueing Delay
![Page 30: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/30.jpg)
Exp. – Queuing Delay Mitigation
10/24/2014 30
0
10
20
30
40
0 100 200 300 400
Res
po
nse
del
ay
(s)
Request time (s)
token=1 token=2 TCP
0
200
400
600
token = 1 token = 2 tcp
nu
mb
er o
f fr
am
es
On-time Late
Dropped at Backend Dropped at Glass
![Page 31: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/31.jpg)
Fidelity vs. Response Time
10/24/2014 31
![Page 32: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/32.jpg)
S1. Crisp Interactive Response
10/24/2014 32
Justification of offloading
Table 1: Evolution of hardware performance
Table 2: Experiment result with OCR
![Page 33: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/33.jpg)
Scale-Out of Cognitive Engines
• Cognitive Engine: Motion Classifier
– Slaves run in separate VMs
• Feature extraction
10/24/2014
• Classification
– A subset of classes for
each slave
• Waving
• Clapping
• …
33
![Page 34: Gabriel: Towards Wearable Cognitive Assistancezhuoc/papers/gabriel_MobiSys2014_slides.pdf · Gabriel uses two-level token-based flow control Cognitive Engines require different FPS](https://reader034.fdocuments.us/reader034/viewer/2022051607/60392925881343285c73dd78/html5/thumbnails/34.jpg)
Scale-Out of Cognitive Engines
# of slaves 1 2 4
Frame Rate / fps 9.8 15.9 19.0
10/24/2014 34