Tools for Ubiquitous Computing Research
Transcript of Tools for Ubiquitous Computing Research
![Page 1: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/1.jpg)
Tools for Ubiquitous Computing Research
Emmanuel Munguia Tapia, Stephen Intille, Kent Larson, Jennifer Beaudin, Pallavi Kaushik, Jason
Nawyn, Randy Rockinson
House_nMassachusetts Institute of Technology
![Page 2: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/2.jpg)
Outline
• Motivation: Health care crisis• House_n research agenda• House_n Tools for studying behavior
– The PlaceLab living laboratory– Portable kit of sensors (MITes)
![Page 3: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/3.jpg)
Some statistics
From the 2000 census of population• Nearly 1 in 5 U.S. residents suffer some
kind of disability• Approximately 40% of people 65 and
older have a disability• Over 20% require continuous
monitoring and help performing activities of daily living (ADLs)
![Page 4: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/4.jpg)
Some statistics
From the 2000 census of population• In 2030, nearly one out of two
households will include someone who needs help performing basic ADLs
![Page 5: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/5.jpg)
House_n research agenda
Goals• Increase the time that people
remain healthy, independent and safe in the comfort of their homes.
• Enable novel context-sensitive applications to be built and piloted
![Page 6: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/6.jpg)
Health care interventions
Medical staff
Changes in behavior
(dementia/independence)
Encouraging healthy behavior
![Page 7: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/7.jpg)
Current work: proactive health• Switch/bend sensors
–Doors–Cabinets–Drawers–Thresholds–Appliances–Objects
• Wearable sensors–Accelerometers –Heart rate monitor–Self report
• Multi-purpose sensors–People-locator tags–Auditory sensors–Optical sensors
Activity recognitionEating mealsTalkingSleeping patternsTaking medicationsCleaningCooking…
newML algorithms
Detect change in activity;Motivate behavior changes
healthapplications
![Page 8: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/8.jpg)
House_n tools: The PlaceLab Living laboratory
![Page 9: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/9.jpg)
PlaceLab
• Is not to show off “new technology”• It is a residential observational research facility
Goals– Run different research studies– Real people living at PlaceLab 24/7 (weeks/months)– Collect necessary data for doing research
Design constraints– Reliable sensing infrastructure– Add/remove sensors on the fly (Modular)
![Page 10: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/10.jpg)
Why another live-in laboratory?Our design benefited from lessons learned bythose who created prior “living labs”:
• Georgia Tech Aware Home (Abowd, Mynatt, and others)
• UVA’s Smart Home Monitor (Alwan)
• Smart House (Matsouoka)
• Welfare Techno House (Suzuki)
• Philips HomeLab • Sleep laboratories • Others…
![Page 11: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/11.jpg)
The PlaceLab combines these uniquecharacteristics:
• A unified, extensible, multi-modal, and truly ubiquitous sensor and observational infrastructure
• Designed for shared data generation/distribution and collaboration
• Sensors integrated into architectural aesthetic
• Genuinely live-in
Why another live-in laboratory?
![Page 12: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/12.jpg)
Goal: Context-aware technologies at home
Three key challenges (among others):
1. Need for complex, naturalistic environmentsSimulated behavior is overly simplistic
2. Need for comprehensive sensingActivity occurs throughout environment; realisticdatasets costly to obtain; head-to-head comparisions
3. Need for labeled training datasetsMany context-recognition algorithms need labeledexample data; annotation required for evaluation
![Page 13: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/13.jpg)
Laboratoryprototyping
Larger n,in-homestudies
Innovativedesign
ideas
Ethnographic/HCIresearch
Testing ubicomp technology in the home
![Page 14: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/14.jpg)
The PlaceLab: filling a gap
Laboratoryprototyping
Pilot DataDesign insight
Important questionsThe PlaceLab
Larger n,in-homestudies
Innovativedesign
ideas
Ethnographic/HCIresearch
![Page 15: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/15.jpg)
PlaceLab can complement…
• Surveys and interviews• Experience sampling• Direct observation• Portable kits of sensors for in-home
studies• Demonstration labs• Short tests in parts of live-in labs;
tests with limited sets of sensors
![Page 16: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/16.jpg)
The PlaceLab Infrastructure
![Page 17: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/17.jpg)
![Page 18: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/18.jpg)
PlaceLab
![Page 19: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/19.jpg)
Interior entrance
![Page 20: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/20.jpg)
Living room
![Page 21: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/21.jpg)
Living room
• Most visible technology a standard TV
![Page 22: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/22.jpg)
Living room
![Page 23: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/23.jpg)
Kitchen
• Apartment allowsthe study of natural home behavior
• Interested in complex behaviorsuch as – Decision making– Interruptions– Searching– Communication
![Page 24: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/24.jpg)
Sensor integration
• Sensors blend into aesthetics of environment(so easy to ignore)
![Page 25: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/25.jpg)
Kitchen
![Page 26: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/26.jpg)
Office
![Page 27: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/27.jpg)
Bedroom
![Page 28: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/28.jpg)
Master bath
![Page 29: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/29.jpg)
Microcontroller; connections server closetSpeakers
Optional: CO2 sensorOptional barometric pressure sensor
Humidity sensorTemperature sensor
IR video cameraColor video camera
Top-down counter cameraLight sensor
IR illuminators
Microphone
Switches to detect “open/close”
Temperature sensor
Subwoofer
Embeddedsensors
Emphasison ubiquityand quantityover quality
Reasonablelocations
![Page 30: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/30.jpg)
Easy access to sensor infrastructure
![Page 31: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/31.jpg)
Wireless object movement MITes
• Real-time, wireless transmission
• Receivers scattered throughout apartment
• 100-200 sensorsdepending on task
![Page 32: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/32.jpg)
MITes sensors installation
Single point of contact, no multi-point alignment is required
![Page 33: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/33.jpg)
Object usage MITes
![Page 34: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/34.jpg)
Activity recognition from sensors in the environment
86%Doing laundry
36%Preparing a beverage
89%Grooming
64%Dressing
87%Bathing
45%Preparing breakfast
71%Toileting
59%Preparing lunch
Activity detected at least once criteria
![Page 35: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/35.jpg)
Example sensor data
![Page 36: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/36.jpg)
RFID reader wristband
Measures: RFID tagged objects + wrist accelerationRange: 10cm Cost: ≈ $181 US
Based on Intel Research Seattle RFID glove (Perkowitz ETAL ‘04)And in collaboration with Ambient Intelligence MIT Media Lab
Determine motion when holding an object
![Page 37: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/37.jpg)
Wireless limb accelerometers and HR
• Real-time, wireless transmission
• Receivers scattered throughout apartment
• Up to 5 locations• HR monitor (Polar chest strap)
HR
3-axis accelerometers20-30Hz
![Page 38: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/38.jpg)
Activity recognition from wearable sensors
Activity recognition from wearable accelerometers
• 5 points–Right hip–Dominant wrist–Non-dominant upper arm–Dominant ankle–Non-dominant thigh
![Page 39: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/39.jpg)
Recognition results for 20 activities
• Walking 89.7• Sitting & relaxing 94.8• Standing still 95.7• Watching TV 77.3• Running 87.7• Stretching 41.4• Scrubbing 81.1• Folding laundry 95.1• Brushing teeth 85.3• Riding elevator 43.6
• Walking carrying 82.1• Work computer 97.5• Eating/drink 88.7• Reading 91.8• Bicycling 96.3• Strength train 82.5• Vacuuming 96.4• Lying down95.0• Climbing stairs 85.6• Riding escalator 70.6
Using decision trees and leave-one-subject out crossvalidation
![Page 40: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/40.jpg)
Subject self report
• Random or context-awareself-report sampling on phone (activities, mood and other statesof mind, etc.)
• Apps on phone can respond to PlaceLabsensors
• Standard surveys or ethnography can also be used
![Page 41: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/41.jpg)
Ubiquitous sensors
![Page 42: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/42.jpg)
Automatic selection of most informative audio-visual views using motion and camera location heuristics
![Page 43: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/43.jpg)
Control “closet”
![Page 44: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/44.jpg)
Running an experiment
• Recruit participant(s) • Participant(s) move in • Home disconnected from Internet
(data saved to portable disk)• Minimal interaction with researchers during
stay • At end of stay, data is collected and stored • Data annotated for items of interest• Datasets become more valuable as more
researchers annotate them
![Page 45: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/45.jpg)
Recruiting
![Page 46: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/46.jpg)
Take away• The PlaceLab is a live-in residential home laboratory
developed for health and ubiquitous computing research
• Unlike prior facilities, the home has a truly ubiquitous, synchronized, and multi-modal sensor infrastructure built non-obtrusively into the architecture
• The lab can be used as a hypothesis generation and testing facility and can help projects transition from laboratory testing to larger-n, in-home studies with portable sensors
• We are trying to operate the facility as a shared resource
![Page 47: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/47.jpg)
Thank you!
Questions? Contact:
– Emmanuel Munguia Tapia [email protected]
– Stephen [email protected]
– Kent [email protected]
![Page 48: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/48.jpg)
![Page 49: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/49.jpg)
House_n sensing tools:
MITes: MIT environmental sensorsA portable kit of sensors for studying behavior in natural settings
![Page 50: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/50.jpg)
Goal: allow context awarenessActivity Examples
(labels)Sensors
Machine Learning and Pattern recognition
(training/inferencing)
Activity probabilities/detection
orActivity Models
Toileting•Toilet Flush
•Faucet
•Soap
Most sensors are simple and
binary
![Page 51: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/51.jpg)
Avoid microphones and cameras
• avoid usingaudio, visual,electromagnetic orother sensorsplaced in theenvironment
• Why?– Sensors may be perceived as invasive– Succeptible to environmental conditions– Signal interpretation extremely difficult– Difficulty of signal interpretation depends on sensor
placement (increasing installation difficulty)
![Page 52: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/52.jpg)
MITes (MIT Environmental Sensors)
Goal: collect data from hundreds of multi-modal sensors (environmental and wearable) from single receiver in non-laboratory deployments
• Easy of installation• Ease of use • Adequate performance• Affordable for research• Well characterized/tested
![Page 53: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/53.jpg)
MITes sensor kit includes
Six environmental sensors (low bandwidth)
Five wearable sensors (high bandwidth)
current sensingtemperatureobject-usage-detection
Proximitylightmovement
location beaconsheart rate
ultra-violet radiation exposure
RFID reader wristbandonbody acceleration
![Page 54: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/54.jpg)
Proximity MITes (MERL)
![Page 55: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/55.jpg)
Our Hardware sensing Tool #2MITes: Portable toolkit of sensors
Temperature and Light
![Page 56: Tools for Ubiquitous Computing Research](https://reader031.fdocuments.us/reader031/viewer/2022012410/616a56cf11a7b741a3516a1b/html5/thumbnails/56.jpg)
MITes receiver
Single receiverUSB or serialConnector receives all sensor data