Ted Selker - Singularity University - 6 July 2010
-
Upload
singularity-university -
Category
Documents
-
view
713 -
download
0
description
Transcript of Ted Selker - Singularity University - 6 July 2010
![Page 1: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/1.jpg)
User System Ergonomic ResearchIBM Almaden Research Center,
Stanford University
Physical,Graphical,
& Cognitive
Human Computer Interface
MITMedia Lab
Context Aware Group1999 - 2008
ALUReg
D
M
TLB
CMU Silicon Valley
Business mentoring
The Generator Fund
TED SELKER
![Page 2: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/2.jpg)
HCI Field• Input techniques
– Direct Manipulation: Select, steer, multidimensional,, orientation, location,
– Audio: sound, voice
– Camera: features, figures, face…
• Output techniques– Screen: text, GUI, Ecological, abstract
– Audio
– Physical; paper, tangible, robot
• User modeling– Task, User, System
• Typical scenarios– Office, data, teaching, collaboration, control
• Evaluation– Demonstration,, Systematic (Preference, Performance), Ethnographic, …
![Page 3: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/3.jpg)
Finger Control Everywhere!•In Keyboard Pointing
•Remote Controls
•Air Traffic Control
•Surgical Instruments
•Sightless Pointing
•Prosthetics pointing
•Steering
•Two Handed
•Industrial Controllers
•Arcade Games
•In Mouse Scrolled
•Multi-touch the endgame?
TrackPoint
![Page 4: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/4.jpg)
T1 T2 T3 T4
Day 1
0
2.5
5
7.5
10
12.5
15
17.5
20
Sec.Graphical interface mappings
-20 -40 -60 -80 -1000
2
4
6
8
10
12
14
16
Free
Restrained
0
2
4
6
8
10
12
14
16
No Mask
Unmasked
Masked
Selction Performance: Masked (top) and Unmasked (bottom)
![Page 5: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/5.jpg)
Towards a Behavioral Motor Match
• Placement
• Eye Tracking
• Wiggly Fingers
• Going Fast
• Movement Feedback
• Gripiness
• Mouse Lockout
Select and Type
Mouse WheelMouseJSMouse 2hand
-3
-2
-1
0
1
2
3great
very good
good
OK
poor
very poor
terrible Sensors:2d 3d, linear, eye motion…
![Page 6: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/6.jpg)
HCI field: things to learnCard & Moran, …,ACM CHI, Interact, IUI
• Cognitive science– Social, behavioral, cognitive, perception
• Statistics
• Design– Product design, interaction design– ME, EE, Materials
• Computer Science
![Page 7: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/7.jpg)
MobileProductivity tools
![Page 8: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/8.jpg)
MobileCommunication tools
![Page 9: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/9.jpg)
Mobile Content tools
![Page 10: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/10.jpg)
Mobile Personal management
![Page 11: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/11.jpg)
Ted SelkerAssociate Director, Cylab Mobility Research
• NSF FODAVA with Ole Mengshoel– Multifocus multi view network visualization: Michele Cossalter
• MS RA project MobileEssenceII– ACMCHI, CTS2010, HotMobile2010 Shoou-Jong Yu
• MS RA project Remember and Compare– Sona Chaudhuri ,Russel Reed
• Tremor as a metric of health– Will Dayton, Patricia Collins
• Creative Input …
![Page 12: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/12.jpg)
Understanding Considerate Systems UCSTed Selker, Carnegie Mellon University Silicon Valley
![Page 13: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/13.jpg)
Context Aware Computing groupTed Selker, Associate Professor MIT Media lab
Using Sensors and Virtual sensorsTo understand and respect human intention
![Page 14: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/14.jpg)
Context Aware PlatformsDemonstrations and design tools for recognizing and respecting intention across domains and scenariosTed Selker [email protected]
1. Annotated smart kitchen Recognizing and coaching activity in kitchen2. Alzheimer's Living Center Adaptive interface to support communication for people with dementia3. Attention vending machine Socially interpreted interactive media in a vending machine4. Audio interface tool Tool for exploring audio interface for voting and telephones5. Attention meter Camera input for interactive6. Bike alert Auto system for telling bike car door might open7. Car coach Feedback to improve driving8. Climbing interface Platform for museum interactive9. Context builder Context aware application engine10. Considerate thing Sensor/ electronics platform for making considerate things 11. Chameleon tables Computer interactive tables with height control and sensing 12. Digital cigarette Motivation and relationship building demonstration13. Disruption manager Semantic based system to mediate desktop interactions 14. Driftcatcher Socially aware email annotation interactive15. Dishmaker Recycling kitchen manufacturing16. e-Bed Eye gesture based GUI17. e-Clay 24 degree of freedom design input tool18. e-Floor Socially aware floor interactive19. e-Helmet Wearable for mediating communication for bicyclist20. e-Threshold Context aware receptionist 21. EyeaRe Glasses that can tell interest alertness22. Exercar Pedaling accelerator improves acuity and reduces fatigue23. Face Interface Caricature teaching of facial gesture interface affordances24. Gesture music ball Instrument builds itself for you25. Haptic tuner Adaptive feedback to coordinate and improve peoples musical collaboration26. Interruption Manager GUI mediation for any platform 27. Kitchen phone System to control and enhance grocery experiences28. Invision Eye gesture based preference system29. Low Error Voting Interface New approach to ballot design for improved access and accuracy30. Media jukebox Model and feedback smart interactive for selection of media31. Media windshield Augmented reality car 32. Minerva Viewing food to suggest recipes33. Mobile essence Meeting collaboration support system34. MrWeb, Collaborative web tools35. PlaceMap Location aware engine36. Power bra Power harvesting system collects 1 watt from breathing37. Secure Architecture For Voting Electronically New approach to reliability/security 38. SMS mediator No cell phone software meeting mediator39. Smart dice Progressive math teaching interactive40. Smart spoon Sensors for teaching cooking41. Smart sink Sensing use of sink for ease and safety42. Smart refrigerator Energy reducing, food improving preservation43. Smart utensils Sensing to teach and coach food preparation44. Talking Trivet, e-sleeve, shoulder pet One sensor systems that react to intention45. Thought for food Systems to use common sense and feedback to interact with food and recipes46. USPS concept truck Annotation, location awareness support for commercial delivery, (8 demos)47. Voyager Location aware system for interactive campus, grocery store, etc.48. Wireless dashboard Harvested power for sensing and control
New access
New scenario
![Page 15: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/15.jpg)
Intention Matters
• Recognize intention– Sensors that know what they are looking for– Static and dynamic models
• of user, system and task
• Respect intention– Human Social goals come first– Human attention & flow– Human task is central – Solicitors beware
• Act on intention – When it supports flow
…And Social Response Essential…
![Page 16: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/16.jpg)
ALZ:Productively Challenged,Critically sophisticate
• Room • Places• Puzzle• Arranging flowers• Art• Radio• Catalog
![Page 17: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/17.jpg)
Can Computers read our minds?
• Language based directive?
• Intentions, Reaction,
• Possibility…
8secdoor.wmv
![Page 18: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/18.jpg)
What you do says so much …
• Where you are tells a story– Floor imagery
comments on what people are doing
• How you react to what you are doing– Laughcam listens to
camera person
• Watching your mouse
![Page 19: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/19.jpg)
Where We Use Computers• Wishes from Eye Gestures
• Closed: going to sleep
• Open: no alarm
• Blinks: doesn’t like something
• Wink: selection
• Stare: interest
• Gaze: thinking
This bed, invented by an MIT scientist, has many high-tech features. (ABCNEWS.com)
Sleep With Your Computer?
MIT Researcher Computerizes Life
April 10 — Imagine a bed that gently wakes you up at the crack of dawn and reminds you of your upcoming appointments.
![Page 20: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/20.jpg)
Mind full of intentionsLearning by watching
• I look around– Interest Tracker,
Invision
• Eye aRe Personal gaze– Looking for a sign?
- Robot seeks work as fuel tank inspector
![Page 21: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/21.jpg)
Attending visually (Yarbus 1967).
1) Free examination. Before subsequent recordings, the subject was asked to:2) estimate the material circumstances of the family; 3) give the ages of the people;4) surmise what the family had been doing before the arrival of the "unexpected visitor;" 5) remember the clothes worn by the people; 6) remember the position of the people and -objects in the room;7) estimate how long the "unexpected visitor" had been away from the family
Learning by watching
![Page 22: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/22.jpg)
Invision Your gaze organizes things
![Page 23: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/23.jpg)
LaughCam
…Index video with laughter and skin conductivity
A Lockerd, F.Mueller
Reaction based interface:– Your reactions edit your video
![Page 24: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/24.jpg)
AI Can help Human/Human Communication
• Empathy Buddy- Hugo Liu
• Driftcatcher- Andrea Lockerd
![Page 25: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/25.jpg)
Car COACH (Cognitive Adaptive Computer Help)
• When to comment on driving?– When driver is open to comment– When driver is doing a good job – When driver is making mistakes:
• Driving smoothly• Signaling turns, lane changes• Gas, break, steering • Honking• Gas mileage• Visually checking instruments • Scanning scene• Checking blind spots• Finding controls easily
-
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
Accel. Successes
Immediate
Delay
![Page 26: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/26.jpg)
Face InterfaceTeaching face interaction from experience
• Facial gestures – Caricaturized explanation
– Create keystroke input
– Vending/interaction scenarios
![Page 27: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/27.jpg)
IM Disruption ManagerErnesto Arroyo, Shawn Sullivan
• User Activity– Implicit mouse behavior
– User schemas about IM
• Task-oriented strategies• Task type
• Task stage
• Goal-oriented strategies • Quantity, Quality, Time, …
Implicit Behavior Detection
Interruption Disruption Model
User Satisfaction
Document Relevance
User Interest
User Availability to Interruption
Examine Retain Reference Annotate
Scrolling Reading Interacting Thinking Hesitating ?? ??
Task & Goal
??
![Page 28: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/28.jpg)
•25% or more increased performance with disruption manager•People prefered it•Worked as well in daily work (10 day exeriment)
![Page 29: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/29.jpg)
Autonomous Considerate things Augmenting existing things
• Materials Chameleon mug– Look like a mug when coffee – Look like a tumber when ice tea
• Electronic s Left-overs timer– Count days in fridge
• Smarts … Talking Trivet…– Needs re-warming– Hot and ready to eat– Cooked and ready to take out– Fire!
![Page 30: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/30.jpg)
Progressive relationships…diceCapacitive 3 d sensor teaches-Patterns,-numbers, -counting, -arithmetic
![Page 31: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/31.jpg)
Smart sink
• Temperature colored water – Warm for hands,– hot for filling– Cold for washing vegetables,
• Height appropriate
![Page 32: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/32.jpg)
A Spoon that teaches cooking
• Better food
• Teaching cooking
![Page 33: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/33.jpg)
Digital cigaretteWin Burleson
• Comforting or Calming device• Set pace and timing for activity• Break ice• Something for me• Something to give• Limit distractions• Entertainment • Give me
– Courage, stamina, patience• Define me …
– outsider, insider, dangerous, grown up, courageous, available
http://web.media.mit.edu/~win/Digital%20Cigarette.htm
![Page 34: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/34.jpg)
Bicycling August 2004
• Loud outside -amplify noise• Tip head -blinkers• Talk -telephone• Yell -horn• Quiet -music• Shake head -”pothole”
GPS and description
Mediating communication: a person and their environment
![Page 35: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/35.jpg)
Postal Truck
Business :GPS/ 802.11Delivering EverythingTargeted marketingEnlist for traffic controlLocation & time sensitive services
SafetyCameras Surround vehicleSonic, vibration and LED indicatorsLear anti Whiplash seatAutomatic emergency break…
![Page 36: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/36.jpg)
Physical virtual world interface
• Cart Companion or phone – Teach about products
• Round roast: $1.99 Lean meat, Gourmet whenslow cooked in wine and tarragon
– Buy products• Meat counter• Infinite Aisle assists purchases
– Crock Pot for roasts…
• Watch kids– Camera in arcade– Controls Xbox
![Page 37: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/37.jpg)
Chandelier Video Projector
![Page 38: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/38.jpg)
Augment and Integrate with annotation
![Page 39: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/39.jpg)
Forms and their implicationsKeyboard size and implicationsHinge and implications
• Keyboard size and implications• Hinge implications• Durability, maintainability• look
![Page 40: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/40.jpg)
HCI ?
• Office, data, teaching, collaboration, control…
• Instruments, Home, Mobile, Auto, kitchen, industrial,…
• Assistive Agents
• Advisory Agents
• Persuasive interfaces
• Energy monitoring, control,
• Disaster: photos, communication, coordination, , planning,
• Activist tools: JIT organizing, reflective art,
![Page 41: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/41.jpg)
Product Design?
• Art: creating ideas to create an emotional reaction
• Science: creating ideas that other people cite (cause other people to create ideas)
• Design: is creating things of value
![Page 42: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/42.jpg)
Invention is a performance Personal credo
• Inventing is like an action movie– Grab for a branch, it brakes, land in the water and swim… – The way it doesn’t moves you forward
• Audition Stories and roles• Think of many possible ways out
• keep thinking about it
• All the world is a stage.– Slowly replace pantomime for real sets and actors
• Have things/ideas compete not people• Pretend things are the way you want them• Create it from anything
– All tools are made of other tools– Prototype here and now
![Page 43: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/43.jpg)
Technology will help Expectations
• Coordination ? • Planning ?• more control over Shared -VS- personal• more control over Voyeur -VS- participant
![Page 44: Ted Selker - Singularity University - 6 July 2010](https://reader038.fdocuments.us/reader038/viewer/2022102823/5495930fb47959a7508b47ce/html5/thumbnails/44.jpg)
Future Feel of Tools•We have been using computers for cataloging
•Context Based computing•Information is becoming Personal and Personalized •Technology is inevitably used for Socialization
•Computers will be aware
•reduce transcription
•Computers will understand…
•Teach, augment, proxy