Beyond the PCKiosks & Handhelds

Albert HuangLarry Rudolph

Oxygen Research GroupMIT CSAIL

Overview

Computation is useful everywhere We need to explore interaction modes

beyond the desk: Kiosks Handhelds

These are mutually complimentary

OK-Net Kiosk

Kiosk Specifications

Touch screen monitor Small computer inside kiosk Minimal infrastructure Hacker-hardened Nothing exposed

Under the hood

Self contained

Interaction Modes

General Public Similar to web browser: point-and-click Content harvested automatically

seminars, events, directory, news, etc.

Adapt to user extension of user’s digital world extension of user’s mobile devices

Identify User by Doodle

People initially interact with a doodle

Conjecture: doodles are unique like signature

Provides a moderate amount of id strong id not necessary

Other Interaction

s User input

touch & speech phone and PDA as remote finger

Information Transfer SMS email Bluetooth (OBEX push)

Simple Kiosk App:

Stata Guidance

Stata is confusing Kiosk provides several guide modes Passive:

show & push map to Bluetooth-enabled device

Active: guide user along the way

A graph for each floorNodes: junctions or destinations

Given start and destinationcompute a path

Send it all to phone

Real-time navigation Track user (phone) in Stata Trivial deployment

Bluetooth beacon in each PC < $20 per beacon

Indoor GPS for phones scan for Bluetooth beacons map detected beacons to a

location

Real-time navigation

Real-time navigation

Real-time navigation

Real-time navigation

Real-time navigation

Improvements needed

Takes too long to recognize beacons Much better results

with two beacons Signal comes and goes

Incorporate model of human motion

Probabilistic filtering

Some people cannot read maps

A “human-centric” navigation guide without sound without abstraction

A picture is worth 1000 words photo sequence

Digital Assistant

Kiosks can send content to phones But users also generate content

pictures audio memos text memos selections

Human-centric organization

Organize as it happens Associate content for subsequent

retrieval Display associated content together Content can be displayed manually or

automatically

External Triggers

Time (the usual alarms) Location

enter super market, home, car, office

Meeting someone Phone call from someone Many others to be discovered

Testbed:Conference Domain

When attending technical conference Select interesting talks at kiosk Send details to phone

Take notes during a talk camera for photos, videos, printed text microphone for audio notes, annotations keypad, laptop for text notes

Associate content with events

Process content offline

Apply recognition technologies for images OCR, Object, People, Place audio speech transcription video sketch & gesture

Build semantic web to integrate with services

Potential use Automatically generate conference

report

Sharing content

Content tagged as public or private Phone-phone or phone-kiosk interaction

synchronizes public content Content spreads throughout community

each new recipient tags it anew More complicated than public/private

User may want to share content among her own devices, e.g. laptop, phone, iPod, etc.

User may want to share some content with her boyfriend’s devices

Conclusion about Kiosks

Kiosks: new interaction model? are they just glorified web browsers? interaction with hand-held devices proximity provides simple, physical

interface