Context Fabric: Infrastructure Support for Context-Aware Systems Jason I. Hong Qualifying Exam...

Context Fabric: Infrastructure Support for Context-Aware Systems

Jason I. HongQualifying Exam Proposal

Mar 20 2001 2

Why Context-Awareness?

Context TypesExisting Examples Human Concern

Room ActivitySmoke Alarm Safety

Room ActivityAuto Lights On / Off Convenience

Object IdentityBarcode Scanners Efficiency

Personal Identity & Time

File Systems Finding Info

TimeCalendar Reminders Memory

Mar 20 2001 3

Technology Trends

• Sensors GPS, Active Badges, Active Bats Smart Dust Cameras and microphones

• Recognition algorithms MSR Radar location from 802.11 Smart Floor footstep force (Orr & Abowd 2000)

• Wireless technologies Bluetooth, 802.11, cell phone

Mar 20 2001 4

Existing Examples


Context Types Potential Examples Human Concern

Activity Safety

Activity Convenience

Identity Efficiency

Identity & Time Finding Info

Time Memory

Identity

Time

Location

Proximity

Activity

History

…

Smoke Alarm

Auto Lights On / Off

Barcode Scanners

File Systems

Calendar Reminders

Health Alert

Auto Cell Phone Off In Meetings

Service FleetDispatching

Tag Photos

Proximal Reminders

Mar 20 2001 5

A New Class of Context-Aware Apps

Active Badge(Olivetti)

ParcTabs(Xerox PARC)

Cyberguide(Abowd et al)

Mar 20 2001 6

Problems with State of the Art

• Coordinating all the context Will have many distributed, heterogeneous

sensors, services, and devices out there Won't be designed a priori for interoperability

• Building robust context-aware apps Dynamically changing computing environment Sensors, services, and devices will vary in

capability and availability

• Developers are forced to devote a lot of time and energy to non-application issues

Mar 20 2001 7

Proposed Meta-Solution

• Shift burden of context-awareness from individual devices onto infrastructure Discovering, Acquiring, Transforming, Refining,

Routing, and Reacting to Context

• Provide services that can be easily deployed everywhere and used by any device Have simple, uniform system interfaces Any device with a network connection can

discover and use these services

• Abstract out messy details of sensors and services

Mar 20 2001 8

Thesis Statement

A set of core infrastructure services coupled with a specification language for stating context needs will reduce the effort in developing robust context-aware applications.

Mar 20 2001 9

Sensor Management Service

Services

Proposed Solution

ContextEvent

Service

ContextQuery

Service

Automatic Path Creation

AppApplication

Layer

ContextLayer

Logical SensorLayer

Physical SensorLayer

ContextSpecification

Language

Agents

Device Agents

User Agents

Mar 20 2001 10

Research Contributions

• A way of explicitly specifying context needs independently of specific sensors and services (+ ontology for doing it)

• Automatically assembling a dataflow from heterogeneous sensors and services at run time to satisfy context needs

• Seeing how far queries, events, and data flow processing can go for context

Mar 20 2001 11

High-Level Plan (Mir Space Station View)

• Design Context Specification Language• Design, implement, and deploy core

context services• Evaluate by implementing several simple-

to-medium complexity apps on top• Re-iterate design and implementation

based on what was learned

Mar 20 2001 12

Outline

Background and MotivationProblem and Proposed SolutionContext Fabric ArchitectureEvaluationRelated WorkProposed ScheduleSummary

Mar 20 2001 13

Context Specification Language

<context version=1.0 type=subscribe><requestor>

[requestor identification][requestor authentication][requestor input]

</requestor><request>

[context need]</request>

</context>

Mar 20 2001 14


• Need to express context about and relationships between People, Places, Things

• Predicates Identity (Who is…? What is…? Is with…?) Location (Near? Nearest? Distance? Path?) Activity (Is busy? Is in meeting? Current task?) Time (In past? In present? In future? On date?)

• Some of this vocabulary done by Schilit Implicitly encoded in his APIs One goal is to extend his work in spec language Another is to make it extensible for future

context types

Mar 20 2001 15


• Common parameters Max number of results wanted Return name Return data type (e.g. String, List, Table) Minimal probability of correctness desired Relevant sensor input requestor has

• Event parameters Event rate (e.g. at most 1 event per second) Event callback (e.g. RPC, socket port) Max number of events desired Granularity of change (e.g. 1 meter)

Mar 20 2001 16

Context Event ServiceContext Query Service

• Uniform abstraction for apps Pass a Context Specification to Context Event

or Context Query service Everything is handled by infrastructure

• Event and queries two sides of same coin Query is synchronous and one-time Event is asynchronous and repeating

• Bulk of work is done by Automatic Path Creation though

Mar 20 2001 17

ZIP Weather

GPS ZIP

Cell GPS

Cell ZIP


• A way of assembling small services into larger, more interesting ones

• Operators

• PathsZIP TheatersGPS ZIP

ZIP WeatherGPS ZIP

Mar 20 2001 18


Sensor Input ZIP Theaters

GPS

GPS ZIP

What are the nearby movie theaters?


Cell

GPS ZIPCell GPS


Cell

Context OutputCell ZIPMovie

Theaters

• Automatically assemble path based on resources• Don't have to know specific sensors and services• Easier to add new ad hoc functionality• Will need a Service Description Language

Mar 20 2001 19

• Re-implement from ParcTabs (Schilit 1995)• Device Agents

– Bridge for physical devices into infrastructure– Holds context state about devices– Centralizes context to and from device (better for

preferences and filtering)– Job is to handle all context issues that

cannot / should not be handled on device itself

• User Agents– Holds context state about people– Preferences, location info, filters, etc

Agents

Mar 20 2001 20

Context Fabric ExampleContext-Aware Tour Guide

• Mobile PDA tour guide1. Display map of local area2. Display nearby points of interest3. Bring up info on proximity4. Display location of others5. Capture history

Mar 20 2001 21


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

PalmPilot

CellPhone

Infrastructure

Mar 20 2001 22


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

1. Display map of local area1. Turn on device, local GPS registerswith Sensor Management Service


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 23


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

1. Display map of local area2. Start up tour guide application


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 24


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

1. Display map of local area3. Setup context query for changes in location (granularity depends on if indoors or outdoors)


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 25


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

1. Display map of local area4. Receive location updates


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 26


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

1. Display map of local area5. Display location updates


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 27


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

2. Display nearby points of interest1. Periodically query for nearby points of interest and Cache results (e.g. restaurants, landmarks, etc)


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 28


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

2. Display nearby points of interest2. Local Context Query Servicedelegates to Automatic Path Creation Service


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 29


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

2. Display nearby points of interest3. Automatic Path Creation Service takes GPS info andtries to answer it locally (fails).


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 30


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

2. Display nearby points of interest4. Local APC delegates to infrastructure APC


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 31


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

2. Display nearby points of interest5. Infrastructure APC looksfor services that can providelocation-based points of interest info or parts of this info


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 32


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

2. Display nearby points of interest6. Infrastructure APC returnsrelevant information (or "failure due to x" if it cannot)


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 33


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

3. Bring up info on proximity1. Setup proximity events on points of interest. Attach notes to eventthat display information about point of interest.


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 34


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

4. Display location of others1. Subscribe to location changeevents from others.

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q


Service Discovery

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 35


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

4. Display location of others2. Can't process locally, subscription request forwarded to mirroring Device Agent.


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 36


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

4. Display location of others3. Device Agent makes individualsubscription requests.


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 37


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

4. Display location of others4. Individual device agents makea subscription request to theirmirroring devices.

PalmPilot


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

E

Q

CellPhone

E

Q

Mar 20 2001 38


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

4. Display location of others5. Location change events are routed back to tour guide and displayed.


Service Discovery

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q

Mar 20 2001 39


Palm Pilot

ContextEvent

Service

ContextQuery

Service

GPS


Sensor Mgt Service

Tour GuideApp

MapDatabase

5. Capture history1. Just store all interesting events locally.

DeviceAgent

DeviceAgent

DeviceAgent

E

Q

E

Q

E Q

PalmPilot

E

Q

CellPhone

E

Q


Service Discovery

Mar 20 2001 40

Outline

Background and MotivationProblem and Proposed SolutionContext Fabric ArchitectureEvaluationRelated WorkProposed ScheduleSummary

Mar 20 2001 41

Evaluation

• Iterative design process Work out designs, deploy, build apps, refine

• Initial evaluation apps will be existing ones Reimplement some existing context-aware apps Informally evaluate apps for robustness Informally evaluate apps for defaults (e.g.

privacy policy)

• Continued evaluation will be novel apps Focus on robustness and defaults again Also get others to build context-aware apps

Mar 20 2001 42

Related Work

• ParcTab System (Schilit 1995) Device Agents, User Agents, Active Maps Plan to re-implement these and new ones too

• Context Toolkit (Dey 2000) Widgets, Interpreters, Aggregators Much is subsumed in Automatic Path Creation

• Interactive Workspaces (Fox et al 2000) Paths path creation and blackboard architecture Self-describing and extensible events

Mar 20 2001 43

How Context Fabric Differs

• Separate what context is needed from how it is processed via Context Spec Language

• Shift processing from static to dynamic Say what you need at design-time Figure out how to get it at run-time through


Mar 20 2001 44

Proposed Schedule

• Phase 1 (0-8 months / PARC) Initial design and implementation of Context

Specification Language and core services Evaluate with simple context-aware apps

• Phase 2 (8-12 months) Re-iterate and incorporate what was learned Release source code Evaluate with medium complexity apps

• Phase 3 (12-18 months) Incorporate more feedback (smaller changes) Write dissertation

Mar 20 2001 45

Exit Conditions and Deliverables

• Spec for Context Specification Language Basic ontologies for context Expressive enough for majority of context apps

• Working core services Context Query, Context Event, Automatic Path

Creation, plus basic operators Service Description Language for APC operators

• Source code available for general use

Mar 20 2001 46

Summary

• Infrastructure approach to context-awareness

• Context Specification Language for stating context needs

• Context Event Service, Context Query Service as primary APIs

• Automatic Path Creation for processing• Evaluation focused on defaults and

robustness

Mar 20 2001 47

A New Cal Record!!

• Only one day to schedule quals…

• …through email…

• …with a professor at Stanford even!

Mar 20 2001 48

Extra Slides

Mar 20 2001 49

What is Context?

• Situated Action (Suchman) Responsiveness, improvisation to situation Actions are moment-by-moment, highly fluid on

situation

• Activity Theory (Nardi) Subject, Object[ive], Operations, Artifacts, Env Activity defines context but it changes as list

above changes Context depends on sensing and inferring, but

difficult to know people or goals

Mar 20 2001 50

What is Context?

• Provides fair criticisms on existing context-aware apps Response: Focus on the more routine and

predictable actions Response: Focus on simple first-order approx Examples: Smoke alarms, Calendar reminders

Response: Is occasionally wrong, but won't get better unless we do more research here

Response: Infrastructure enables more research in this area

Mar 20 2001 51

What is Context?

• Merging of virtual with physical and social Mostly implicit input, often secondary to

task at hand

• People / Places / Things / Virtual / Time People: Location, Identity, Task, Affect Places: Activity, People, Temperature, Audio Things: Location, Identity Virtual: Services, Bandwidth Time: All of these Past, Present, and Future

Mar 20 2001 52

What is Context?

• But it's a rathole to try to define it… Workflows? (Greenberg 2001) Intent and Desire? Natural Language Processing? What isn't context?

• Instead, operationalize it Distributed, multi-device, sensor-based Fairly well-defined and computable concepts

(e.g. location and identity, but not intent or workflow)

Mar 20 2001 53


• Sensors are coming• Increase channels of input / output

GraphicsSound

MouseKeyboard

Mar 20 2001 54

Why Context-Awareness?Skeptics toward the ubiquitous computing movement quickly point out that those involved in promoting Mark Weiser's initial vision are mainly interested in pushing technology…

As a reaction against this fair criticism, many researchers spend countless hours trying to dream up the "killer application" that will cause everyone to adopt ubiquitous computing technology.

I think this is the wrong goal to have in mind as a researcher for two reasons. First, we are not schooled in the practices to judge a market…

Second, my interpretation of Weiser's vision is not that there would be a single, most compelling use of ubicomp technology, but rather that the totality of the experience would cause a significant paradigm shift in the way we as humans perceive our relationship to interactive computing.

In short, Weiser argued for what I would like to call a "killer existence," in which many disparate devices would provide an array of services in ways that seamlessly complement our lifestyles.

-- Gregory Abowd, 2001

Mar 20 2001 55


Sensors Computers

Context-Aware Computing

Physical World Virtual World

Mar 20 2001 56

Why Infrastructure Approach?

• Sharing of data Collaborative filtering, large data sets

• Sharing of processing power Heavy-duty processing on dedicated machines

• Sharing of sensors Sensors in the environment

• In-place evolution of services New algorithms, new features

• Supports heterogeneity Sensors, Devices, OS, Programming Langs

Mar 20 2001 57

Why Infrastructure Approach?

• Building and evaluating now can Provide more design space for interaction Let us rapidly prototype and evaluate these apps Find problems now Develop the right mechanisms Evolve the right policies

Mar 20 2001 58

Other Related Work

• Context Toolkit Widgets, Interpreters, Aggregators

• ParcTab system User agents, Device agents, Active Maps

• Interactive Workspaces Event heap, Path Creation

• MUSE Jini services, Bayesian reasoning

Mar 20 2001 59

Some Sensors

• Temperature• Pressure• Humidity• Soil makeup• Acceleration• Noise• Material stress

• Biometric• Motion• Distance• Touch• Location• Orientation• Light

Mar 20 2001 60

Related WorkContext Toolkit

• Widgets, Interpreters, Aggregators• Hardwired data flow

Face Recognition

LocationWidget

LocationWidget

In/Out Board

Smart Card Reader

ID to NameInterpreter

BuildingAggregator

Mar 20 2001 61

Related WorkParcTab System

• Will need to re-implement parts of ParcTab system Device Agents / User Agents / Active Maps

Mar 20 2001 62

Related WorkInteractive Workspaces

• Event Heap Scaling of tuple spaces will be a problem Can use how events are described

Mar 20 2001 63

Related WorkMUSE

• Sensor management issues Sensor discovery (currently via Jini) Power consumption

• Probabilistically modeling sensor data Sensor fusion (via Bayesian nets and HMMs)

Mar 20 2001 64

Rough Taxonomy ofContext-Aware Apps

• Triggers• Metadata Tagging• Reconfiguration and Streamlining• Input specification• Presentation

Mar 20 2001 65


• Triggers On X do Y "Notify doctor and nearby ambulances if

serious health problem detected" (Citris)

"Remind me to talk to Chris about user studies next time I see him" (Proxy Lady)

• Metadata Tagging "Where was this picture taken?" "Find all notes taken while Mae was talking" Memory prosthesis (Forget-me-not)

Mar 20 2001 66


• Reconfiguration and Streamlining Telephone forwarding and Teleport (Active Badges)

Turn off cell phone in theaters Automatically adjust brightness / volume Automatic file pre-caching (OceanStore)

Select modes in multimodal interaction Multimedia/Bandwidth adaptation (Odyssey)

Mar 20 2001 67


• Input specification Send mail to people in building (Geocasting)

Print to nearest printer (ParcTabs)

"Find gas stations nearest me"

• Presentation Current location (Cyberguide)

Idle? (Yahoo Messenger, finger)

Currently in? (In / Out board)

Contextual info about objects (CoolTown)

Proximate selection (ParcTab)

Mar 20 2001 68

Context-Aware Mechanisms

• Events Something interesting has just happened e.g. "Larry has entered room 306"

• Queries Introspect current context state (discrete) e.g. "Where is Larry now?"

• Filters Modify events and queries based on

preferences and privacy e.g. "Only show others if in office or not"

Mar 20 2001 69

Are Queries and Events Enough?

Triggers

Metadata Tagging

Reconfiguration

Input Specification

Presentation

Query

Query for relevantcontext state

Query current context state before and while running

Query current context stateto fill in the blanks

Query current statefor more information

Event

On Event do Action

On Event do Reconfigure

On Event do Update Presentation

Mar 20 2001 70

What I Will Not Be Working On

• Building a new service infrastructure• Developing new sensors• Extremely sophisticated forms of context

Natural language processing User intention

• New algorithms for… Sensor / context fusion Machine learning Power consumption Just use algorithms that already exist!

Mar 20 2001 71

What I Will Not Be Working On

• Sophisticated preferences Enable us to explore this space more

• Comprehensive security and privacy mechanisms Will implement basic preferences, filters, and

access control One subgoal is to get the defaults right

• Rigorous user evaluations

Context Fabric: Infrastructure Support for Context-Aware Systems Jason I. Hong Qualifying Exam...

Documents

Transcript of Context Fabric: Infrastructure Support for Context-Aware Systems Jason I. Hong Qualifying Exam...