Location-Aware Computing John Krumm Microsoft Research Redmond, Washington, USA.

Location-Aware Computing

John KrummMicrosoft Research

Redmond, Washington, USA

Thank You

• Toshio Hori

• Takeo Kanade

• Chie Nakamura

Digital Human Research Center

Location

Why Bother to Sense Location?• Find conference room near me• Who is in this meeting with me?• Where are the people who are supposed to be here?• If I’m in conference room, don’t allow cell phone, alerts, or IM• How long will it take to get from here to next appointment?• Route planning in strange buildings

No Really, Why Bother?• Remind me when I’m near a certain customer• Where are my kids, buddies, colleagues?• Index my documents (email, photos) by location• Change default printer and network settings based on location• Electronic graffiti, e.g. “There’s a better Thai restaurant one block north.”

Why Not Use GPS?

• Does not work indoors• Needs view of satellites

Location Sensing

Hazas, Scott, Krumm, “Location-Aware Computing”, IEEE Computer Magazine, February 2004.

Rosum, promising but where is it going?

Indoor/outdoor, coverage spreading

Still a hard problem, can give much more than location (good & bad)

Beyond Location

sensors → location → context

Patterson, Liao, Fox, Kautz, “Inferring High-Level Behavior from Low-Level Sensors”, 2003

Use GPS tracking to infer user’s mode of transportation as (car, bus, walk)

Sparacino, “Sto(ry)chastics: A Bayesian Network Architecture for User Modeling and Computational Storytelling for Interactive Spaces”, 2003

Use indoor location sensing in MIT museum to classify visitor into (greedy, busy, selective)

Outline

• Introduction• Video Tracking• Active badge – SmartMoveX• Coarse Wi-Fi location – “Here I Am”• Fine Wi-Fi location – LOCADIO

• SPOT Wristwatch Location – RightSPOT• Wi-Fi proximity – NearMe• “Longhorn” Location Service – Sensor Fusion

Video Tracking

EasyLiving Project

Steve ShaferBarry BrumittSteve HarrisBrian MeyersGreg SmithMike HaleJohn Krumm

Video Tracking Is Very …• … accurate (centimeters)• … easy for user (no devices to carry)• … hard to set up (camera calibration)• … hard to get right (live demos still rare)• … CPU intensive (one PC per camera)• … intrusive

Research can solve

Outline



SmartMoveX

TransmitterReceiver

Microsoft Research’s entry into the active badge space (along with Xerox PARC, UW, Intel, AT&T Cambridge, MIT, etc.)

Hardware: Lyndsay Williams (Microsoft Research Cambridge UK)Software: John Krumm & Greg Smith (Microsoft Research Redmond)

Multiple receivers for position triangulation from signal strengths

Receiver Network

DB PC

RX PC

RX PC

RX PC

RX PC

RX

RX

RX

RX

Graph Algorithm• Compute path instead of single locations• Constrain path to allowable routes• Process with Hidden Markov Model (HMM – same as used for speech recognition)• Average error 3.05 meters

Constraints make things easier

Signal Strength vs. Time

110

120

130

140

150

160

170

180

190

0 5 10 15 20 25 30 35 40 45 50 55 60

Time (sec)

Sig

nal

Str

eng

th

SmartMoveX EvaluationGood• Cheap hardware• Uses existing network infrastructure• Graph algorithm imposes natural constraints on paths

Needs Improvement• Privacy – depends on central server• Convenience

– Extra device to wear– No display on device

• Infrastructure – requires special receivers

Outline



Active Badge

“Here I Am” – Coarse Location

With Steve Shafer (Microsoft Research)

“Here I Am”

MAC Address Building Floor Room

0:40:96:29:e:7 18 2 2075

0:40:96:29:f:78 18 1 1407

0:40:96:29:1a:6c 18 2 2229

0:40:96:29:24:c7 18 1 1055

0:40:96:29:3f:c3 18 1 1212

0:40:96:29:48:37 18 2 2115

0:40:96:29:4a:40 18 2 2039

0:40:96:29:55:8 18 1 1081

0:40:96:29:57:a0 18 2 2498

0:40:96:29:58:2f 18 1 1265

0:40:96:29:59:16 18 1 1426

0:40:96:29:5b:1d 18 1 1137

0:40:96:29:63:1a 18 3 3367

802.11 Access Point Data Room Numbers Hand-Entered from Maps

“Here I Am” returns position of strongest access point

Outline



Location from 802.11 with LOCADIO*

• Mobile device measures signal strengths from Wi-Fi access points• Computes its own location

Wi-Fi (802.11) access point

* Location from Radio

John Krumm & Eric Horvitz

http://www.colorado.edu/physics/2000/laptops/images/lapTop.gif

LOCADIO - Fine Location

Radio survey to get signal strength as a function of position

LOCADIO - Constraints

No passing through walls No speeding

200 400 600 800 1000 1200

still

moving

still

moving

still

moving

A Posteriori Probability of Move

time (seconds)

actual

unsmoothed

smoothed

We know when you move

Make the client as smart as possible to reduce calibration effort

LOCADIO - Results

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

0.1

0.2

0.3

error in meters

rela

tive

fre

qu

ency

Error Histogram

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

0.5

1

error in meters

rela

tive

fre

qu

ency

Cumulative Error Distribution

Hidden Markov model gives median error of 1.53 meters

Outline



SPOT Watch

weather traffic

dining movies

Commercial FM: transmit new data every ~2 minutes

Filter on watch to take what it wants

Watch displays “personalized” data

Location-Sensitive Features

Nice to have

• Local traffic• Nearby movie times• Nearby restaurants

Need to know location of device …

Use FM Radio Signal StrengthsKPLU 88.5KAOS 89.3KASB 89.3KNHC 89.5KWFJ 89.7KGHP 89.9KGRG 89.9KUPS 90.1KEXP 90.3KSER 90.7KVTI 90.9KBCS 91.3KBTC 91.7KLSY 92.5KUBE 93.3KMPS 94.1KRXY 94.5KUOW 94.9KJR 95.7KYPT 96.5KBSG 97.3KING 98.1KWJZ 98.9KISW 99.9KQBZ 100.7KPLZ 101.5KZOK 102.5KMTT 103.7KMIH 104.5KFNK 104.9KCMS 105.3KBKS 106.1KRWM 106.9KNDD 107.7

Scan signal strengths of 32 FM radio stations at 1 Hz

0 100 200 300 400 500 600 700560

570

580

590

600

610

620

630

640

650

660

Sample Number

RS

SI

RSSI Values in One Suburb

raw rssimedian filtered

Clustering Approach

KPLU

KM

TT

Redmond

Bellevue

Seattle

Woodinville

Sammamish

But• Each watch scales signal strengths differently• Impractical to calibrate every watch

Input PowerM

easu

red

RS

SI

A B C

Ranking ApproachRedmond: KPLU < KMTT < KMPSBellevue: KMTT < KPLU < KMPSIssaquah: KMTT < KMPS < KPLU…

n radio stations n! possible rankings

f = (f1, f2, f3, …, fn) = scanned frequencies

s = (s1, s2, s3, …, sn) = signal strengths

r = (r1, r2, r3, …, rn) = ranks of signal strengths

e.g. s = (12, 40, 38, 10)r = (2, 4, 3, 1)

R(r) = permutation hash code = [0, 1, 2, … n!-1]

Any monotonically increasing function of signal strength preserves ranking

Test

RedmondWoodinville

KirklandBellevue

IssaquahSammamish

01

23

45

0

0.2

0.4

0.6

0.8

1

LocationPermutation Hash Code

Six suburbs and six radio stations

81.7% correct from 8 radio stations

Avoid Manual Training

Seattle KMPS 94.1 MHz KSER 90.7 MHz

Classify Into Grid Cell

• Find location in grid

• Use predicted signal strengths to avoid manual training

≈ 8 kilometers average error

Summer intern Adel Youssef, U. Maryland

Outline



“NearMe”

Person

conferencerooms

printers

bathroom reception desk

people

Find people and things nearby

NearMe Basic Idea

802.11 Access Points

NearMe Server

NearMe ≠ Location

Hightower, Fox, Borriello, “The Location Stack”, 2003

Why compute absolute locations when you only need relative locations?

Tomasi, Kanade, “Shape and Motion from Image Streams: a Factorization Method”, 1991

Short Circuit

Get signal strengths Get signal strengths

Get locationGet location Compare

Compare

NearMe Screen Shots

1. Register with server

3. Nearby people

2. Report Wi-Fi signals

4. Nearby printer(s)

NearMe Server• SQL Server• .NET Web Service

NearMe Distance Estimate

1

2

3

4

5 -1 -0.75 -0.5 -0.25 0.0 0.25 0.5 0.75 1.0

5

10

15

20

25

Estimated Distance Between Clients

Number of Access Points in Common, n∩

Spearman Correlation, ρs

Dis

tanc

e in

Met

ers

Distance = f(n∩,ρs) n∩ = number of access points seen in common

ρs = Spearman rank correlation of signal strengths

Estimate distance between two clients by comparing “Wi-Fi signatures”

15 meters rms error

NearMe Applications

Look up URLs of nearby people/things

Send email to people nearby

Device association (with Ken Hinckley, Microsoft Research)

Outline



“Longhorn” Location Service

Location Service

AN/PLR-3 Helmet-Mounted Radar(I am not making this up.)

GPS

cell phone

Wi-Fi

Bluetooth

other unanticipated

other location resolvers

your location

“Longhorn” PC knows its location

Longhorn Location Service

MapPoint Db / Serv.

AD

Mgmt App (shell, netxp, OEM)

Win

FS

Wireless (802.11) Zero Configuration Service

BT Configuration Service

OEM Service

Plugin Manager

Master ResolverFuser

LocProv API

LocMgmt API

802.11 Provider

Blue Tooth Provider

OEM Provider

Use

r R

esol

ver

AD

R

esol

ver

Map

Poi

nt

Res

olve

r

Use

r P

ref.

Db.

Lo

cati

on

AP

I

No

tific

atio

n

Se

rvic

e

App

(S

hell,

OE

M)

Cache

Tracey Yao, PMFlorin Teodorescu, DevVivek Bhanu, DevJim Seifert, TestMadhurima Pawar, Test

Fusion

-10-5

05

10

-10

-5

0

5

100

0.02

0.04

0.06

0.08

0.1

0.12

XY-10-5

05

10

-10

-5

0

5

100

0.02

0.04

0.06

0.08

0.1

0.12

XY

Metric Measurements

Building 40 (0.9) 40 (0.4) 41 (0.4)

Floor 3 (0.8) 3 (0.2) 1 (0.3)

Room 3019 (0.2) 1502 (0.2)

Hierarchies

Kalman Filter Weighted Hierarchical Voting

Proper fusion of measurements depends on knowledge of uncertainty

The End

Location-Aware Computing John Krumm Microsoft Research Redmond, Washington, USA.

Documents

Transcript of Location-Aware Computing John Krumm Microsoft Research Redmond, Washington, USA.