Post on 30-Dec-2015
description
Using Area-based Presentations and Metrics for Localization Systems in Wireless LANs
Eiman Elnahrawy, Xiaoyan Li, and Richard Eiman Elnahrawy, Xiaoyan Li, and Richard MartinMartin
Dept. of Computer Science, Rutgers UniversityDept. of Computer Science, Rutgers University
WLN, November 16WLN, November 16thth 2004 2004
Dept. of Computer Science, Rutgers University
WLAN-Based Localization
Localization in indoor environments using Localization in indoor environments using 802.11 and Fingerprinting802.11 and Fingerprinting
Numerous useful applicationsNumerous useful applications
Dual use infrastructure: a huge advantageDual use infrastructure: a huge advantage
Dept. of Computer Science, Rutgers University
Background: Fingerprinting Localization
Classifiers/matching/learning Classifiers/matching/learning approachesapproaches
Offline phase:Offline phase:Collect training data Collect training data (fingerprints)(fingerprints)
Fingerprint vectors: [(x,y),SS]Fingerprint vectors: [(x,y),SS]
Online phase:Online phase:MatchMatch RSS to existing RSS to existing fingerprints fingerprints probabilistically or probabilistically or using a distance metricusing a distance metric
[-80,-67,-50]
RSS
(x?,y?)
[(x,y),s1,s2,s3]
[(x,y),s1,s2,s3][(x,y),s1,s2,s3]
Dept. of Computer Science, Rutgers University
Output:Output:
A single location: the closest/best matchA single location: the closest/best match
We call such approaches We call such approaches “Point-based “Point-based Localization”Localization”
Examples:Examples:
RADARRADAR
Probabilistic approachesProbabilistic approaches[Bahl00, Ladd02, Roos02, Smailagic02, Youssef03, Krishnan04]
Dept. of Computer Science, Rutgers University
Contributions: Area-based Localization
Returned answer is area/volume Returned answer is area/volume likely to contain the localized likely to contain the localized objectobject
Area is described by a set of Area is described by a set of tilestiles
Ability to describe uncertaintyAbility to describe uncertainty
Set of highly possible Set of highly possible locationslocations
Dept. of Computer Science, Rutgers University
Contributions: Area-based Localization
Show that it has critical advantages over point-Show that it has critical advantages over point-based localizationbased localization
Introduce new performance metricsIntroduce new performance metrics
Present two novel algorithms: SPM and ABP-cPresent two novel algorithms: SPM and ABP-c
Evaluate our algorithms and compare them against Evaluate our algorithms and compare them against traditional point-based approachestraditional point-based approaches
Related Work: different technologies/algorithms Related Work: different technologies/algorithms [Want92, Priyantha00, Doherty01, Niculescue01, Savvides01, Shang03, He03, Hazas03, Lorincz04]
Dept. of Computer Science, Rutgers University
Why Area-based?
Noise and systematic errors introduce position uncertainty Noise and systematic errors introduce position uncertainty
Areas improve system’s ability to giveAreas improve system’s ability to give meaningful meaningful alternativesalternatives
A tool for understanding the confidence A tool for understanding the confidence
Ability to trade Ability to trade PrecisionPrecision (area size)(area size) for for AccuracyAccuracy (distance the localized object is from the area)(distance the localized object is from the area)
Direct users in their searchDirect users in their search
Yields higher overall accuracyYields higher overall accuracy
Previous approaches that attempted to use areas only use Previous approaches that attempted to use areas only use them as intermediate result them as intermediate result output still a single location output still a single location
Dept. of Computer Science, Rutgers University
Area-based vs. Single-Location
Object can be in a single room or multiple rooms Object can be in a single room or multiple rooms
Point-based to areasPoint-based to areas
Enclosing circles --Enclosing circles -- much largermuch larger
Rectangle? no longer point-based!Rectangle? no longer point-based!
0 2000
10
20
30
40
50
60
70
0 200
80
Dept. of Computer Science, Rutgers University
Outline
Introduction, Motivations, and Related WorkIntroduction, Motivations, and Related Work
Area-based vs. Point-based localizationArea-based vs. Point-based localization
MetricsMetrics
Localization AlgorithmsLocalization AlgorithmsSimple Point Matching (SPM)Simple Point Matching (SPM)
Area-based Probability (ABP-c)Area-based Probability (ABP-c)
Interpolated Map Grid (IMG)Interpolated Map Grid (IMG)
Experimental EvaluationExperimental Evaluation
Conclusion, Ongoing and Future WorkConclusion, Ongoing and Future Work
Dept. of Computer Science, Rutgers University
Performance Metrics
Traditional: Traditional: Distance error between returned and true Distance error between returned and true positionposition
Return avg, 95Return avg, 95thth percentile, or full CDF percentile, or full CDF
Does not apply to area-based algorithms!Does not apply to area-based algorithms!
Does not show accuracy-precision tradeoffs! Does not show accuracy-precision tradeoffs!
Dept. of Computer Science, Rutgers University
New Metrics: Accuracy Vs. Precision
Tile AccuracyTile Accuracy % true tile is returned % true tile is returned
Distance AccuracyDistance Accuracy distance between distance between true tile and returned tiles (sort and true tile and returned tiles (sort and use percentiles to capture use percentiles to capture distribution)distribution)
PrecisionPrecision size of returned area (e.g., size of returned area (e.g., sq.ft.) or % floor sizesq.ft.) or % floor size
Dept. of Computer Science, Rutgers University
Room-Level Metrics
Applications usually operate at the level of roomsApplications usually operate at the level of rooms
Mapping: divide floor into rooms and map tiles Mapping: divide floor into rooms and map tiles
(Point (Point ↔↔ Room): easy Room): easy
(Area (Area ↔↔ Room): tricky Room): tricky
Metrics: accuracy-precision Metrics: accuracy-precision Room AccuracyRoom Accuracy % true room is the returned room % true room is the returned room
Top-n rooms AccuracyTop-n rooms Accuracy % true room is among the % true room is among the returned rooms returned rooms
Room PrecisionRoom Precision avg number of returned rooms avg number of returned rooms
Dept. of Computer Science, Rutgers University
1. Simple Point Matching (SPM)
Build a regular grid of tiles, tile Build a regular grid of tiles, tile ↔↔ expected fingerprint expected fingerprint
Find Find ∩ tiles which fall within a tiles which fall within a “threshold”“threshold” of RSS for each AP of RSS for each AP
Eager:Eager: start from low threshold (= start from low threshold (= δδ, 2 × , 2 × δδ, …), …)
Threshold is picked based on the standard deviation of the Threshold is picked based on the standard deviation of the received signalreceived signal
Similar to Maximum Likelihood EstimationSimilar to Maximum Likelihood Estimation
∩ ∩
=
Dept. of Computer Science, Rutgers University
2. Area-Based Probability (ABP-c)
Build a regular grid of tiles, tile Build a regular grid of tiles, tile ↔↔ expected expected fingerprintfingerprint
Using Using “Bayes’ rule”“Bayes’ rule” compute compute likelihood likelihood of an RSS matching of an RSS matching the fingerprint for each tilethe fingerprint for each tile
p(Ti|RSS) p(Ti|RSS) αα p(RSS|Ti) . p(Ti) p(RSS|Ti) . p(Ti)
Return top tiles bounded by an overall probability that the Return top tiles bounded by an overall probability that the object lies in the area object lies in the area (Confidence: user-defined)(Confidence: user-defined)
Confidence Confidence ↑↑ →→ Area size Area size ↑↑
Dept. of Computer Science, Rutgers University 40
45
50
55
60
65
70
75
80
85
90
x in feet
y in
fee
t
0 210
140
Measurement At Each Tile Is Expensive!
Interpolated Map Grid: Interpolated Map Grid: (Surface Fitting)(Surface Fitting)
Goal:Goal: Extends original training data to cover the entire floor Extends original training data to cover the entire floor by deriving an expected fingerprint in each tileby deriving an expected fingerprint in each tile
Triangle-based linear interpolation using Triangle-based linear interpolation using “Delaunay “Delaunay TriangulationTriangulation”
Advantages:Advantages:
Simple, fast, and efficientSimple, fast, and efficient
Insensitive to the tile sizeInsensitive to the tile size
Dept. of Computer Science, Rutgers University
Impact of Training on IMG
BothBoth locationlocation and and numbernumber of training samples of training samples impact accuracy of the map, and localization impact accuracy of the map, and localization performanceperformance
Number of samples has an impact, Number of samples has an impact, but not strongbut not strong!!Little difference going from 30-115, no difference Little difference going from 30-115, no difference using > 115 training samplesusing > 115 training samples
Different strategies Different strategies [Fixed spacing vs. Average [Fixed spacing vs. Average spacing]spacing]: as long as samples are : as long as samples are “uniformly “uniformly distributed”distributed” but not necessarily but not necessarily “uniformly spaced”“uniformly spaced” methodology has no measurable effectmethodology has no measurable effect
Dept. of Computer Science, Rutgers University
Experimental Setup
CoRECoRE
802.11 data: 286 fingerprints (rooms + hallways)802.11 data: 286 fingerprints (rooms + hallways)
50 rooms50 rooms
200x80 feet200x80 feet
4 Access Points4 Access Points
Dept. of Computer Science, Rutgers University
Area-based Approaches: Accuracy-Precision Tradeoffs
Improving Improving AccuracyAccuracy worsens worsens PrecisionPrecision (tradeoff)(tradeoff)
0 50 100 150 200 25060
65
70
75
80
85
90
95
100
Training data size
% a
ccu
racy
Average Overall Room Accuracy
SPM
ABP-50
ABP-75
ABP-95
0 50 100 150 200 2500
2
4
6
8
10
Training data size
% f
loo
r
Average Overall PrecisionSPM
ABP-50
ABP-75
ABP-95
Dept. of Computer Science, Rutgers University
0 20 40 60 80 1000
0.2
0.4
0.6
0.8
1
pro
ba
bil
ity
ABP-75: Percentiles' CDF
Minimum25 PercentileMedian75 PercentileMaximum
0 20 40 60 80 1000
0.2
0.4
0.6
0.8
1
pro
ba
bil
ity
ABP-50: Percentiles' CDF
Minimum25 PercentileMedian75 PercentileMaximum
0 20 40 60 80 1000
0.2
0.4
0.6
0.8
1
distance in feet
pro
ba
bil
ity
SPM: Percentiles' CDF
Minimum25 PercentileMedian75 PercentileMaximum
0 20 40 60 80 1000
0.2
0.4
0.6
0.8
1
distance in feet
pro
ba
bil
ity
ABP-95: Percentiles' CDF
Minimum25 PercentileMedian75 PercentileMaximum
A Deeper Look Into “Accuracy”
Dept. of Computer Science, Rutgers University
Sample Outputs
SPM
ABP-75ABP-50
ABP-95Area expands into the true roomArea expands into the true room
Areas illustrate bias across different dimensions (APs’ Areas illustrate bias across different dimensions (APs’ location) location)
Dept. of Computer Science, Rutgers University
Comparison With Point-based localization: Evaluated Algorithms
RADARRADARReturn the Return the “closest”“closest” fingerprint to the RSS in the fingerprint to the RSS in the training set using training set using “Euclidean Distance in signal space” “Euclidean Distance in signal space” (R1)(R1)
Averaged RADAR Averaged RADAR (R2)(R2), Gridded RADAR , Gridded RADAR (GR)(GR)
Highest ProbabilityHighest ProbabilitySimilar to ABP: a typical approach that uses Similar to ABP: a typical approach that uses “Bayes’ “Bayes’ rule”rule” but returns the but returns the “highest probability single “highest probability single location” location” (P1)(P1)
Averaged Highest Probability Averaged Highest Probability (P2)(P2), Gridded Highest , Gridded Highest Probability Probability (GP)(GP)
Dept. of Computer Science, Rutgers University
Comparison With Point-based
Localization: Performance Metrics
Traditional error along with percentiles CDF for Traditional error along with percentiles CDF for area-based algorithms (min, median, max)area-based algorithms (min, median, max)
Room-level accuracyRoom-level accuracy
Dept. of Computer Science, Rutgers University
CDFs for point-based algorithms fall in-between the min, CDFs for point-based algorithms fall in-between the min, max CDFs for area-based algorithmsmax CDFs for area-based algorithms
Point-based algorithms perform more or less the same, Point-based algorithms perform more or less the same, closely matching the median CDF of area-based algorithmsclosely matching the median CDF of area-based algorithms
Min
Max
Median
Dept. of Computer Science, Rutgers University
Similar top-room accuracySimilar top-room accuracy
Area-based algorithms are Area-based algorithms are superior at returning multiple superior at returning multiple rooms, yielding higher overall room accuracyrooms, yielding higher overall room accuracy
If the true room is missed in point-based algorithms the user If the true room is missed in point-based algorithms the user has no clue!has no clue!
Dept. of Computer Science, Rutgers University
Conclusion
Area-based algorithms present users a more intuitive Area-based algorithms present users a more intuitive way to reason about localization uncertaintyway to reason about localization uncertainty
Novel area-based algorithms and performance metricsNovel area-based algorithms and performance metrics
Evaluations showed that qualitatively all the algorithms Evaluations showed that qualitatively all the algorithms are quite similar in terms of their accuracyare quite similar in terms of their accuracy
Area-based approaches however direct users in their Area-based approaches however direct users in their search for the object by returning an ordered set of search for the object by returning an ordered set of likely rooms and illustrate confidencelikely rooms and illustrate confidence
Dept. of Computer Science, Rutgers University
Ongoing and Future Work
Eiman Elnahrawy, Xiaoyan Li, and Richard P. Martin, Eiman Elnahrawy, Xiaoyan Li, and Richard P. Martin, “The Limits “The Limits of Localization Using Signal Strength: A Comparative Study”,of Localization Using Signal Strength: A Comparative Study”, In In IEEE SECON 2004IEEE SECON 2004
All algorithms behave the same: there is a fundamental All algorithms behave the same: there is a fundamental uncertainty due to environmental effects!uncertainty due to environmental effects!
Point-based approaches find the peaks in the PDFsPoint-based approaches find the peaks in the PDFs
Area-based approaches explore more of the PDF but cannot Area-based approaches explore more of the PDF but cannot narrow it (cannot eliminate the uncertainty)narrow it (cannot eliminate the uncertainty)
Useful accuracy, different ways to describe it to usersUseful accuracy, different ways to describe it to users
Future workFuture work: : additional HW or complex models to improve additional HW or complex models to improve accuracyaccuracy
Dept. of Computer Science, Rutgers University
Thank YouThank You