“Honest GIS”:Error and Uncertainty

Blinded by Science?

Result of “accurate” Result of “accurate” scientific scientific measurementmeasurement

Reveal agenda, Reveal agenda, biases of their biases of their creatorscreators

• GIS databases built from mapsNot necessarily objective, scientific measurements

• Impossible to create perfect representation of world

The Necessity of “Fuzziness”

““It’s not easy to lie with maps, it’s It’s not easy to lie with maps, it’s essential...to present a useful and essential...to present a useful and truthful picture, an accurate map must truthful picture, an accurate map must tell white lies.” tell white lies.” -- Mark Monmonier-- Mark Monmonier

distort 3-D world into 2-D abstractiondistort 3-D world into 2-D abstraction

characterize most important aspects of characterize most important aspects of spatial realityspatial reality

portray abstractions (e.g., gradients, portray abstractions (e.g., gradients, contours) as distinct spatial objectscontours) as distinct spatial objects

Fuzziness (cont.)

All GIS subject to uncertaintyAll GIS subject to uncertainty

What the data tell us about the real worldWhat the data tell us about the real world

Range of possible “truths”Range of possible “truths”

Uncertainty affects results of analysisUncertainty affects results of analysis

Confidence limits - “plus or minus”Confidence limits - “plus or minus”Difficult to determineDifficult to determine

““If it comes from a computer it must be If it comes from a computer it must be wrightwright””

Longley et al., chapter 6

A Conceptual View of Uncertainty (U)

Longley et al., ch. 9, p. 208

Digitizing Errors

Longley et al., ch. 9, p. 209

Error Induced by Data Cleaning

Longley et al., ch. 9, p. 209

Yikes Rubbersheeting Needed!

Uncertainty

Measurements not perfectly accurateMeasurements not perfectly accurate

Maps distorted to make them readableMaps distorted to make them readableLines repositionedLines repositioned

5th St. and railroad through Corvallis at scale 5th St. and railroad through Corvallis at scale of 1:250,000of 1:250,000

At this scale both objects thinner than map At this scale both objects thinner than map symbolssymbols

Map is generalized Map is generalized

Definitions vague, ambiguous, subjectiveDefinitions vague, ambiguous, subjective

Landscape has changed over timeLandscape has changed over time

Berry “Shadow Maps of Uncertainty”

http://dusk.geo.orst.edu/buffgis/http://dusk.geo.orst.edu/buffgis/shadow.htmlshadow.html

Towards an “Honest GIS”

can map a simple feature locationcan map a simple feature location

can also map a continuum of certaintycan also map a continuum of certainty

model of the propagation of error (when model of the propagation of error (when maps are combined)maps are combined)

assessing error on continuous surfacesassessing error on continuous surfacesverify performance of interpolation schemeverify performance of interpolation scheme

More Strategies

Simulation strategySimulation strategyComplex modelsComplex models

Describing uncertainty as “a spatially Describing uncertainty as “a spatially autoregressive model with parameter rho” not autoregressive model with parameter rho” not helpfulhelpful

How to get message acrossHow to get message across

Many models out thereMany models out thereRecent research on modeling uncertainty Recent research on modeling uncertainty (NCGIA Intiative 1)(NCGIA Intiative 1)

Users can’t understand them allUsers can’t understand them all

Strategies (cont.)

Producer of data must describe uncertaintyProducer of data must describe uncertainty““RMSE 7 m” (Lab 6, your Mt. Hood DEM)RMSE 7 m” (Lab 6, your Mt. Hood DEM)MetadataMetadata

FGDC - 5 elementsFGDC - 5 elementsPositional accuracyPositional accuracyAttribute accuracyAttribute accuracyLogical consistency (logical rules? polygons Logical consistency (logical rules? polygons close?)close?)CompletenessCompletenessLineageLineage

Strategies (cont.)

What impact will uncertainty have on results What impact will uncertainty have on results of analysis??of analysis??

(1) Ignore the issue completely(1) Ignore the issue completely

(2) Describe uncertainty with measures (2) Describe uncertainty with measures (shadow map or RMSE)(shadow map or RMSE)

(3) Simulate equally probable versions of data(3) Simulate equally probable versions of data

Simulation Example:http://www.ncgia.ucsb.edu/~ashton/demos/propagate.html

Geographic Data Uncertainty and Ethics

Typical users take digital data for granted, Typical users take digital data for granted, assuming their quality is high and fits the assuming their quality is high and fits the intended usageintended usage

An increasing number of incidents and An increasing number of incidents and accidents result from the inappropriate use of accidents result from the inappropriate use of geospatial datageospatial data““Erroneous, inadequately documented, or Erroneous, inadequately documented, or inappropriate data can have grave inappropriate data can have grave consequencesconsequences for individuals and the for individuals and the environment.”environment.” (AAG Geographic Information Ethics Session Description, 2009)

1919

http://dataquality.scg.ulaval.ca

Geographic Data Uncertainty and Ethics

From an ethics point of view:From an ethics point of view:Poor quality data should not be used for Poor quality data should not be used for sensitive applications where it poses a risk of sensitive applications where it poses a risk of harm harm

Need appropriate safeguards to avoid the harm, Need appropriate safeguards to avoid the harm, and to provide effective warnings and to provide effective warnings

Not enough just to anticipate intended uses and Not enough just to anticipate intended uses and data quality requirements of a system. Must data quality requirements of a system. Must anticipate the anticipate the possible misuses possible misuses of the system as of the system as well well

Data Uncertainty: Today’s Approaches

SpatialDatabase

Internet

DataDiffusion

Dataproduction

Datacollection

DataSelection

Paper map

Web services

DataUsage

Users

Error-aware GIS,Fuzzy operators

Quality analysissystem

Metadatamanagement

Context-sensitivewarnings

Methods to selectbest sources

SpatialIntegrity constraints

Specifications,Quality control

webservices

Users...

-Training-Manuals-Access control

from Bedard et al., U. of Laval

Data Uncertainty: today’s approaches

Victims’ approaches and reactionsDon’t buy

Never useagain…

Don’t follow

Don’tuse

from Bedard et al., U. of Laval

Data Uncertainty: today’s approaches

Ethics-related issue

Professional self-regulatory bodies have Professional self-regulatory bodies have codes of ethics contained in regulationscodes of ethics contained in regulations

These regulations are enacted by These regulations are enacted by governmentsgovernments

Professionals’ primary duty is to the public Professionals’ primary duty is to the public welfarewelfare

Data Uncertainty: today’s approaches

Ethics-related issue

Codes of ethics influence « Good Practices »Codes of ethics influence « Good Practices »Ex. professionals must care about individuals and Ex. professionals must care about individuals and environmentenvironment

« Professional misconduct » is typically set « Professional misconduct » is typically set out in regulationsout in regulations

Ex. Negligence, failure to report or remedy to a Ex. Negligence, failure to report or remedy to a danger, to protect peopledanger, to protect people

In case of lawsuits, Codes of ethics have In case of lawsuits, Codes of ethics have impactsimpacts

Data Uncertainty: today’s approaches

Ethics-related issue Data uncertainty issues end up in the hands Data uncertainty issues end up in the hands

of legal systems, but they begin in the hands of legal systems, but they begin in the hands of systems designersof systems designers

Software engineering methods based on formal models are Software engineering methods based on formal models are recognized as the most rigorous approaches to develop quality recognized as the most rigorous approaches to develop quality systemssystems

Good practices require to understand clearly Good practices require to understand clearly data quality requirements and fitness-for-usedata quality requirements and fitness-for-use

It is a duty for the expert to care about users It is a duty for the expert to care about users and to inform them about inappropriate and to inform them about inappropriate usages of spatial datausages of spatial data

from Bedard et al., U. of Laval

Data Uncertainty: today’s approaches

Ethics-related issue Involve client in every phase of a system Involve client in every phase of a system

development methoddevelopment method

This involvement must include decisions This involvement must include decisions about the risks related to spatial data about the risks related to spatial data definition, selection, production, definition, selection, production, dissemination and potential reuse dissemination and potential reuse (intended or not)(intended or not)

Risk-related decisions must be understood and Risk-related decisions must be understood and approved by the clientapproved by the client

from Bedard et al., U. of Laval

C.A.R.E.F.U.L.CComputer-AAssisted RRisk EEvaluation FFor UUsage

LLimitation

Yvan BédardYvan Bédard11, Jennifer Chandler, Jennifer Chandler22, , Rodolphe DevillersRodolphe Devillers33, Marc Gervais, Marc Gervais11

11Univ. Laval, GeomaticsUniv. Laval, Geomatics22Univ. of Ottawa, LawUniv. of Ottawa, Law

33Memorial Univ. of Newfoundland, Memorial Univ. of Newfoundland, GeographyGeography

« C.A.R.E.F.U.L. »

+CAREFULextension

Formal method+

Data modeling tool

CAREFULknowledgeabout risk

traditionalknowledgeabout risk

« C.A.R.E.F.U.L. »

CAREFUL: WHAT: better risk management of potential spatial data misusesHOW: extending system design methods and modeling tools to add risk-related infoWHY: professional ethics, liability

New needs

New usages

Risk analysis

Warnings

Risk strategy

Training, doc.

Informed andprotected users

-Indifferent-Avoid-Transfer-Control

-Identify-Evaluate

« C.A.R.E.F.U.L. »

Risk-relatedmetadata

in Data modeling

tool

« C.A.R.E.F.U.L. »

ISO-3864-2 Symbols

for Warnings in Data

Modeling Tool

« C.A.R.E.F.U.L. »

Risk-Related Reportingwith the help of Data Modeling Tool -user manual -training material -fitness-for-use report - …

« C.A.R.E.F.U.L. »

Context-sensitive Warnings Generated from Data Modeling Tool

Canada GEOIDE Project #PIV-23

Privacy

Data mashup

Quality of GI

Civil liabilitySocial

and legal issues

Protection of investment

and copyright

Selection and usage

of GI

Objective: to develop innovative solutions to evaluate GI quality and contribute to its responsible commercialization and hence achieve an healthy protection of the public

GeomaticsEngineering

Facultyof Law

GeomaticsEngineering

Geography

On the U.S. side, NSF Ethics Education

Impacts on Professional System Designers, GIS Users

Ethics leads Ethics leads to protecting users against harmto protecting users against harm Several approaches exist to reduce risksSeveral approaches exist to reduce risks

Ethics leads to manage the risks related to uncertain Ethics leads to manage the risks related to uncertain data or inappropriate uses of datadata or inappropriate uses of data including unintended usesincluding unintended uses

CAREFUL is a new ethics-centered approach CAREFUL is a new ethics-centered approach extending formally proven software engineering extending formally proven software engineering methods methods

GGisprofessionalethics.org contain new GIS ethics-isprofessionalethics.org contain new GIS ethics-centered graduate curricula in progress centered graduate curricula in progress 

Gateway to the Literature Plewe, B. The nature of uncertainty in historical Plewe, B. The nature of uncertainty in historical

geographic information, geographic information, Transactions in GISTransactions in GIS, 6(4): , 6(4): 431-456, 2002.431-456, 2002.

UCGIS. UCGIS. Uncertainty in Geographic Data and GIS-Uncertainty in Geographic Data and GIS-Based AnalysesBased Analyses, UCGIS Research Priority White , UCGIS Research Priority White Paper, Leesburg, VA: UCGIS, 2002.Paper, Leesburg, VA: UCGIS, 2002.

DiBiase, D., Harvey, F., Wright, D., and Goranson, DiBiase, D., Harvey, F., Wright, D., and Goranson, C. The GIS professional ethics project: Practical C. The GIS professional ethics project: Practical ethics education for GIS professionals, in Unwin, ethics education for GIS professionals, in Unwin, D., Foote, K., Tate, N., and DiBiase, D. (eds.), D., Foote, K., Tate, N., and DiBiase, D. (eds.), Teaching Geographic Information Science and Teaching Geographic Information Science and Technology in Higher Education, Technology in Higher Education, London: Wiley London: Wiley and Sons, in press, 2011and Sons, in press, 2011..

Gateway to the LiteratureBater, C. W. and N. C. Coops (2009). "Evaluating error associated with LIDAR-derived DEM interpolation." Comp. Geosci 35: 289-300.

Dendoncker, N., C. Schmit, et al. (2008). "Exploring spatial data uncertainties in land-use change scenarios." Int. J. Geog. Inf. Sci. 22(9): 1013-1030.

Xiao, N., C. A. Calder, et al. (2007). "Assessing the effect of attribute uncertainty on the robustness of choropleth map classification." Int. J. Geog. Inf. Sci. 21(1-2): 121-144.

Aguilar, F. J., M. A. Aguilar, et al. (2007). "Accuracy assessment of digital elevation models using a non-parametric approach." Int. J. Geog. Inf. Sci. 21(6-7): 667-686.

Zhou, Q., X. Liu, et al. (2006). "Terrain complexity and uncertainties in grid-based digital terrain analysis." Int. J. Geog. Inf. Sci. 20(10): 1137-1148.

Oksanen, J. and T. Sarjakoski (2006). "Uncovering the statistical and spatial characteristics of fine toposcale DEM error." Int. J. Geog. Inf. Sci. 20(4): 345-370.

Lindsay, J. B. (2006). "Sensitivity of channel mapping techniques to uncertainty in digital elevation data." Int. J. Geog. Inf. Sci. 20(6): 669-692.

Henley, S. (2006). "The problem of missing data in geoscience databases." Comp. Geosci 32: 1368-1377.

Gregory, I. N. and P. S. Ell (2006). "Error-sensitive historical GIS: Identifying areal interpolation errors in time-series data." Int. J. Geog. Inf. Sci. 20(2): 135-152.

Bishop, T. F. A., B. Minasny, et al. (2006). "Uncertainty analysis for soil-terrain models." Int. J. Geog. Inf. Sci. 20(2): 117-134.

Wu, J., T. H. Funk, et al. (2005). "Improving spatial accuracy of roadway networks and geocoded addresses." Trans. GIS 9(4): 585-602.

Shi, W. Z., Q. Q. Li, et al. (2005). "Estimating the propagation error of DEM from higher-order interpolation algorithms." Int. J. Remote Sensing 26(14): 3069-3084.

Shi, W. Z., M. Ehlers, et al. (2005). "Uncertainties in integrated remote sensing and GIS." Int. J. Remote Sensing 26(14): 2911-2916.

Kardos, J., G. Benwell, et al. (2005). "The visualisation of uncertainty for spatially referenced census data using hierarchical tessellations." Trans. GIS 9(1): 19-34.