1/17
A Framework for Sensitivity Analysis
in Spatial Multiple Criteria Evaluation
Ligmann-Zielinska A, Jankowski P
REPRINT Published in GIScience 2008, LNCS 5266, T.J. Cova et al. (Eds.)
Springer-Verlag Berlin, 217–33
Abstract The paper presents a framework for sensitivity analysis (SA) in spatial multiple
criteria evaluation (S-MCE). The framework focuses on three aspects of S-MCE: spatiality,
scope, and cardinality. Spatiality stresses the importance of spatial criteria and spatial
weights that should be explicitly considered in GIS-based MCE. Scope relates to the extent
of SA, ranging from local one-at-a-time criterion examination to global testing of
interdependencies among the multiple criteria model components. Cardinality addresses the
duality of motivation for performing SA, namely, single-user learning and group consensus
building. The framework organizes the existing SA techniques according to spatiality and
scope and can be used as a conceptual guide in selecting SA techniques fitting a task at
hand.
Keywords Spatial Multiple Criteria Evaluation, Sensitivity Analysis, SDSS
2/17
1 Introduction
Spatial multiple criteria evaluation (S-MCE) belongs to one of the basic analytical methods
of GIS (DiBiase et al. 2006). Over the last two decades, much work has gone into
integrating MCE techniques with GIS to support formulation, modeling, and evaluation of
spatial decision problems (Carver 1991; Jankowski 1995; Laaribi et al. 1996; Eastman
1999; Malczewski 1999; Thill 1999; Malczewski 2006). S-MCE is a method supporting a
rational decision process in which a set of geographical options is evaluated based on a
number of decision criteria in search of the best choice (Malczewski 1999). Many
applications of spatial decision support systems (SDSS), of which S-MCE is a part, suggest
that S-MCE has become a well established procedure for solving spatial choice problems.
However, S-MCE models have also been criticized for the inadequate treatment of
uncertainty present in model outcomes. For example Uran and Janseen (2003), in their
assessment of five SDSS models, identify the shortcomings of spatial post-processing
analysis of option rankings generated with S-MCE. The uncertainty arises more often than
not from the preliminary character of data and unstable human preferences. We may argue
that the potential of using S-MCE lies in its exploratory approach to analyzing the decision
problem (Malczewski 2006). Thus, ironically, the very essence of S-MCE might be at the
same time its weakest point.
In this paper, we argue for strengthening the exploratory role of S-MCE by focusing
on sensitivity analysis (SA) as part of the decision support methodology. In response to the
lack of systematic treatment of SA in the S-MCE literature, we propose a framework for
organizing and guiding the use of SA techniques. According to the framework, the
objective of SA in S-MCE is to strengthen the confidence in the obtained solution or, in the
case of weak confidence, help to redefine the set of acceptable alternative solutions. In the
proposed framework, we extend SA beyond the criterion data uncertainty to account for the
spatial characteristics of a geographic decision situation.
The roots of formal SA may be traced to engineering and scientific predictive
modeling (French 1992; Pannell 1997; Gómez-Delgado and Tarantola 2006). Its role has
been also recognized in decision sciences, where the main purpose of SA is strengthening
the bases for a decision recommendation. In Simon’s decision process framework (1960) of
intelligence-design-choice SA is perceived as the core of the final choice stage, where the
decision maker evaluates and selects the desirable solution (Jankowski 1995; Malczewski
1999; Feick and Hall 2004). Still, a comprehensive SA included in S-MCE models is more
the exception than the norm (Delgado and Sendra 2004).
The importance of SA may be attributed to the complexity of decision processes
dealing with spatial choice. Complex spatial problems involve irreducible (aleatory)
uncertainty (Helton and Burmaster 1996) caused by the difficulty of arriving at a stable
preference structure for decision makers. Aleatory uncertainty is a result of semi- or ill-
structured decision problems, where the decision makers are unable to define fully the
problem (Densham 1991; Ascough et al. 2002; Malczewski 2006). A semi-structured
decision problem may involve, for example, an incomplete or vague knowledge of decision
option impacts, in which case SA could be used to examine the sensitivity of evaluation
results derived from various plausible impact characteristics.
3/17
The spatial nature of geographic problems amplifies the complexity of decision
making through spatial interdependencies and spatio-temporal dynamics. Future impacts of
proposed choices are often stochastic and deeply uncertain (Lempert 2002). The solutions
to such problems should be thoroughly evaluated to ensure their robustness under a wide
range of possible conditions (Malczewski 1999; Lempert et al. 2003). Unlike the optimal
outcomes, which are based on normative computational tools, there is a need to look for
robust solutions in the presence of a broad spectrum of beliefs and values and under
varying future conditions. Robustness is defined here as the minimal response of a model
solution caused by changing input conditions. In particular, robust S-MCE solutions are
characterized by rank-order stability (Jankowski et al. 1997), where the prioritization of
options is not significantly affected by minor changes in evaluation components
(Andrienko and Andrienko 2005).
Traditionally, SA has been defined as the analysis of response of a model to
changes in input parameters (Voogd 1983; Malczewski 1999; Krivoruchko and Gotway-
Crawford 2005; Longley et al. 2005). Usually, the general question asked in SA is: given
the outcome, what is its sensitivity to changes in initial conditions? Therefore, in order to
perform S-MCE SA, the decision maker must have a ranking of options already in hand. In
this paper, we conceptualize SA more broadly. We define SA in S-MCE as a “thought
experiment” (Alexander 1989: 323) or “computer-assisted reasoning” (Lempert 2002:
7309) aimed at quantitative and qualitative assessment of the stability of a given option
ranking. This definition is not restricted to the analysis of the stability of ranking given
changes in input parameters. Such a strict definition is counter to the uncertain nature of
spatial decision making. An overly structured SA does not fit into an often substantially
unstructured spatial problem (Feick and Hall 2004). Hence, we should account for various
other – intangible and qualitative – decision factors that may influence the choice.
Furthermore, the spatially-explicit nature of geographical problems calls for new, spatially-
explicit methods of SA (Feick and Hall 2004). The ranking of alternatives should be
analyzed based on both site-specific criterion outcomes and on spatial relations, such as
proximity, contiguity, or clustering.
Finally, we should distinguish between sensitivity analysis and uncertainty analysis
(UA). According to Saltelli et al. (2000) UA is forward-looking in nature. Therefore,
performing UA, assumptions are mapped onto inferences (Saisana et al. 2005), whereas in
SA a backward-looking or reverse analysis is undertaken. UA embraces multiple solutions
and does not refer to any specific (initial) solution. SA focuses on the robustness of a
specific solution. This paper focuses on theoretical and methodological foundations of SA
in S-MCE.
In the following section we present a framework for SA in S-MCE, followed by a
review of SA methods. In the final section, we outline research challenges and recommend
directions for further research on SA in S-MCE.
2 A framework of sensitivity analysis in spatial multiple criteria evaluation
The goal of the proposed framework is to provide an organizational outline of SA including
many methods and multiple analysis pathways. We start from a three-dimensional
4/17
conceptualization of SA, followed by a methodology and a review of techniques that may
be used to uphold the scope and spatiality perspectives.
The following questions guided the formulation of the framework:
1. Which elements of the decision process are the least informational and thus
especially suited for identification in the course of SA? Are they spatially sensitive?
What types of measures should be used to analyze the sensitivity of these decision
components?
2. What is the informational extent of SA? Does it embrace ‘the big picture of the
decision problem’ or ‘a more focused exploration’?
3. What is the motivation for performing SA? Is it individual knowledge discovery or
group consensus building?
To address these questions and to present the fabric of SA in S-MCE, we suggest a 3-
dimensional representation of SA, called the SA cube (Figure 1).
Figure 1 Sensitivity analysis cube of spatial multiple criteria evaluation
2.1 The SA Cube
Each of the axes in Figure 1 represents one of the following characteristics of SA in S-
MCE: spatiality, scope, and cardinality. Spatiality stresses the importance of spatial criteria
and spatial weights that should be explicitly considered in GIS-based MCE. Scope relates
to the extent of SA ranging from local one-at-a-time criterion examination to global testing
of interdependencies among the multiple criteria model components. Cardinality addresses
the duality of motivation for performing SA, namely, single-user learning and group
consensus building.
2.1.1 Spatiality
The spatiality axis comprises both the aspatial and spatial nature of the decision situation
and falls under the rubric of technicality proposed by Belton and Stewart (2002). Based on
the inventory by Delgado and Sendra (2004), the vast majority of the reported SA studies
concern only the aspatial nature of the decision situation. Within this category of SA
methods, the major factors analyzed relate to: the diversity of choice alternatives (option
list), the choice of attributes (criteria), the stability of solution to changes in weights
(weighting), and the uncertainty of evaluation method (e.g. standardization, weighting, and
aggregation techniques). The majority of evaluation methods come from general decision
theory and embrace spatial variability only implicitly. For instance, in the well-established
GIS procedure of weighted overlay, the decision maker de facto performs traditional
weighting by assigning the same importance value to every spatial unit of a given criterion
layer (Lodwick et al. 1990; Lowry et al. 1995).
Only recently it has been recognized that a spatially explicit decision component
may potentially influence the rank-order of alternatives (Herwijnen and Rietveld 1999;
Feick and Hall 2004; Rinner and Heppleston 2006). Spatial SA involves the use of
5/17
topological and non-topological relations in S-MCE (Figure 2). For example, the analyst
may use GIS to calculate distance between spatial decision alternatives and some attractor.
The distance criterion can be further analyzed using traditional SA, for example, by
changing its importance and recalculating the weighted option utility. However, given the
spatial nature of the problem, the analyst should seek a more geographically oriented SA.
Continuing the example, he/she may perform SA by varying the criterion importance over
space and assigning different weights to different locations (Feick and Hall 2004). We call
this spatial weighting (spatial bias) – “a non-uniform weighting of the spatial units”
(Herwijnen and Rietveld 1999: 78), which is a way of articulating stakeholders’ sense of
place. Each decision participant may have an individual spatial frame of reference like
home, work, daily activity route, or other place of importance, which impacts the
perception of the proposed courses of action. Note that this concept of spatial weight
involves the perception (judgment) of criterion importance varying over geographical space
and it is different from another concept of spatial weights that measure the level of
interaction between features in geographical space (Getis and Aldstadt 2004).
A different type of problem arises from the spatial distribution of options and their
criteria values. Following the above example, the analyst may be more interested in options
that form a cluster because their proximity may reinforce positive or negative effects. Such
spatial characteristics (contiguity, compactness, proximity etc.) are called spatial criteria
(Brookes 1997; Malczewski 2006; Rinner and Heppleston 2006) and are derived from
various spatial relations (Figure 2). Consequently, SA applied to spatial criteria does not
involve subjective perception of place (spatial weighing), but rather the uncertainty of
spatial distributions, interactions, impacts, and relationships, which can be studied by
performing repetitive spatial transformations.
Consequently, unlike the geographical SA defined by Lodwick et al. (1990), we
argue that spatially explicit SA pertains rather to relative location than to spatial
coincidence of absolute location. Absolute location manifests itself in the traditional SA
approach of weighted overlay, where the composite score of geographic option is derived
by weighting and re-weighting criteria values at specific locations. Additionally, it involves
the uncertainty of criteria evaluation scores measured at particular sites. Relative location
refers to geographical variability, which manifests itself via situational relations (e.g.
contiguity, compactness, proximity) representing spatial organization and spatial
configuration in reference to a particular location (Couclelis 1991). Within the context of
GIS, such spatially explicit SA complements traditional (aspatial) SA.
6/17
Figure 2 Spatial relations applicable to SA
2.1.2 Scope
Spatiality concerns the structure of the decision process and, in particular, the spatial
component of this structure. However, if the objective is to understand the behavior of a
selected subset of the criteria and their weights in the decision model (one, few, many, or
all) then the scope of SA is more relevant.
Scope may range from a detailed component-focused study to a generalized
simultaneous testing of interdependencies among decision elements. Conventionally, the
practice of S-MCE treats SA as a method of examining one specific component of S-MCE,
where the analyst changes one parameter at a time and evaluates how sensitive the output is
to the change (Malczewski 1999; Longley et al. 2005). Additionally, the analyst may vary
multiple factors but within a small range around the favored values (Saltelli 2000). This
type of SA is very interactive in nature and is termed local SA (Saltelli 2000). Conversely,
if we perturbate one factor within its whole distribution or vary multiple factors
simultaneously over the entire problem space, we perform global SA (Saltelli et al. 1999a).
The latter is much more data intensive and therefore is usually done through a sampling-
based simulation (Saltelli 2000).
The above local-global categorization of SA is specific to an aspatial analysis. The
spatiality of geographic problems, however, puts the scope of SA in a different context.
The division into local spatial SA and global spatial SA is similar to the local-global notion
in spatial statistics. Specifically, spatial SA is defined as the examination of one or more
spatial relations (Figure 2) within the extent of either the proximal (neighborhood) space or
the whole space of the study area (Lodwick et al. 1990). For example, adjacency and
7/17
proximity are more neighborhood-related and should be used when dealing with local
spatial associations. Conversely, the reference frame may span from a single location to the
whole study area. The latter results in a constant spatial weight value, which is simply the
traditional weighting in weighted overlay.
2.1.3 Cardinality
The third dimension of the cube, cardinality, reflects two potential motivations for SA,
namely, 1) insight into individual’s values, and 2) learning about the group values.
Cardinality follows the line of thought proposed by Belton and Stewart (2002), who
divided SA into two distinct perspectives, namely single user and group. The single user
perspective of SA deals with intuition and understanding. In this respect, we study the
convergence or divergence of our favored options with the options suggested by the model
(Belton and Stewart 2002). The user discovers his/her individual viewpoint of the decision
problem. In the context of S-MCE, this may manifest itself in the individualistic spatial
weighting. Group SA follows a different logic and is concerned more with the perspectives
of others than with the individual perspective. Whereas single-user SA has its roots in
operations research, the group SA is more related to collaborative learning. Given the
value-laden nature of group decision making, group SA should incorporate analytic-
deliberative functionality (Nyerges et al. 2006), to enhance place-based qualitative
perceptions and consensus building among stakeholders (French 1992; Insua 1999;
Andrienko et al. 2003). A possible quantitative application of group SA in S-MCE relates
to the analysis of spatial equity (Talen 1998).
3 Methods and measures of SA
This section examines various methodological approaches to SA within the SA cube
framework (Figure 1). Since none of the methods developed so far is all-inclusive, and
since different methods of SA produce different outcomes, a good understanding of SA
methodology and trade-offs involved in using different methods is needed in order to
effectively apply SA (Alexander 1989; Saltelli 2000; Andrienko and Andrienko 2005).
3.1 Traditional local and global SA methods
Given an initial decision option prioritization, we may focus on analyzing the sensitivity of
various components that contribute to computing the rank-order. Therefore, we divided the
methods into four broad, partly overlapping categories of options, criteria, weights, and
option scoring/ranking (Figure 3).
8/17
Figure 3 Aspatial SA methods and their scope
The first group of methods concentrates on modifying the list of examined options.
The analyst may delete some options which score low or are otherwise inferior or he/she
may add back the previously deleted option. It is also possible to impose a constraint on a
criterion value and thus modify the ranking by removing these options that do not meet the
constraint (Andrienko et al. 2003).
Similarly to adding/deleting options, we may modify the rank-order by deleting or
adding a criterion. According to Alexander (1989) a criterion that changes the order of the
best option (by being added or deleted) may be termed sensitive. We modify the list of
criteria if they do not reflect our values and preferences (Voogd 1983). In weighted
overlay, changing the attribute list is equivalent to map removal sensitivity (Lodwick et al.
1990). Additionally, we might be uncertain about criteria evaluation scores (Voogd 1983).
This problem is best addressed by using ranges/distribution of values rather than a specific
value. With such constructed parameters, we may perform a Monte Carlo (MC) simulation
of output variability due to the uncertainty of criteria scores (Saisana et al. 2005).
Criterion uncertainty may also be related to a standardization method used to
convert criteria to a common scale (Alexander 1989; Saisana et al. 2005) or the valuation
of a criterion (Jankowski et al. 1997). For example, a person may choose to maximize the
proximity to a proposed transportation improvement project since the project will benefit
him/her by shortening his/her daily commute. Another person may deem such a proximity
criterion as cost because it reduces public safety and hence, he/she wants to locate the
projects elsewhere (minimize proximity). A novel method of determining the most critical
decision criteria was proposed by Triantaphyllou (2000), where the criticality is defined by
the minimum change in performance measures (evaluation scores) causing the rank
9/17
reversal for any two options. For the highly uncertain criteria it may also be useful to
perform rescaling (Voogd 1983) from a higher to a lower measurement level, for example
from ratio/interval to ordinal scale. The resultant decrease in accuracy may in fact better
reflect the true reliability of data.
Weights have often been criticized as the subjective component of S-MCE and
hence have been the focus of SA methods. The basic method of examining the sensitivity
of weights relates systematic changes in weight values to changes in option ranking.
Weighting plays a double role in S-MCE – it either represents a relative criterion
importance or a substitution rate among criteria (Feick and Hall 2004). Due to the large
number of explicit weighting methods that have been proposed (Malczewski 1999), the
choice of weighting method can also be a subject of SA (Saisana et al. 2005). It is also
possible to determine critical weights for these criteria for which a relatively small change
in the weight value causes the rank reversal of any two options (Triantaphyllou 2000;
Bojórquez-Tapia et al. 2005). Additionally, for one parameter at a time weight change
analysis, there is a rule of thumb for establishing criterion criticality, which states that if the
decision maker changes the parameter by n-percent and the result will change by less than
n-percent then he/she may conclude that this parameter does not significantly influence the
result (Longley et al. 2005).
The methods so far discussed are mainly local in scope. A more complex approach
to establishing the importance of criteria scores and weights involves global SA, which
decomposes the variance of the output of the MCE process into a variety of explanatory
factors. An example of a global SA method is the extended Fourier Amplitude Sensitivity
Test (FAST). The extended FAST method uses first and total order indices as SA
measures. The first order sensitivity index is defined as a fractional contribution to the
variance of MCE model output (e.g. option ranking) due to the uncertainty of a given input
parameter treated independently from other parameters (Saltelli et al. 1999a; Saltelli et al.
1999b; Crosetto and Tarantola 2001; Gómez-Delgado and Tarantola 2006; Saisana et al.
2005). The total order index represents the overall contribution of a given parameter (e.g.
criterion weight) including its interactions with other parameters. Computation of the
indices requires a large number of rank-order calculations performed with weight vectors
derived from the decision maker’s weight distribution functions (Saisana et al. 2005).
The key challenge for SA in S-MCE is to determine the stability of the rank-order
of decision options. While all four groups of aspatial SA methods in Figure 3 can be used
to determine the stability of rank-order, the option scoring and ranking methods address
this task directly. Combining weighted multiple criteria values is inherently uncertain since
none of the developed aggregation methods is flawless. Thus, SA may also comprise of
comparing the stability of the rank-order under different aggregation methods (Massam
1991; Laaribi et al. 1996).
Sometimes, highly ranked decision options are very close to each other in terms of
their overall evaluation scores being very similar. Such a situation warrants a careful
investigation of critical score difference (Figure 3). For these options, it may be interesting
to discover what changes in weights make them score equally well, a procedure which
Pannell (1997) calls the “break even value”. If the break even value is within an acceptable
range, then we may justify the switch in rank and select an option which we value more
10/17
given other intangible criteria. The scale of the necessary weight modification may also
provide information about options that are more likely to be ranked first – a method called
proximity ranking (Wolters and Mareschal 1995).
Insua and French (1991) proposed a more generic framework for SA scoring and
ranking. According to their framework, we should first find the non-dominated alternatives,
then narrow down the non-dominated set to those alternatives that are optimal, find the
adjacent potentially optimal alternatives, and establish for them the least change in weights
that is needed to switch the highest ranked option. Such analysis not only pinpoints the
most efficient options and their competitors, but also provides information about the
minimum tradeoffs between the options (French 1992). A more extensive SA of option
scoring relies on the division of multidimensional space into subsections of weight value
ranges where a selected favored alternative wins (Belton and Stewart 2002). These
subsections are then called preference regions. Sampling-based MC simulation is another
useful approach to assessing rank stability (Voogd 1983; Malczewski 1999; French and Xu
2005; Longley et al. 2005). For example, for a large number of rank-orders from MC runs
we can calculate different summary statistics like minimum, maximum, and mean position
of the option thus revealing how stable the option rank is under changing parameter values
(Andrienko and Andrienko 2005).
3.2 Spatial SA methods
Spatial SA is an underdeveloped component of SA in S-MCE. Studies that utilize spatially-
explicit SA are rare. Table 1 is a proposition of a GIS-based methodology that addresses
spatially-explicit components of S-MCE problems. Based on these components, spatial SA
methods are grouped into options, criteria, and weights-focused methods.
The first group of spatial SA methods refers to geographical distributions of the
decision options represented in Table 1 by four categories of relations: adjacency,
proximity, pattern, and direction. For example, the decision maker may prefer to select
high-ranked options because they are located in the direct proximity to other high-ranked
options (Rinner and Heppleston 2006). Such spatial autocorrelation can lead to positive
spatial externalities. Alternatively, the decision maker may use spatial statistics to analyze
and map significant clusters of high scoring options. Other spatial SA operations,
pertaining to spatial distribution of options, include map algebra, topology rules, or density
analysis. Such operations can be used to derive geographically adjusted option evaluation
scores (Rinner and Heppleston 2006).
The sensitivity of decision criteria stems from the uncertainty of the evaluation
scores (criteria values). In spatial decision situations, such sensitivity could be analyzed
based on traditional value measurement uncertainty (error) or value assessment ambiguity,
and spatial uncertainty stemming from the geographical distribution of a given criterion.
For example, if the decision maker uses rainfall as a decision criterion in raster-based S-
MCE, he/she should consider the uncertainty of the rainfall measure at a particular
location, together with the uncertainty associated with a selected interpolation method. One
way of analyzing the spatial sensitivity of a criterion involves adding an uncertainty surface
to the criterion surface. The uncertainty surface should be derived from a spatial
11/17
distribution of the criterion under consideration (Krivoruchko and Gotway-Crawford
2005).
Table 1 Methods supporting spatial SA
S-MCE
component
Sensitivity Spatial
Relation
SA Procedure Exemplary Operations
Options Spatial distribution of options (e.g.
dispersed, clustered, contiguous)
Adjacency Map options that are adjacent Topology Operations (Share a Line Segment)
Proximity Locate proximate options Spatial Statistics (Measures of Centrality, Cluster Analysis) Distance Decay Functions
Pattern Map the dispersion of options and
the shape of option clusters
Spatial Statistics (Mapping
Clusters) Point/Line/Kernel Density Map Algebra (Neighborhood Operations)
Direction Map the direction of options distribution
Directional Distribution Directional Mean
Criteria Spatial distribution of evaluation scores
Pattern Modify the criterion layer with an uncertainty (‘noise’) random layer derived from a spatial distribution of the criterion
Spatial Statistics (Cluster Analysis, Spatial Autocorrelation) Point/Line/Kernel Density
Spatial Interpolation
Use a more generalized criterion layer
Map Algebra (Neighborhood Operations) Reclassify/Remap
Aggregate
Weights Reference frame (point, line or area of interest) and
its spatial distribution
Proximity Use different reference frames as spatial weight layers
Straight Line Distance Distance Decay Functions
Spatial Statistics (Standard Distance, Standard Deviational Ellipse)
Containment Variable-size Buffer Overlay (Point on Polygon, Line
on Polygon, Polygon on Polygon) Map Algebra (Boolean Operations, Zonal Operations)
12/17
Another method of analyzing criterion spatial sensitivity involves attribute
generalization (Krivoruchko and Gotway-Crawford 2005), similar to the more traditional
rescaling proposed by Voogd (1983).
Spatial weight sensitivity pertains to the subjective reference frame of the decision
maker. Participants may perceive certain locations as more favorable from a given
perspective. For example, they may conceive the rainfall criterion as being more important
in rural areas than elsewhere. Consequently, spatial sensitivity of criteria weights may have
a direct effect on option scoring and ranking and should be explicitly addressed. Two
spatial relations that capture the spatial sensitivity of criteria weights are the proximity of
and the containment within the areas of interest (Table 1). The former can be applied using
different distance-decay functions (Rinner and Heppleston 2006) or modeling situation
factors (Cromley and Huffman 2006). The latter can be implemented by a variable-size
buffer analysis (Halls 2002; Tarantola et al. 2002; Krivoruchko and Gotway-Crawford
2005).
3.3 Methods supporting group SA
The SA methods discussed above support the dynamics of group SA only implicitly. When
used in a collaborative setting, the results of individual SA must be aggregated by some
means. Jankowski et al. (1997) and Feick and Hall (2004) present consensus maps that
show the dispersion of summarized votes using graduated circles or colors. The votes relate
to a number of decision elements like option scores, criteria weights or criteria selection.
Borda or Copeland voting protocols are used to aggregate the votes. Additionally, the
variance of voting results can be displayed to identify the most contentious issues that need
further discussion. These analytical techniques rely solely on non-spatial social voting
functions. Spatially-explicit and scope-variant group SA remains to be explored in future
research.
4 Discussion and challenges
Like any method of S-MCE, SA should embody a range of techniques that fit multiple
styles of decision making (Merkhofer 1998; Andrienko et al. 2003; French and Xu 2005)
and, thus, the need for appropriate SA methods is self-evident. Many well-developed
techniques for SA in S-MCE exist; however, each of them has limitations.
The aspatial methods of SA have been used for years and there is an extensive
record of SA applications. However, these methods ignore a geographic aspect of S-MCE
expressed by the spatial variability of criteria values and weights. We have suggested that
spatial variability and hence, the sensitivity of S-MCE solutions to criteria values and
weights can be addressed by the analysis of spatial relations. The methodology of spatial
SA is currently still in its infancy and lacks techniques for rank stability testing.
Furthermore, not much is known on whether spatial SA can enhance group decision
making (Feick and Hall 2004).
13/17
Until recently, the majority of studies reported on local one parameter at a time SA
techniques. Local SA proved to be useful for human-mediated dynamic testing of rank-
order stability. Yet, local SA makes sense only if we deal with perfectly linear models
(Saltelli et al. 1999b), which is unlikely in the majority of spatial decision problems. The
most appropriate technique for non-linear processes is the model-independent variance-
based global SA (Saisana et al. 2005). Also, due to its theoretical underpinnings, global SA
constitutes an important advance over the local SA methods.
It is also worth noting the role of qualitative group SA, in which the stakeholders
negotiate using in-depth descriptive information that may reinforce or change their initial
selection. Although only mentioned in this paper, such soft SA presents an intriguing area
for future research and has the potential to bridge quantitative and qualitative approaches in
decision making (Lempert 2002).
4.1 Visual representations of SA
SA is a useful approach to explore spatial decision options, but its utility can be further
enhanced by the use of a cognitively straightforward visual feedback to decision makers. In
this respect, the greatest progress has been made in the techniques of traditional local SA.
The developments have been based on the principles of human-computer interaction and
include user-friendly interfaces, computationally efficient algorithms, data brushing and
dynamic linking between the maps, tables, and graphs. A variety of tools are available in
today’s software including weight sliders (Jankowski et al. 2001; Andrienko and
Andrienko 2005), value path plots (Pannell 1997; Belton and Stewart 2002; Andrienko et
al. 2003), spider-web charts (Pannell 1997), appraisal roses (Voogd 1983), or pie and bar
chart histograms (Chen et al. 2001). More complex display methods have been proposed to
account for the interrelated multidimensional nature of a decision problem. Examples
include graphs of time-variant decomposition (Saltelli et al. 1999b), decision maps
(Jankowski et al. 1999), and policy landscapes combined with policy regions (Bankes
2002).
A future research challenge for visual representations concerns effective techniques
of representing the sensitivity of S-MCE solutions at spatial locations. The exemplary
cartographic representations of SA in the form of rank maps, rank stability maps, or utility
symbol maps should be enhanced with more specific sensitivity maps.
4.2 Representational and computational issues
Krivoruchko and Gotway-Crawford (2005) notice that the results of uncertainty and SA
may be numerous and call for creative summary tools. Standards for defining statistical
descriptors and improvements in computational techniques bode well for enhancing the
ease of use and comprehensiveness of SA (Feick and Hall 2004). An exemplary possibility
is the dynamic visualization of global SA in the form of a pre-computed approximation of a
solution hypercube, which is dynamically sliceable depending on the specified parameter
set. SA creates new information about the decision process (Pannell 1997) and, within GIS,
this emergent information can be introduced in the form of maps (Jankowski et al. 1999).
14/17
Another important issue to be considered relates to the spatial and aspatial scales of data
granularity, representative to a specific decision situation. A possible solution may involve
a nondeterministic fuzzy approach to SA in S-MCE (Eastman 1999).
5 Conclusions
This article began with declaring SA as the simultaneous advantage and shortcoming of S-
MCE. In the course of the article, we presented a framework for a holistic SA within the
context of S-MCE. The goal of this framework is to assist the decision makers with the
selection of SA methods and techniques that are appropriate for a specific decision
situation. We formalized the structure of SA into spatiality, scope, and cardinality. The first
dimension recognizes the duality of the spatial and aspatial nature of geographic decision
making. The second dimension relates to analytical methods ranging from the low level
exploration of a single decision parameter to the global synthesis of decision situation
sensitivities. The final dimension reflects the divergent motivation for performing SA,
namely individual learning or group consensus building.
We also discussed a number of SA techniques, delineating the advantages and
disadvantages associated with parametric and nonparametric (judgmental) uncertainty.
While the spatiality of alternatives is widely recognized, little progress has been made on
including spatial aspects of decision criteria and decision weights into MCE-based SA.
Therefore, the development of spatially explicit techniques of SA deserves further research.
Additionally, the SA cube framework points to the lack of both spatial and aspatial scope-
variant group SA methods, which are especially needed to enhance collaborative decision
making.
SA is not a substitute for decision making analysis. Instead, it is a way of making
the participants of the decision processes aware of the uncertainties inherent in any
decision situation. Unlike a ‘typical’ perception of scientific analysis, in which we put
probably too much trust in precise (but not necessarily accurate) results, SA emphasizes the
impossibility of providing an ‘always-best’ solution (Saltelli et al. 2004). Ascough et al.
(2002) state that multicriteria SDSS should be an environment where decision makers can
define, explore, redefine and understand the problems they deal with. To accomplish this
goal, we need more research into the methods of spatial SA. The SA framework presented
here is intended to be a step in building the foundation for further progress in spatial
decision support.
Acknowledgements This material is based on research supported in part by National
Science Foundation Grant No. EIA 0325916, funded through the Information Technology
Research Program, and managed by the Digital Government Program. Any opinions,
findings, and recommendations expressed in this material are those of the authors and do
not necessarily reflect the views of the National Science Foundation. The support of NSF is
gratefully acknowledged. The authors would also like to thank three anonymous reviewers
who provided valuable comments on an earlier version of this manuscript.
15/17
References
Alexander ER (1989) Sensitivity Analysis in Complex Decision Models. JAPA 55:323-333
Andrienko G, Andrienko N, Jankowski P (2003) Building Spatial Decision Support Tools for
Individuals and Groups. J Dec Sys 12:193-208
Andrienko N, Andrienko G (2005) Exploratory Analysis of Spatial and Temporal Data: A
Systematic Approach, Berlin-Heidelberg: Springer-Verlag
Ascough II JC, Rector HD, Hoagh DL, McMaster DL, Vandenberg BC, Shaffer MJ, Weltz,
MA, Ahja LR (2002) Multicriteria Spatial Decision Support Systems: Overview,
Applications, and Future Research Directions. Proceedings iEMSs, 24-27 June 2002
Lugano, Switzerland 3:175-180
Bankes SC (2002) Tools and techniques for developing policies for complex and uncertain
systems. PNAS 99:7263-7266
Belton V, Stewart TJ (2002) Multiple Criteria Decision Analysis An Integrated Approach,
Kluwer Academic Publishers
Bojórquez-Tapia LA, Sánchez-Colon S, Martinez AF (2005) Building Consensus in
Environmental Impact Assessment Through Multicriteria Modeling and Sensitivity
Analysis. Env Management 36:469-481
Brookes CJ (1997) A parameterized region growing program for the site allocation on raster
suitability maps. IJGIS 11:375-396
Carver SJ (1991) Integrating multi-criteria evaluation with geographical information systems.
IJGIS 5:321-339
Chen K, Blong R, Jacobson C (2001) MCE-RISK: integrating multicriteria evaluation and GIS
for risk decision-making in natural hazards. Env Modelling & Software 16:387-397
Couclelis H (1991) Requirements for planning-relevant GIS: a spatial perspective. Papers Reg
Sci The Journal of the RSAI 70:9-19
Cromley RG, Huffman FT (2006) Modeling Situation Factors Used in MCE Procedures for
Raster GIS. Transac GIS 10:239-251
Crosetto M, Tarantola S (2001) Uncertainty and sensitivity analysis: tools for GIS-based model
implementation. IJGIS 15: 415-437
Delgado M, Sendra JB (2004) Sensitivity Analysis in Multicriteria Spatial Decision-Making: A
Review. Human Ecol Risk Assess 10:1173-1187
Densham PJ (1991) Spatial Decision Support Systems. Geographical Information Systems:
Principles and Applications, 1st Edition: 403-412, London: Longman
DiBiase D, Demers M, Johnson A, Kemp K, Taylor-Luck A, Plewe B, Wentz E (2006)
Geographic Information Science and Technology Body of Knowledge. UCGIS, AAG
Eastman JR (1999) Multi-criteria evaluation and GIS. Geographical Information Systems:
Principles, Techniques, Management and Applications, 2nd Edition: 493-502, New
York: John Wiley & Sons
Feick RD, Hall GB (2004) A method for examining the spatial dimension of multi-criteria
weight sensitivity. IJGIS 18:815-840
French S (1992) The role of sensitivity analysis in decision analysis. Executive Information
Systems and Decision Support, UNICOM, Applied Information Technology 15:99-
123, Chapman and Hall
French S, XU D-L (2005) Comparison Study of Multi-attribute Decision Analytic Software. J
Multi-Criteria Dec Analysis 14: 65-80
16/17
Getis A, Aldstadt J (2004) Constructing the spatial weights matrix using a local statistic. Geogr
Analysis 36:90-104
Gómez-Delgado M, Tarantola S (2006) GLOBAL sensitivity analysis, GIS and multi-criteria
evaluation for a sustainable planning of a hazardous waste disposal site in Spain. IJGIS
20:449-466
Halls J (2002) A Spatial Sensitivity Analysis of Land Use Characteristics and Phosphorus
Levels in Small Tidal Creek Estuaries of North Carolina, USA. J Coast Res 36:340-351
Helton J, Burmaster DE (1996) Guest Editorial: treatment of aleatory and epistemic uncertainty
in performance assessments for complex systems. Reliab Eng Sys Safety 54:91-94
Herwijnen MV, Rietveld P (1999) Spatial Dimensions in Multicriteria Analysis. Spatial
Multicriteria Decision Making and Analysis A geographic information sciences
approach: 77-99, London: Ashgate
Insua DR (1999) Introduction to Special Issue on Sensitivity Analysis. J Multi-Criteria Dec
Analysis 8:117-118
Insua DR, French S (1991) A framework for sensitivity analysis in discrete multi-objective
decision-making. Eur J Oper Res 54:179-190
Jankowski P (1995) Integrating Geographical Information Systems and Multiple Criteria
Decision-Making Methods. IJGISys 9:251-273
Jankowski P, Andrienko N, Andrienko G (2001) Map-centered exploratory approach to
multiple criteria spatial decision making. IJGIS 15:101-127
Jankowski P, Lotov A, Gusev D (1999) Application of Multiple Criteria Tradeoff Approach to
Spatial Decision Making. Spatial Multicriteria Decision Making and Analysis A
geographic information sciences approach: 127-148, London: Ashgate
Jankowski P, Nyerges TL, Smith A, Moore TJ, Horvath E (1997) Spatial group choice: a SDSS
tool for collaborative spatial decision-making. IJGIS 11:577-602
Krivoruchko K, Gotway-Crawford CA (2005) Assessing the Uncertainty Resulting from
Geoprocessing Operations. GIS, Spatial Analysis, and Modeling: 67-92, Redlands:
ESRI Press
Laaribi A, Chevallier JJ, Martel JM (1996) A spatial decision aid: a multicriterion evaluation
approach. Comp Env Urban Sys 20:351-366
Lempert RJ (2002) A new decision science for complex systems. PNAS 99:7309-7313
Lempert RJ, Pooper SW, Bankes SC (2003) Robust Decisionmaking. Shaping the Next One
Hundred Years New Methods for Quantitative, Long-Term Policy Analysis: 39-67,
Santa Monica: The RAND Pardee Center
Lodwick WA, Monson W, Svoboda L (1990) Attribute error and sensitivity analysis of map
operations in geographical information systems: suitability analysis. IJGISys 4:413-428
Longley PA, Goodchild MF, Maguire DJ, Rhind DW (2005) Spatial Modeling with GIS.
Geographical Information Systems: Principles, Techniques, Management and
Applications, 2nd Edition: 363-382, New York: John Wiley & Sons
Lowry JH, Miller HJ, Hepner GF (1995) A GIS-Based Sensitivity Analysis of Community
Vulnerability to Hazardous Contaminants on the Mexico/U.S. Border. Photogram Eng
Remote Sens 61:1347-1359
Malczewski J (1999) GIS and Multicriteria Decision Analysis, New York: John Wiley & Sons
Malczewski J (2006) GIS-based multicriteria decision analysis: a survey of the literature.
IJGIS: 20:703-726
17/17
Massam B (1991) The location of waste transfer stations in Ashdod, Israel, using a multi-
criteria Decision Support System. Geoforum 22:27–37
Merkhofer MW (1998) Assessment, Refinement, and Narrowing of Options. Tools to Aid
Environmental Decision Making: 231-281, New York: Springer
Nyerges TL, Ramsey KS, and Wilson MW (2006) Design Considerations for an Internet Portal
to Support Public Participation in Transportation Improvement Decision Making. In
Balram S, Dragićević S, (Eds.), pp. 208-236, Idea Group Inc
Pannell DJ (1997) Sensitivity Analysis of normative economic models: theoretical framework
and practical strategies. Agric Econom 16:139-152
Rinner C, Heppleston A (2006) The Spatial Dimensions of Multi-Criteria Evaluation – Case
Study of a Home Buyer’s Spatial Decision Support System. In GIScience 2006, LNCS
4197, M. Raubal et al. (Eds.), pp. 338-352, Springer-Verlag Berlin Heidelberg
Saisana M, Saltelli A, Tarantola S (2005) Uncertainty and sensitivity analysis techniques as
tools for the quality assessment of composite indicators. J Royal Stat Soc A 168:307-
323
Saltelli A, Chan K, Scott EM (2000) Sensitivity Analysis, New York: John Wiley & Sons
Saltelli A, Tarantola S, Campolongo F, Ratto M (2004) Sensitivity Analysis in Practice: A
Guide to Assessing Scientific Models, New York: John Wiley & Sons
Saltelli A, Tarantola S, Chan K (1999a) A Quantitative Model-Independent Method for Global
Sensitivity Analysis of Model Output. Technometrics 41:39-56
Saltelli A, Tarantola S, Chan K (1999b) A Role for Sensitivity Analysis in Presenting the
Results from MCDA Studies to Decision Makers. J Multi-Criteria Dec Analysis 8:139-
145
Simon H (1960) The new science of management decision, New York: Harper & Row
Talen E (1998) Visualizing fairness: Equity maps for planners. JAPA 64:64-75
Tarantola S, Giglioli N, Jesinghaus J, Saltelli A (2002) Can global sensitivity analysis steer the
implementation of models for environmental assessments and decision-making?
Stochastic Env Res Risk Assess 16:63-76
Thill JC (1999) GIS and Multiple Criteria Decision Making: A Geographic Information
Science Perspective, London: Ashgate
Triantaphyllou E (2000) A Sensitivity Analysis Approach for MCDM Methods. Multi-Criteria
Decision Making Methods: A Comparative Study: 131-175, Kluwer Academic
Publishers
Uran O, Janssen R (2003) Why are spatial decision support systems not used? Some
experiences from the Netherlands. Comp Env Urban Sys 27:511-526
Voogd H (1983) Multicriteria Evaluation for Urban and Regional Planning, London: Pion
Limited
Wolters WTM, Mareschal B (1995) Novel types of sensitivity analysis for additive MCDM
methods. Eur J Oper Res 81:281-290
Top Related