Distance-based and stochastic uncertainty analysis for multi-criteria decision analysis in Excel ...

Environmental Modelling & Software 21 (2006) 1695e1710www.elsevier.com/locate/envsoft

Distance-based and stochastic uncertainty analysisfor multi-criteria decision analysis in Excel

using Visual Basic for Applications

K.M. Hyde*, H.R. Maier

Centre for Applied Modelling in Water Engineering, School of Civil & Environmental Engineering,

The University of Adelaide, Adelaide, SA 5005, Australia

Received 6 January 2005; received in revised form 26 July 2005; accepted 5 August 2005

Available online 21 October 2005

Abstract

A program has been developed in Excel and written in Visual Basic for Applications, which enables a decision maker to examine therobustness of a solution obtained when using multi-criteria decision analysis (MCDA). The distance-based and stochastic uncertainty analysisapproaches contained in the program allow a decision to be made with confidence that the alternative chosen is the best performing alternativeunder the range of probable circumstances. The uncertainty analysis methodology overcomes the limitations of existing sensitivity analysis tech-niques for MCDA by enabling all of the input parameters to be varied simultaneously within their expected ranges. The Weighted Sum Method(WSM) and PROMETHEE are the MCDA techniques available for the user to select in the program. The program is illustrated by applying it toa sustainable water resource development problem in the Northern Adelaide Plains, South Australia.� 2005 Elsevier Ltd. All rights reserved.

Keywords: Multi-criteria decision analysis; Visual Basic; Uncertainty analysis; Excel; Water resources

Software availability

Developers: K.M. Hyde, H.R. MaierMinimum hardware requirements: Intel Pentium II 1.1 GHz,

512 MB RAMSoftware requirements: Operating system Win XP, Microsoft

Excel 2000, @Risk Professional Edition v4.5.2 orabove

Language: EnglishSize: 15 MBAvailability: From the authors upon request for research pur-

poses only

* Corresponding author. Tel.: C61 8 8303 5033; fax: C61 8 8303 4359.

E-mail addresses: [email protected] (K.M. Hyde), hmaier@

civeng.adelaide.edu.au (H.R. Maier).

1364-8152/$ - see front matter � 2005 Elsevier Ltd. All rights reserved.

doi:10.1016/j.envsoft.2005.08.004

1. Introduction

Multi-criteria decision analysis (MCDA) has been utilisedto assist in making complex decisions for a number ofdecades, as it facilitates stakeholder participation and collabo-rative decision-making, does not necessarily require theassignment of monetary values to environmental or socialcriteria, and allows the consideration of multiple criteria inincommensurable units (i.e. combination of qualitative andquantitative criteria). The inherent uncertainties and subjectiv-ities of the input parameters to an MCDA model (i.e. criteriaweights (CWs) and criteria performance values (PVs)) havebeen found to have an influence on the rankings of alternatives(Roy and Vincke, 1981). Despite this, the impact of thevariability of the input parameters has been largely overlookedin studies in which MCDA has been applied (see for exam-ple Duckstein et al., 1994; Flug et al., 2000; Ulvila andSeaver,1982). The effective incorporation, management and

mailto:[email protected]



http://www.elsevier.com/locate/envsoft

1696 K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

understanding of uncertainty in the input parameters remain themost fundamental problems in MCDA (Felli and Hazen, 1998).

Numerous sensitivity analysis methods designed to quantifythe impact of parametric variation on MCDA model outputhave been proposed in the literature, however, these are mainlyfocused on the assessment and influence of the CWs (Barronand Schmidt, 1988; Rios Insua and French, 1991; Woltersand Mareschal, 1995). Few sensitivity analysis methods havebeen developed to assess the impact of PVs on the ranking ofalternatives (Triantaphyllou and Sanchez, 1997) and therefore,the impact of the uncertainty and variability in the criteria PVsis frequently disregarded. The commonly used sensitivity anal-ysis methods are also limited, not only because they are fo-cused on one type of input parameter, but also becausepredominantly only one parameter is varied at a time, whichis inadequate, as it may be the case that the ranking of the alter-natives is insensitive to the variations of some parameters ina set individually, but sensitive to their simultaneous variation.In addition, the frequently applied sensitivity analysis methodsare generally only applicable to certain MCDA techniques.Distance-based and stochastic uncertainty analysis approacheshave been proposed by Hyde et al. (in press, 2004, 2003), Hydeand Maier (submitted for publication), respectively, whichovercome the aforementioned limitations of the commonlyused sensitivity analysis approaches for MCDA, as both ofthe proposed uncertainty analysis methods determine how sen-sitive the ranking of alternatives is to the simultaneous varia-tion of all of the input parameters over their expected range.

There have been numerous specially developed computerpackages that support the application of MCDA methodse.g. DEFINITE (Janssen, 1996), MULINO-DSS (Guipponiet al., 2003; Mysiak et al., 2005), ASSESS (Hill et al.,2005), VIP (Dias and Climaco, 2000), TOPSIS (Deng et al.,2000), MACBETH (Bana e Costa and Vansnick, 1999), andDECISION LAB (Geldermann and Zhang, 2001). In addition,Podinovski (1999) developed a decision support system (DSS)which helps the user to identify candidate solutions from a fi-nite set of decision alternatives when only the upper and lowerbounds of the CWs are specified. Most of these programs,however, are limited in that they only utilise one MCDA tech-nique and the sensitivity analysis (if included) only takes oneform of uncertainty into consideration. This paper presents anddescribes a program that has been developed in MicrosoftExcel that incorporates the ability to undertake deterministicMCDA, the distance-based uncertainty analysis approach ofHyde and Maier (submitted for publication) and the stochasticuncertainty analysis approach of Hyde et al. (2004). Thechoice of uncertainty analysis method may depend on theamount of data available and the output required by the deci-sion maker (DM). The program is illustrated by applying it toa water resource management decision problem which wasassessed using MCDA by Fleming (1999).

2. Program description

The program is written in Visual Basic for Applications(VBA), which is the programming language incorporated in

Microsoft Excel. The advantage of using Microsoft Excel asa development environment is that it provides capabilitiesthat allow for analysis and manipulation of the data and thevisualisation of the results. In addition, Microsoft Excel is fa-miliar, not to mention readily available, to a large majority ofpeople. Consequently, using the program does not necessitatebecoming familiar with a new software environment. Helpfiles are included throughout the program, which provide the-oretical information on the analysis that is implemented, andinformation on how to use the program itself. The structure,methodology and use of the program are illustrated in Fig. 1and described below.

2.1. Decision analysis formulation

2.1.1. Program initialisationThe first stage of any MCDA approach involves the trans-

lation of the decision analysis situation into a set of alterna-tives and criteria, which are generally developed by theactors involved in the decision-making process. The numberof alternatives, criteria and actors, which define the decisionanalysis problem being undertaken, are entered on the Pro-gram Initialisation form, which is shown in Fig. 2. The pro-gram is restricted to assessing decision problems witha maximum number of 30 alternatives and 24 criteria.

The program currently supports two existing MCDA tech-niques: the value focused Weighted Sum Method (WSM)(Janssen, 1996), and the PROMETHEE outranking method(Brans et al., 1986), which are utilised to determine the totalvalue of each alternative for the assigned input parameters.The WSM involves calculating an appraisal score for each al-ternative (V(an)) by multiplying each criterion PV (xm,n) by itsappropriate CW (wm), followed by summing the weightedscores for all criteria as follows (Janssen, 1996):

VðanÞZXM

mZ1

wmxm;n ð1Þ

where m is the criterion number, M is the total number of cri-teria and n is the alternative number.

Alternatively, the basic PROMETHEE methods build a val-ued outranking relation. The preference function associatedwith each criterion gives the degree of preference, expressedby the DM, for alternative a with respect to alternative b oncriterion xj. The function relating the difference in perfor-mance to preference is called the generalised criterion functionin the outranking MCDA technique, PROMETHEE, and is se-lected by the DM. The preference functions, according to thePROMETHEE algorithms, are used to compute the degree ofpreference associated with the best action in pairwise compar-isons. Six types of generalised criterion functions have beensuggested by Brans et al. (1986) with the aim of realisticallymodelling the DMs’ preference, which gradually increasefrom indifference to strict preference, and to facilitate the in-clusion of the inherent uncertainty in the criteria PVs in the de-cision analysis process. Further details of the PROMETHEE

1697K.M. Hyde, H.R. Maier / Environmental Modelling & Software 21 (2006) 1695e1710

SelectUncertainty

AnalysisMethod

InputData

Program Initialisation (1)

- Enter title of the project- Enter number of

alternatives- Enter number of criteria- Enter number of actors- Select PROMETHEE or

WSM MCDA method

Alternative (2) & InputParameter (3) Description

- Enter the name of each of thealternatives

- Enter the name of each of thecriteria and select a preferencedirection for each criterion

Enter Input ParameterValues (4) & (5)

- Enter the criteria performancevalues. If the WSM is selectedin (1) then the user can selectfrom a range of standardisationmethods, if required

- Enter the criteria weights.Select whether the criteriaweights sum to 1, 100 or some ‘other’ amount

Distance-Based (6)

- Select pair of alternatives- Select which actor criteria weights to be used- Select input parameters to vary-

-

- Select distance metric

PERFORM ANALYSIS

Deterministic AnalysisUndertaken using selected MCDA technique and input parameters to obtain total

values and hence rankings of each of the alternatives using each of the actor’s CWs

Stochastic (7)

-

-

-

- Select Monte Carlo Simulation sampling methodPERFORM ANALYSIS

Distance-Based Output

- Euclidean distance for each pair of alternatives andactor’s CWs

- Critical input parameters

Stochastic Output

- Probability of alternative rankings- Range of total values of alternatives- Wilcoxon Rank-Sum Test- Spearman Rank Correlation

Note: () refers to form number, which is where the relevant information is entered by the user

Define input parameter distributions (e.g. fitted, uniform or normal) with either user specified or actual data ranges. In addition, select whether to include CW correlations and specify the data type of the PVs (e.g. continuous or discrete)Select whether to input number of Monte Carlo simulations or to allow to run until convergence

Select Deterministic, Partial Deterministic or Stochastic Analysis

Select “Engine”. If Solver selected, enter number of iterations to undertake. If Genetic Algorithm selected,use default parameters or enter values

Select data range or user range for input parameter constraints

Fig. 1. Program structure.

method are contained in Brans et al. (1986). It should be notedthat Level 1 generalised criterion functions are utilised in theproposed uncertainty analysis approaches for each ofthe criteria, therefore, the user does not need to select the

generalised criterion functions or the associated thresholds.This is because uncertainties associated with the criteria PVsare considered elsewhere in the proposed uncertainty analysisapproaches (refer to Hyde et al., 2003).


The user is required to select one of the techniques de-scribed above on the Program Initialisation form (Fig. 2).For a new decision problem, the user must save the file asa unique workbook before continuing, which enables it to beopened and utilised again, if required, following the comple-tion of the analysis.

2.1.2. Alternative and input parameter descriptionA description of each of the alternatives and the criteria can be

entered in respective forms. The preference direction (i.e. mini-mise or maximise) for each of the criteria must also be selectedby the user on the Criteria Description form, as shown in Fig. 3.

2.1.3. Input parameter valuesThe next step in the decision analysis process is to assess the

alternatives by the criteria that have previously been defined.The PVs may be obtained from models, available data or by ex-pert judgement based on previous knowledge and experience.The type of value assigned to each criterion PV may be quanti-tative or qualitative. The CWs are elicited from the actors usingone of a variety of available techniques. Once the relevant datahave been obtained, the PVs and CWs can be entered, or copiedfrom existing files, by the user into the spreadsheets available(see Fig. 4 for an example of the input PVs worksheet). It shouldbe noted that the PVs are required to be standardised to com-mensurable units when the WSM is used. Therefore, two stand-ardisation methods are available for use if the WSM is theselected MCDA technique and if the PVs are entered in incom-mensurable units. The only additional information required onthe CWs form is the total sum of the CWs.

2.2. Decision analysis

The user is asked to save the input data that have been en-tered, as described above, before continuing with the decisionanalysis process. A selection is then able to be made by theuser between undertaking deterministic analysis, distance-based uncertainty analysis or stochastic uncertainty analysis,as shown in Fig. 5.

2.2.1. Deterministic analysisDeterministic MCDA is the traditional decision analysis

method used to determine the total values of the alternativesand hence the ranking of each alternative for each set ofactor’s CWs, using the selected MCDA technique. If thePROMETHEE method is utilised, the user may choose oneof a number of generalised criterion functions that havebeen defined by Brans et al. (1986) for each criterion. Thisflexibility in the program enables a range of analyses to be un-dertaken, such as comparison of the results of the proposed un-certainty analysis approaches, which only utilise the Level 1generalised criterion functions, with, for example, existingcase studies that have utilised a number of the commonlyused generalised criterion functions. A ranking of the alterna-tives is obtained for each of the actors’ CWs in addition to thetotal value of each of the alternatives, which is displayed intabular and graphical form.

Following the deterministic analysis, the user may chooseto implement either the distance-based uncertainty anal-ysis methodology or the stochastic uncertainty analysisapproach.

Fig. 2. Example of MCDA Uncertainty Analysis Initialisation form.


Fig. 3. Criteria descriptions and preference directions.

2.2.2. Distance-based uncertainty analysis methodology

2.2.2.1. Methodology. The purpose of the distance-based un-certainty analysis approach developed by Hyde and Maier(submitted for publication) is to determine the minimummodification of the MCDA input parameters (i.e. CWs andPVs) that is required to alter the total values of two selectedalternatives (e.g. ax and ay), such that rank equivalence occurs.The minimum modification of the original input parameters isobtained by translating the problem into an optimisation prob-lem and exploring the feasible input parameter ranges. The ob-jective function minimises a distance metric, which providesa numerical value of the amount of dissimilarity between theoriginal input parameters of the two alternatives under consid-eration and their optimised values. Optimised refers to the setof input parameters that is the smallest distance from the orig-inal parameter set, such that when the optimised set is used,

the total values of the two alternatives being assessed areequal. The user may select the Euclidean Distance, de (Eq.(2)), the Manhattan Distance, dm (Eq. (3)), or the KullbackeLeibler Distance (i.e. relative entropy), dk (Eq. (4)):

Min deZ

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXM

mZ1

�w#

jmi�w#jmo

�2Cðx#

mnli � x#mnloÞ

2Cðx#

mnhi� x#mnhoÞ

2

vuut

ð2Þor

Min dmZXM

mZ1

��

w#jmi�w#

jmo

��C��x#mnli � x#

mnlo

��

C��x#

mnhi � x#mnho

�� ð3Þ

Fig. 4. Input criterion performance values worksheet.


or

Min dkZXM

mZ1

w#jmo ln

w#jmo

w#jmi

Cx#mnlo ln

x#mnlo

x#mnli

Cx#mnho ln

x#mnho

x#mnhi

ð4Þ

The objective function equations (Eqs. (2)e(4)) are subjectto the following constraints:

XM

mZ1

wjmiZXM

mZ1

wjmo ð5Þ

V�ay

�optZVðaxÞopt ð6Þ

LLxl%xmnlo%ULxl for mZ1 to M ð7Þ

LLxh%xmnho%ULxh for mZ1 to M ð8Þ

LLw%wjmo%ULw for mZ1 to M; for actor j and LLwO0

ð9Þ

where wjmi is the initial CW of criterion m for actor j, wjmo isthe optimised CW of criterion m for actor j, xmnli is the initialPV of criterion m of initially lower ranked alternative n, xmnlo

is the optimised PV of criterion m of initially lower ranked al-ternative n, xmnhi is the initial PV of criterion m of initiallyhigher ranked alternative n, xmnho is the optimised PV of

Fig. 5. Choice of uncertainty analysis method.

criterion m of initially higher ranked alternative n, de is theEuclidean Distance, dm is the Manhattan distance, dk is theKullbackeLeibler Distance, M is the total number of criteria,V(ay)opt is the modified total value of the initially lowerranked alternative obtained using the optimised parameters,V(ax)opt is the modified total value of the initially higherranked alternative obtained using the optimised parameters,LLxl and ULxl are the lower and upper limits, respectively,of the PVs of each criterion for the initially lower ranked alter-native, LLxh and ULxh are the lower and upper limits, respec-tively, of the PVs of each criterion for the initially higherranked alternative, LLw and ULw are the lower and upper lim-its, respectively, of each of the CWs for the selected actor’sCWs, and # refers to the standardised values of these param-eters (see Eq. (10)). It should be noted that there is only oneterm for the CWs in Eqs. (2)e(4) because the CWs are com-mon to all alternatives.

To ensure that the scale of the input parameters does not in-fluence the optimisation, the values used in the distance metric(i.e. Eqs. (2)e(4)) are standardised using the following formula:

x#mnliZ

xmnli

sXm

ð10Þ

where xmnli# is the standardised initial PV of criterion m of

initially lower ranked alternative n, xmnli is the initial PV ofcriterion m of initially lower ranked alternative n, sXm is thestandard deviation of the set of PVs of criterion m. This formulais also applied to the other parameters in Eqs. (2)e(4),respectively.

In some situations, one alternative will always be superiorto another, regardless of the values the input parameterstake. In this case, the ranking of the alternatives is robust, asit is insensitive to the input parameters. However, in many in-stances, this is not the case, and a number of different combi-nations of the input parameters will result in rank equivalence.By determining the smallest overall change that needs to bemade to the input parameters (e.g. CWs and PVs) in orderto achieve rank equivalence, the robustness of the ranking oftwo alternatives (ax and ay) is obtained. The total values ofthe alternatives are determined using the selected MCDA tech-nique (i.e. WSM or PROMETHEE).

2.2.2.2. Implementation. The user of the program has a numberof parameters to input and selections to make before thedistance-based uncertainty analysis can be run. The pair ofalternatives that the approach is applied to is entered by theuser on the Distance Metrics user form (Fig. 6). The actors’CWs that are to be utilised for the analysis must also be spec-ified. The main advantage of the proposed distance-basedmethodology is the ability to simultaneously vary the CWsand PVs within expected ranges of uncertainty. However, flex-ibility is incorporated into the program by allowing the user toselect that only the CWs or only the PVs are varied while theother parameters remain fixed (Fig. 6).

The minimum and maximum ranges of the input parame-ters also need to be specified by the user, which define the


Fig. 6. Example of the Distance Metric form.

feasible range that each parameter is able to be varied betweenin order to achieve a reversal in ranking (i.e. ay O ax) (Eqs.(7)e(9)). The ranges enable the expected uncertainty and var-iability in the data to be taken into consideration, therefore, ifthe PROMETHEE method is selected, the generalised criteri-on functions are not required to be defined. The range of val-ues (i.e. upper and lower bounds) that must be assigned to eachPV represents the set of possible values for that variable,which can either be based upon knowledge of experts (e.g.select the User Range button on the form shown in Fig. 6and enter the data, as shown in Fig. 7) or the data that areavailable (e.g. select the Data Range button on the DistanceMetrics form (Fig. 6) which uses the minimum and maximumPVs for each criterion based on the values specified for each ofthe alternatives). Similarly, the CW ranges can be defined byeither the DM or actors or, alternatively, actual ranges of theavailable data can be utilised (i.e. the minimum and maximumvalues of the CWs elicited from a range of actors involved inthe decision process).

In order to obtain the robustness of the ranking of each pairof alternatives (i.e. ax and ay) for each actors’ set of CWs, theoptimisation problem given by Eqs. (2)e(10) needs to besolved. Two ‘engines’ are available for selection by the userto minimise the objective function: Solver and Genetic Algo-rithm (GA). Solver is a Microsoft Excel Add-In Function,which is based upon the Generalised Reduced Gradient(GRG2) nonlinear optimisation method. GRG2 works by firstevaluating the function and its derivatives at a starting value ofthe decision vector and then iteratively searching for a bettersolution using a search direction suggested by the derivatives

(Stokes and Plummer, 2004). The GRG2 optimisation methodrequires that starting values are specified for the decisionvariables, therefore, random numbers are generated using theMicrosoft Excel RANDBETWEEN function between thespecified input parameter ranges for the CWs and PVs to beused as the starting values for the optimisation. GRG2 is nota global optimisation algorithm, therefore, to increase thechances of finding global or near-global optima, the optimisa-tion is repeated a number of times using different randomlygenerated starting values (which will be referred to as ‘trials’for the remainder of the paper). This aims to minimise the im-pact that the starting values have on the outcome of theanalysis.

Alternatively, a GA may be selected, which is a heuristiciterative search technique that attempts to find the best solu-tion in a given decision space based on a search algorithmthat imitates Darwinian evolution and survival of the fittestin a natural environment (Goldberg, 1989). An advantageGAs have over traditional optimisation techniques (such asGRG2) is that they do not require the use of a gradient fitnessfunction, only the value of the fitness function itself. Anotheradvantage of GAs is that they search from a population ofpoints, investigating several areas of the search space simulta-neously, and therefore have a greater chance of finding theglobal optimum. Constraints are unable to be incorporated di-rectly in the formulation of the GA, therefore, they are includ-ed in the objective function and multiplied by penalty valuesto discourage the selection of infeasible solutions by reducingtheir fitness. Default penalty values are provided, however, thevalues can be changed by the user if desired. The previously


defined objective functions (Eqs. (2)e(4)) are therefore refor-mulated and defined as:

Minimise P1

��XM

mZ1

wjmi�XM

mZ1

wjmo

��CðP2dÞ

CP3

��VðaxÞopt�V�ay

�opt�� ð11Þ

subject to the constraints given by Eqs. (5)e(9). P1, P2 and P3

are penalty values and d is the distance metric selected by theuser (i.e. de, dm or dk).

The code for the GA utilised in this program has been de-veloped by Michael Leonard of the School of Civil and Envi-ronmental Engineering, University of Adelaide in 2003 usingstandard algorithms as described in Michalewicz (1994) andSetnes and Roubos (2000). Specific details of the GA are in-cluded in Fig. 8.

Both of the engines included in the program have their ad-vantages and disadvantages. The main advantage of usingGRG2 is its speed of arriving at a solution, however, its disad-vantage is that because it is a gradient method, the chances ofa local solution being obtained are high. The advantage of theGA is that it is a global search technique, however, it takesa longer time to converge compared with the GRG2. The pro-cessing time of the optimisation methods is dependent on thecomplexity of the decision problem that is being assessed (i.e.how many alternatives and criteria are involved in the decisionproblem and the ‘robustness’ of the ranking of the alterna-tives). A trade-off is therefore required between the amountof time taken to perform the analysis and the level of certaintythat the minimum distance has actually been obtained (seeHyde and Maier, submitted for publication).

A number of parameters must be selected/defined beforeusing the GA (see Fig. 8) or Solver. The user may either use

Fig. 7. User-defined PV ranges for distance-based uncertainty analysis.

the default Solver and GA Options and parameter values or in-put their own. Information that may aid the selection of theseparameters is included in the Help file. The Solver trial num-ber or the GA generation number is shown in the Excel taskbar during the analysis so that the user may monitor the prog-ress of the analysis.

More detailed information on the methodology and the pro-gram can be found in Hyde and Maier (submitted for publica-tion, 2004) and Hyde et al. (in press).

2.2.2.3. Interpretation of results. The output of the distance-based uncertainty analysis is the minimum distance metric foreach pair of alternatives, which can be summarised in a matrix.A non-feasible or a very large value of the distance metric be-tween two alternatives informs the DM that one alternativewill predominantly be superior to another, regardless of the in-put parameter values selected between the specified ranges.Conversely, if the distance is small, small changes in the inputparameters will result in rank equivalence and the ranking of thealternatives can therefore be concluded as being sensitive to theinput parameter values. The decision-making process can beimproved considerably by identifying critical input parametersand then re-evaluating their values more accurately. The mostcritical inputs can be identified by examining the relative andabsolute change in input parameter values, as follows:

Absolute DxmnlZxmnlo � xmnli ð12Þ

Fig. 8. Genetic Algorithm Input Parameters form.


Relative DxmnlZxmnlo� xmnli

xmnli

100% ð13Þ

It should be noted that Eqs. (12) and (13) can also be usedto determine the most critical PVs of the initially higherranked alternative, as well as the most critical CWs. The inputparameters that exhibit the smallest relative change in value toachieve rank equivalence between two alternatives are mostcritical to the reversal in ranking. The results therefore providethe DM with further information to aid in making a finaldecision.

2.2.3. Stochastic uncertainty analysis approachA shortcoming of the distance-based approach is that it

does not consider the likelihood that the input parameter val-ues are changed by a certain amount within their specified un-certainty ranges. If sufficient information is available to defineprobability distributions for likely values of each input param-eter, the stochastic uncertainty analysis approach can be usedto obtain additional information on the likelihood of alterna-tives achieving a particular ranking.

2.2.3.1. Methodology. As was the case with the distance-baseduncertainty analysis approach, the reliability based approachdeveloped by Hyde et al. (2004) enables the DM to examinethe robustness of a solution, allowing a decision to be madewith confidence that the alternative chosen is the best perform-ing alternative under the range of probable circumstances. Theproposed stochastic method involves defining the uncertaintyin the input values using probability distributions, performinga reliability analysis by Monte Carlo Simulation (MCS) andundertaking a significance analysis using the Spearman RankCorrelation Coefficient.

The approach involves utilising existing MCDA techniquesto determine the total value of each alternative, however, the ad-vancement is in the application of Monte Carlo Simulation(MCS) (Kottegoda and Rosso, 1997) to enable repetition ofthe selected MCDA method with the range of possible inputvalues defined by probability distributions. Three key stagesare involved in the proposed reliability analysis process: (i)the input values are randomly sampled from their respectiveprobability distributions, assuming the CWand PV distributionsare independent, while maintaining the correlation structure ofthe CWs, (ii) the randomly drawn vector of CWs is normalised,as the sum of all elements of the weight vector must equal theoriginal total sum of the CWs (e.g. 1 or 100) (Janssen, 1996),and (iii) the selected MCDA technique is applied to determinethe total value of each alternative for that realisation (i.e. withthe randomly drawn vector of PVs and normalised CWs).

The proposed approach provides the benefits of jointlyvarying the PVs and CWs, including any correlations betweenthe CWs, in the analysis and allowing all expected uncertaintyand subjectivity in the CWs and PVs to be incorporated so thatposterior analysis is not required to be undertaken. CW corre-lations, obtained through correlation analysis, are incorporatedin the distributions to ensure that sampling from the CW

probability distributions represents the actual assignment ofCWs by the actors. Further details of the methodology areavailable below and in Hyde et al. (2004).

2.2.3.2. Implementation. Flexibility is a key component of theprogram, with the user having many options to assess the var-ious sources of input parameter uncertainty using the stochas-tic uncertainty analysis approach. The user may either selectpartial deterministic (distributions for either CWs or PVsonly) or full stochastic analysis (distributions for both CWsand PVs) on the Stochastic Uncertainty Analysis user form(Fig. 9).

In the situation where a relatively large number of actorshave been included in the decision process, the CWs of theactors can be considered as a representative sample of theCWs of a population of stakeholders. The user may thereforedecide to portray these preferences by fitting a distribution tothe actual CWs, ensuring that all of the information obtainedfrom the actors is explicitly incorporated in the decision-making process. The Microsoft Excel @Risk add-in program(Palisade, 2000) is used to fit distributions to the data and good-ness of fit statistics are reviewed to determine how representa-tive the fitted distributions are of the actual sets of CWs elicitedfrom the actors. The user can also elect to undertake a correla-tion analysis, with the results incorporated in the distributions,to ensure that sampling from the CW probability distributionsrepresents the actual assignment of CWs by actors.

In the circumstance where a small number of actors are in-volved in the decision process, and there are consequentlyinsufficient CWs available to fit a representative distribution,the user can select either a normal or uniform distribution toenable uncertainty and subjectivity in the weights to be incor-porated in the analysis. The user must then also characterisethe uniform distributions, by defining the upper and lowerbounds of the CWs using either actor specified limits orbounds based upon the actual CWs available.

The uncertainty, imprecision and variability in the quantita-tive PVs can be represented by continuous probabilitydistributions such as uniform or normal. A range of values(i.e. upper and lower bounds for uniform distributions) mustbe assigned to each PV by the user, representing the set of pos-sible values for that variable, which can either be based uponknowledge of the experts or the data that are available. A dis-crete uniform distribution can be utilised for qualitative PVs,which is characterised by defining upper and lower limits ofeach PV (e.g. a groundwater rise of ‘medium’ may be between‘medium low’ and ‘medium high’, or in an integer scalebetween 2 and 4). The user must also select whether eachcriterion belongs to a discrete or continuous data type.

The user is able to elect whether the Monte Carlo Simula-tion (MCS) will run until convergence of the input distributionsor until a number of user specified iterations are completed.The user may also choose whether random sampling orLatin-Hypercube Sampling is utilised. Information is providedin the Help file to aid the user in making a decision if they areunfamiliar with MCS. The Microsoft Excel @Risk add-inprogram (Palisade, 2000) is also used to undertake the MCS.


Fig. 9. Example of Stochastic Uncertainty Analysis form.

2.2.3.3. Interpretation of results. The results of the reliabilityanalysis provide the DM with valuable information, includingdistributions of the total values for a single alternative or thedifference between total values for competing alternatives.Knowledge of the likelihood of the total value over the entirerange of possible input values enables the DM to better assessthe risk of an adverse outcome or select an alternative basedupon the likelihood that its total value will exceed that of itscompetitor by a specified amount. The probability that an al-ternative an receives rank r, based on all probable criteria inputparameters, is also available to the DM, in order to assess therobustness of a solution. The DM may also review the inputparameter values that result in the probabilities of one alterna-tive outranking another alternative. The significance of thedifference between output distributions is obtained by under-taking the Wilcoxon RankeSum Test (Kottegoda and Rosso,1997) for each pair of alternatives.

To facilitate interpretation of the results, significance anal-ysis is used to identify the relative contribution that each inputparameter (i.e. each CW and PV) has in determining the total

value of an alternative. The most significant inputs to the anal-ysis are determined using the Spearman Rank Correlation Co-efficient, R (Kottegoda and Rosso, 1997). The value of Ralways lies between �1 and C1, where a value of �1 orC1 indicates perfect association between the parameters, theplus sign occurring for identical rankings and the minus signoccurring for reverse rankings. When R is close to zero, it isconcluded that the variable under consideration (i.e. a particu-lar CW or PV) does not have a significant impact on theranking of the alternative. Once the most important inputparameters are identified from the significance analysis, re-sources can be concentrated on characterising their uncertaintyif further analysis is required to arrive at a final decision.

3. Case study

The use and the potential benefits of the program areillustrated by undertaking analysis of a study carried out byFleming (1999) and details of the study, the problem formula-tion and the results are presented below.


3.1. Decision analysis formulation

The Northern Adelaide Plains (NAP) is an important horti-cultural region on the outskirts of Metropolitan Adelaide,South Australia. The most important water resource in the re-gion is groundwater, however, the groundwater has been usedat a rate that far exceeds the rate of natural replenishment,threatening the sustainability of the resource and the horticul-tural industry that depends upon it. One hundred andthirty-nine actors were involved in elaborating the three devel-opment alternatives and 10 evaluation criteria. This informa-tion has been entered in the MCDA Uncertainty AnalysisInitialisation form, as shown in Fig. 2, in addition to selectingWSM as the MCDA technique to utilise for the analysis, as itis a simple and commonly used MCDA technique. As it isa new decision problem, the relevant data must be entered be-fore the analysis can be undertaken, therefore, after pressingthe ‘Input Data’ button, the Alternatives Description form be-comes visible. A description of the three alternative watermanagement options, including ‘Business As Usual’ (Alterna-tive 1), tertiary wastewater treatment for irrigation of horticul-ture (Alternative 2) and water conservation and reuse(Alternative 3), is entered on this form. These alternativeswere assessed by Fleming (1999) to determine a developmentstrategy that offers a more sustainable future for the area. Thenext step is to enter a description of the 10 criteria, includingenvironmental, social and economic factors, that were selectedto evaluate the three alternatives, on the available form, asshown in Fig. 3. The criteria preference directions are also en-tered on this form. After pressing the ‘Continue’ button, thecriteria PVs are entered into the worksheet available(Fig. 4), which were evaluated by experts, and involved theinterpretation of the documented development alternatives, to-gether with the experts’ own personal experience and under-standing of the issues. A standardisation method is notrequired, as the PVs are already in commensurable units. Inthe next window, the CWs are entered for each of the 139actors, which were elicited via a program of community con-sultation. This completes the data entry process, and after the‘Next’ button is pressed, the user must save the input data be-fore selecting which type of decision analysis to undertake, asshown in Fig. 5.

3.2. Decision analysis

Each of the decision analysis methods available for use inthe program is described below utilising the case study datathat have been entered, as described in Section 3.1.

3.2.1. DeterministicInitially, deterministic analysis is undertaken by clicking

the ‘Deterministic’ button on the form shown in Fig. 5. Theprogram calculates the total value of each alternative and foreach actor’s set of CWs using the WSM. The ranking of thealternatives is also determined. The results of the deterministicanalysis are able to highlight the different priorities of the ac-tors, illustrating common agreements and conflicts. The results

of this analysis are illustrated in Fig. 10 and for this case study,it is evident that all of the actor’s preference values result inthe same ranking where Alternative 3 is ranked highest, Alter-native 2 is ranked second and Alternative 1 is ranked third.

3.2.2. Distance-based uncertainty analysis approachIf the user decides to undertake distance-based uncertainty

analysis by clicking the Distance Analysis button on the formshown in Fig. 5, the form shown in Fig. 6 is displayed and therelevant data must be entered by the user before the analysiscan commence. The user must enter the pair of alternativesthat the analysis will be undertaken for and which set of actorCWs to utilise. To illustrate the methodology, both the CWsand PVs were selected to be altered simultaneously. Therefore,the feasible input parameter range for the actors’ CWs and thePVs of the alternatives must be specified, as defined by Eqs.(7)e(9). No information was provided by Fleming (1999) onthe uncertainty associated with the criteria PVs or the CWs,therefore, the upper and lower limits of the input parameterswere assigned to enable the proposed approach to be illustrated.The data entry form for the ranges of the PVs is shown inFig. 7. A distance metric must also be selected from the threeavailable in the program and for this case study the EuclideanDistance has been chosen as it is one of the most commonlyused distance metrics.

Once the alternatives and input parameters of interest havebeen selected, the user can select and execute one of the twoavailable solution methods. The optimisation was initially un-dertaken using the Microsoft Excel Add-In Solver Functionwith 100 trial solutions in order to test the complexity of thesolution space with the aim of determining whether it is bestto use the GRG2 (Solver) or GA to undertake the complete

0

0.2

0.4

0.6

0.8

1

Alt 1 Alt 2 Alt 3

Alternatives

Tot

al V

alue

Fig. 10. Deterministic MCDA output e total values obtained for each alterna-

tive using the CWs elicited from each actor.

0.20

0.25

0.30

0.35

0.40

0.45

0.50

0 20 40 60 80 100

Trial Number

Euc

lidea

n D

ista

nce

Fig. 11. Final Euclidean Distances obtained for each trial number using GRG2

and comparing Alternatives 3 and 2 with Actor 1 CWs.


analysis. Different Euclidean Distances were obtained witheach of the random starting values when using GRG2 (as illus-trated in Fig. 11), which indicates that the solution space iscomplex and that the starting values have an impact on the so-lution. Therefore, the GA was selected as the preferred optimi-sation method, as it is more likely to arrive at a global solutionwhen the solution space is complex.

The performance of a GA is a function of a number of user-defined parameters, including penalty function values, muta-tion and crossover probabilities, generation and populationnumbers (Raju and Kumar, 2004). The optimal values of theseparameters to be used during the analysis were obtained byconsidering suggested values published in the literature(Dandy and Engelhardt, 2001; Mirrazavi et al., 2001; Rajuand Kumar, 2004). Sensitivity analysis was also undertakenby varying the size of the population (100e800) and the num-ber of generations (2500e10,000) to ascertain the impact theseparameters have on the time taken for the analysis to be under-taken, in addition to their influence on the final value of theselected distance metric. The optimal parameters obtained,which were utilised in the analysis, are contained in Fig. 8.

The Euclidean Distances obtained that result in rank equiv-alence for each pair of alternatives using Actor 1 CWs andvarying the CWs and PVs are summarised in Table 1. Onlythe results utilising Actor 1 CWs are presented due to spacelimitations. The initially lower ranked alternatives are listeddown the leftmost column of Table 1, in rank order, whilethe initially higher ranked alternatives are listed across thetop row, in rank order. From Table 1, it can be seen that thesecond ranked alternative (Alt 2) is the most likely to outrankthe highest ranked alternative (Alt 3) when the uncertainty inthe CWs and the PVs is considered in the analysis, as it has thesmallest Euclidean Distance (i.e. 0.291). An additional outputfrom the distance-based uncertainty analysis approach is the

Table 1

Euclidean Distances of pairs of alternatives for Actor 1 CWs using a GA and

user ranges for CWs and PVs

Initially higher ranked alternatives

Alt 3 (R1) Alt 2 (R2) Alt 1 (R3)

Initially lower

ranked

alternatives

Alt 3 (R1) e NA NA

Alt 2 (R2) 0.291 e NA

Alt 1 (R3) 0.571 0.410 e

most significant input parameters. From Table 2 it can beseen that the input parameters that have the most impact onthe analysis are PV3 and PV9 of Alternative 3, as they havethe smallest relative change. A larger variation in the CWsis required for them to have a significant impact on the out-come of the analysis.

3.2.3. Stochastic uncertainty analysis approachIf the user elects to undertake an alternative uncertainty

analysis approach and clicks the Stochastic Analysis buttonon the form shown in Fig. 5, then the Stochastic UncertaintyAnalysis form, shown in Fig. 9, is displayed. In order to dem-onstrate the proposed methodology, stochastic analysis is se-lected and therefore distributions for both the CWs and PVsneed to be defined. Decisions are then required to be madeby the user, based on the available input parameter data, re-garding the types of distribution to utilise. This is undertakenon the form by clicking the appropriate check boxes.

One hundred and thirty-nine actors were involved in the de-cision analysis undertaken by Fleming (1999), therefore, theCWs elicited from the actors could be regarded as a sampleof the CWs that would be obtained from the population ofstakeholders. It was therefore decided to fit distributions tothe CWs and the type of distributions obtained for each crite-rion and the goodness of fit statistics are summarised in Table 3.Correlation analysis of the elicited CWs was chosen to beundertaken, by clicking the relevant check box, as shown inFig. 9, to ensure that the sampling from the CW distributionsreplicated the preference information provided by the actors.The fitted distribution obtained for CW3, compared to the av-erage CW (which was used in the analysis undertaken byFleming, 1999), is shown in Fig. 12, which demonstrates therange of actor preference information that is disregarded ifonly the average CW is utilised.

As stated above, no information is available on the uncer-tainty with regard to the criteria PVs, therefore, a uniform dis-tribution was assumed and the upper and lower limits of thedistributions were taken to be the same as those utilised inthe distance-based uncertainty analysis approach (i.e. betweenthe user specified ranges). The MCS was selected to be rununtil the input parameter distributions converged (i.e. whenthe sampled values approximated the distributions of the inputvalues) and the sampling method utilised was Latin-Hypercube

Table 2

Changes in input parameter values for Alternative 3 Z Alternative 2, CWs of Actor 1

Criteria CWs of Actor 1 PVs of Alt 3

wmi wmo Absolute D %Relative D xmi xmo Absolute D %Relative D

1 0.223 0.250 �0.027 �12 0.850 0.740 0.110 13

2 0.173 0.100 0.073 42 0.700 0.660 0.040 6

3 0.084 0.050 0.034 40 0.750 0.730 0.020 3

4 0.200 0.200 0.000 0 0.700 0.620 0.080 11

5 0.189 0.120 0.069 36 1.000 0.950 0.050 5

6 0.052 0.160 �0.108 �210 0.600 0.530 0.070 12

7 0.019 0.010 0.009 46 0.700 0.700 0.000 0

8 0.010 0.011 �0.001 �16 0.360 0.350 0.010 3

9 0.021 0.090 �0.069 �339 0.460 0.420 0.040 9

10 0.031 0.010 0.021 68 1.000 1.000 0.000 0


Sampling. The analysis commences when the ‘Perform Analy-sis’ button is clicked. The MCS ran for 1500 iterations, whichwas when convergence of the input parameter distributions wasachieved. The time taken for the simulations to complete run-ning was approximately 1 h on a 1400 MHz Intel Pentium with504 MB of RAM.

Following completion of the reliability analysis, the DM isable to assess the output and obtain information, such as theprobability that each of the alternatives will achieve a rankingof 1e3, with the knowledge that the expected range of valuesof the CWs and PVs has been incorporated, the input parame-ters have been sampled from their respective distributions ran-domly while maintaining any CW correlations, and the inputparameters have been varied jointly. The results obtained indi-cate that there is a 98.9% chance that Alternative 3 is the bestalternative when all of the expected values are taken into ac-count (Table 4). The results of the reliability analysis canalso be presented as cumulative distributions of the total valuesof the alternatives over the range of expected input parameters,with Fig. 13 illustrating the probability that an alternative willhave a total value less than or equal to any variable total valuefor each of the alternatives. Fig. 13 also shows the range of pos-sible total values that each alternative may attain, which isdemonstrated to be extremely extensive for this case study.

The single total value obtained for each alternative usingthe average CWs is also illustrated in Fig. 13, in addition toa cumulative distribution of the total values of the alternativesobtained by using each of the 139 sets of CWs in the WSM.The cumulative distribution obtained using the deterministic

Table 3

Fitted criteria weight distributions and goodness of fit results

Criteria Distribution

type

Chi-squared

statistic

KolmogoroveSmirnov

statistic (KeS)

CW1 Lognorm 2 10.7 0.055

CW2 Loglogistic 9.6 0.073

CW3 Weibull 2.5 0.032




CW7 Weibull 4.2 0.055

CW8 InvGauss 3.6 0.054

CW9 Pearson 5 4.9 0.055

CW10 InvGauss 18.2 0.078

0

1

2

3

4

5

6

7

8

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

Weight

Fre

quen

cy

Average CW used inanalysis by Fleming(1999)

WeibullDistribution

Fig. 12. Comparison of fitted distribution and average actor values for CW3.

CWs enables the DM to have more confidence in the selectionof Alternative 3 as the optimal alternative, compared to onlyundertaking the analysis using the average CW, as none ofthe CWs elicited from the actors’ results in a reversal in theranking of the alternatives. However, the impact of the uncer-tainty and variability in the criteria PVs remains unknown,which can be determined by application of the proposed sto-chastic approach.

The significance analysis was undertaken to determinewhich input parameters are of most importance to the rankingof each of the alternatives. The results of the significance anal-ysis for Alternative 3 are shown in Fig. 14, which indicatesthat CW8, PV9, PV8 and PV7 are the most sensitive input pa-rameters to the ranking of Alternative 3.

3.3. Discussion

The use of the program developed in Excel and written inVBA has been illustrated by applying it to the case study orig-inally assessed by Fleming (1999). The overall aim of MCDAis to extract, in a clear and transparent way, the informationneeded for a sound and responsible decision from a complexconflict situation, which this program is able to help facilitate.The decision analysis process has been shown to be expeditedand enhanced by its use, with each user form providing a stepby step process of elucidating the decision analysis problem.The special emphasis of the program is on the considerationof uncertainty, and application of the program to the case studyhas demonstrated how valuable information can be obtainedthrough undertaking the two proposed uncertainty analysis ap-proaches that are included in the program. A summary of theresults obtained to help inform the decision problem assessedby Fleming (1999) using the program is contained below.

Fleming (1999) utilised an average of the 139 actors’ CWsand deterministic PVs when analysing the three management

Table 4

Probability matrix that Alternative m obtains rank r

Rank Alt 1 (%) Alt 2 (%) Alt 3 (%)

1 0.00 1.07 98.93

2 0.13 98.80 1.07

3 99.87 0.13 0.00

0.0

0.2

0.4

0.6

0.8

1.0

0.1 0.3 0.5 0.7 0.9

Total Value

Cum

ulat

ive

Dis

trib

utio

n

Stochastic Deterministic Average CW

Alt 3

Alt 1Alt 2

Fig. 13. Cumulative frequency distribution for results of alternatives.


0.3190.170

0.1370.075

-0.049-0.050

-0.084-0.168

-0.199-0.238

-0.251-0.253

-0.273-0.284-0.285

-0.315-0.321

-0.337-0.342

-0.382

-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4

Alt3CW5Alt3CW1Alt3CW3Alt3CW1Alt3CW7Alt3CW4Alt3CW2Alt3CW9Alt3CW6Alt3PV4Alt3PV3Alt3PV2Alt3PV1Alt3PV5Alt3PV6Alt3PV1Alt3PV7Alt3PV8Alt3PV9

Alt3CW8

Inpu

t P

aram

eter

Correlation Coefficient

Fig. 14. Spearman Rank Correlation Coefficients for Alternative 3.

alternatives for the Northern Adelaide Plains. In contrast, thedeterministic analysis available in the program undertakesthe decision analysis for each actor’s CWs and not just anaverage CW, which enables the different priorities of the ac-tors to be highlighted, by illustrating common agreementsand conflicts. In this particular case, each of the actors’ pref-erences resulted in the same rank ordering of the alternatives.

No analysis of the uncertainty of the input parameters wasundertaken by Fleming (1999) and therefore no information isprovided to the DM on how robust the ranking of the three al-ternatives is to changes in the values of the input parameters.Two uncertainty analysis methods are available for use in theprogram, the selection of which is dependent on the informa-tion available and the output information desired by the DM. Iflimited information is available on the expected uncertainty ofthe input parameters, the user would select the distance-basedapproach. The proposed distance-based uncertainty analysisapproach determines the parameter combinations that are crit-ical in reversing the ranking of two selected alternatives, there-by allowing the DM to test the robustness of the decisionoutcomes to variations in the input data. The analysis is some-what analogous to a traditional sensitivity analysis, where thebehaviour of the ranking of alternatives is explored within theexpected range of CWs and PVs. The proposed approach,however, provides the benefits of jointly varying the CWsand PVs to obtain a single measure of robustness, which isthe Euclidean Distance in this case. The relatively large Eu-clidean Distances obtained for each pair of alternatives suggestto the DM that the ranking of the alternatives is robust and thatlarge changes in the input parameters are required for a reversalin the ranking to occur. The ability of the method to identifythe most critical input parameters also provides the DM withvaluable information which can provide direction for furtheranalysis (i.e. obtaining more information so that the uncertain-ty intervals can be refined) or confidence that a large change inthe input parameters is required before a reversal in the rank-ing occurs, which is the case for this particular case study. Theresults of the analysis also illustrate that the criteria PVs playan equally important role in the decision analysis process asthe CWs, with the majority of the most critical input

parameters being PVs. This is an important result, as existingsensitivity analysis methods generally ignore the uncertaintyin the PVs or consider uncertainty only in CWs or PVs, butnot both simultaneously. The implication of this is that a com-plete and accurate understanding of the uncertainty associatedwith the ranking of the alternatives is not obtained when exist-ing sensitivity analysis methods are used.

A shortcoming of the distance-based approach is that itdoes not consider the likelihood that the input parametervalues are changed by the amounts indicated to require rankreversal. If sufficient information is available to define proba-bility distributions for likely values of each input parameter,the stochastic uncertainty analysis approach can be used to ob-tain additional information on the likelihood of alternativesachieving a particular ranking. Another shortcoming of thedistance-based approach is that only one set of actor’s CWscan be considered at a time. A benefit of the stochastic uncer-tainty analysis approach is demonstrated in this case study, asall of the CWs elicited from the actors were able to be includ-ed in the analysis by fitting a distribution to these values. Theexpected uncertainty and variability in the PVs were incorpo-rated by assigning uniform distributions to the PVs. The re-sults of the stochastic uncertainty analysis provide the DMwith confidence that Alternative 3 is the best performing alter-native under the range of possible values, as it has a probabilityof being the highest ranked alternative of 98.9%. This meansthat there is only a small combination of parameter valueswithin the anticipated range of values that will result in rankreversal between the alternatives, which confirms the outputof the distance-based approach, however, providing more in-formation on the likelihood of this occurring. The output ofthe analysis provides the DM with information to aid discus-sion regarding the alternatives with the actors and guidanceas to the most important input parameters which could bethe focus of further data analysis and reassessment, if required.The output of the Spearman Rank Correlation analysis alsoconfirms the output of the distance-based approach, withchanges in the PVs found to be the most critical for rank rever-sal to occur, although this is unlikely to occur in this casestudy.


4. Summary and conclusions

The output of MCDA techniques (such as WSM andPROMETHEE) depends critically on the data input, therefore,the values of the input parameters should ideally be precise.However, in reality, available information is often uncertain,which cannot be taken into account in existing MCDA techni-ques and associated software programs. Existing sensitivityanalysis methods developed for MCDA are generally unableto adequately consider the various sources of uncertainty, asthey involve the systematic variation of one input parameterat a time, while the remaining parameters are fixed and mainlyfocus on the influence of the CWs.

The main focus of this paper is the introduction of a program,which has been developed in Microsoft Excel using VBA, andextends the analysis of existing softwares for MCDA based onWSM and PROMETHEE. The emphasis in designing the pro-gram was to support two proposed uncertainty analysis meth-ods: a distance-based uncertainty analysis approach (Hydeand Maier, submitted for publication) and a stochastic uncer-tainty analysis approach (Hyde et al., 2004). These two im-proved uncertainty analysis approaches enable the uncertaintyand variability in the input parameters to be included in theanalysis concurrently, as it may be the case that a decision is in-sensitive to the variation of some parameters individually, butsensitive to their simultaneous variation. Flexibility has beenincorporated into the program, however, by enabling individualinput parameters to be varied, if required. The distance-baseduncertainty analysis approach is able to determine the parame-ter combinations, and the distances between them, that are crit-ical in reversing the ranking of two selected alternatives.Alternatively, the stochastic uncertainty analysis approachallows the DM to gain an enhanced understanding of the riskassociated with selecting an alternative and is able to selectan alternative based on the likelihood that it will outrank theother alternatives considered when all of the estimated uncer-tainties are taken into account. The program produces resultsin a way that is transparent and easy to understand in order topromote acceptance among the actors, as well as the otherstakeholders. Information on the robustness of the ranking of al-ternatives to variation in all of the input parameters is thereforeprovided to the DM using the program. Help files are includedto aid in the utilisation of the program and to inform the user ofthe various technical aspects of the program, however, the tar-geted user of the program is the expert MCDA analyst/facilita-tor. It is believed, though, that by enhancing and extendingsome features of the program, the program could be utilisedby DMs and actors without the aid of an expert to provideenhanced support for decision analysis.

The program has been tested successfully on a number ofcase studies, one of which is presented in this paper (the casestudy undertaken by Fleming, 1999). As shown, a user whois familiar with other Windows applications can easily usethe program. The case study presented demonstrates how allof the available information, including uncertainty, can be in-corporated in the analysis concurrently, providing additionalinformation on the robustness of the alternatives to variation

in the input parameters and allowing confidence to be placedin the outcome of the analysis. If limited information is avail-able, the user is able to use the distance-based uncertainty anal-ysis method, which demonstrated that large changes in theparameter values are required for a reversal in the ranking ofthe alternatives to occur. The outcomes of this analysis wereconfirmed by performing a stochastic uncertainty analysis,which requires distributions to be defined for the input para-meters. The output of the stochastic analysis provides theDM with more information than the distance-based approach,as it shows the likelihood that one alternative will outrankanother (e.g. 1.07% chance that Alternative 2 will outrank Al-ternative 3). In addition, the significance analysis availablewith both uncertainty analysis methods informs the DM ofthe input parameters that have the most influence on the rank-ing of each alternative and therefore information is able to beprovided to the DM to direct future analysis and refinementof the input parameter values, if required. The Fleming(1999) case study demonstrated the importance of incorporat-ing the uncertainty of both the PVs and the CWs in the analysis,as the PVs were found, by both uncertainty analysis methods,to have the largest impact on the ranking of the alternatives.

The program has been applied to a water resource allocationcase study in this paper, but the scope of decision problems thatcould be analysed using the program presented in this paper isvery broad: it can be applied to any kind of usage conflictsbetween natural resources and economic development.

Acknowledgements

The authors wish to thank the Australian ResearchCouncil, Department for Water, Land, Biodiversity and Con-servation (DWLBC) and the Office of Economic Develop-ment, State Government of South Australia for funding toundertake this research. In addition, the authors wish to thankMichael Leonard from the School of Civil and EnvironmentalEngineering at the University of Adelaide for his work on thedevelopment and provision of the code for the GA used in theprogram.

Appendix A

Acronyms

CWs Criteria weights

DM Decision maker

GA Genetic Algorithm

GRG2 Generalised Reduced

Gradient nonlinear optimisation method

MCDA Multi-criteria decision analysis

MCS Monte Carlo Simulation

NAP Northern Adelaide Plains

PROMETHEE Preference ranking

organisation METHod for enrichment evaluations

PVs Performance values

TOPSIS Technique for preference

by similarity to the ideal solution

VBA Visual Basic for Applications

WSM Weighted Sum Method


References

Bana e Costa, C.A., Vansnick, J.C., 1999. The Macbeth Approach: Basic

Ideas, Software and an Application. Kluwer Academic Publishers.

Barron, H., Schmidt, P., 1988. Sensitivity analysis of additive multiattribute

value models. Operations Research 36 (1), 122e127.

Brans, J.P., Vincke, P., Mareschal, B., 1986. How to select and how to rank

projects: the PROMETHEE method. European Journal of Operational

Research 24 (2), 228e238.

Dandy, G.C., Engelhardt, M., 2001. Optimal scheduling of water pipe replace-

ment using genetic algorithms. Journal of Water Resources Planning and

Management 127 (4), 214e223.

Deng, H., Yeh, C.H., Willis, R.J., 2000. Inter-company comparison using

modified TOPSIS with objective weights. Computers & Operations

Research 27 (10), 963e973.

Dias, L.C., Climaco, J.N., 2000. Additive aggregation with variable interde-

pendent parameters: the VIP analysis software. Journal of the Operational

Research Society 51, 1070e1082.

Duckstein, L., Treichel, W., Magnouni, S.E., 1994. Ranking of groundwater

management alternatives by multicriterion analysis. Journal of Water

Resources Planning and Management 120 (4), 546e565.

Felli, J.C., Hazen, G.B., 1998. Sensitivity analysis and the expected value of

perfect information. Medical Decision Making 18 (1), 95e109.

Fleming, N., 1999. Sustainability and Water Resources Management for the

Northern Adelaide Plains, South Australia. The University of Adelaide.

Flug, M., Seitz, H.L.H., Scott, J.F., 2000. Multicriteria decision analysis

applied to Glen Canyon Dam. Journal of Water Resources Planning and

Management 126 (5), 270e276.

Geldermann, J., Zhang, K., 2001. Software review: decision lab 2000. Journal

of Multi-Criteria Decision Analysis 10 (6), 317e323.

Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization and

Machine Learning. Addison-Wesley, Reading.

Guipponi, C., Mysiak, J., Fassio, A., Cogan, V., 2003. MULINO-DSS: a com-

puter tool for sustainable use of water resources at the catchment scale.

Mathematics and Computers in Simulation 64, 13e24.

Hill, M.J., Braaten, R., Veitch, S.M., Lees, B.G., Sharma, S., 2005. Multi-

criteria decision analysis in spatial decision support: the ASSESS

analytic hierarchy process and the role of quantitative methods and spatially

explicit analysis. Environmental Modelling & Software 20, 955e976.

Hyde, K.M., Maier, H.R., 2004. Distance-based uncertainty analysis for multi-

criteria decision analysis in Excel using Visual Basic for Applications. In:

Managing Uncertainty in Decision Support Models Conference, Coimbra,

Portugal, 22e24 September, 2004.

Hyde, K.M., Maier, H.R. A distance-based uncertainty analysis approach to

multi-criteria decision analysis using Genetic Algorithms. European

Journal of Operational Research, submitted for publication.

Hyde, K.M., Maier, H.R., Colby, C.B., 2003. Incorporating uncertainty in the

PROMETHEE MCDA method. Journal of Multi-Criteria Decision

Analysis 12, 245e259.

Hyde, K.M., Maier, H.R., Colby, C.B., 2004. Reliability-based approach to

multicriteria decision analysis for water resources. Journal of Water

Resources Planning and Management 130 (6), 429e438.

Hyde, K.M., Maier, H.R., Colby, C.B. A distance-based approach to undertak-

ing sensitivity analysis of criteria weights for water resources decision

making. Journal of Environmental Management, in press.

Janssen, R., 1996. Multiobjective Decision Support for Environmental

Management. Kluwer Academic Publishers, Netherlands.

Kottegoda, N.T., Rosso, R., 1997. Statistics, Probability, and Reliability for

Civil and Environmental Engineers. McGraw-Hill, Singapore.

Michalewicz, Z., 1994. Genetic Algorithms C Data Structures Z Evolution

Programs. Springer-Verlag, New York.

Mirrazavi, S.K., Jones, D.F., Tamiz, M., 2001. A comparison of genetic and

conventional methods for the solution of integer goal programmes.

European Journal of Operational Research 132 (3), 594e602.

Mysiak, J., Guipponi, C., Rosato, P., 2005. Towards the development of a de-

cision support system for water resource management. Environmental

Modelling & Software 20, 203e214.

Palisade, 2000. @Risk e Risk Analysis Add-In for Microsoft Excel. 4.5.2 e

Professional Edition.

Podinovski, V.V., 1999. A DSS for multiple criteria decision analysis with im-

precisely specified trade-offs. European Journal of Operational Research

113, 261e270.

Raju, K.S., Kumar, D.N., 2004. Irrigation planning using Genetic Algorithms.

Water Resources Management 18 (2), 163e176.

Rios Insua, D., French, S., 1991. A framework for sensitivity analysis in dis-

crete multi-objective decision-making. European Journal of Operational

Research 54 (2), 176e190.

Roy, B., Vincke, P., 1981. Multicriteria analysis: survey and new directions.

European Journal of Operational Research 8 (3), 207e218.

Setnes, M., Roubos, H., 2000. GA-fuzzy modeling and classification: com-

plexity and performance. IEEE Transactions on Fuzzy Systems 8 (5),

509e522.

Stokes, S., Plummer, J., 2004. Using spreadsheet solvers in sample design.

Computational Statistics & Data Analysis 44, 527e546.

Triantaphyllou, E., Sanchez, A., 1997. A sensitivity analysis approach for

some deterministic multi-criteria decision-making methods. Decision

Sciences 28 (1), 151e194.

Ulvila, J.W., Seaver, D.A., 1982. Decision analysis for water resources plan-

ning. Omega 10, 185e194.

Wolters, W.T.M., Mareschal, B., 1995. Novel types of sensitivity analysis for

additive MCDM methods. European Journal of Operational Research

81 (2), 281e290.

Distance-based and stochastic uncertainty analysis for multi-criteria decision analysis in Excel ...

Documents

Transcript of Distance-based and stochastic uncertainty analysis for multi-criteria decision analysis in Excel ...