Ecological Indicators 26 (2013) 7686

Contents lists available at SciVerse ScienceDirect

Ecological Indicators

jo ur n al homep ag e: www.elsev ier .com

Multi-criteria decision analysis to select metrics sustain

M. Conve le, a Department ob Florida Climac Badger Techn eam, (ERDC), Concord Department of Civil and Environmental Engineering, Environmental and Water Quality Program, Massachusetts Institute of Technology, Cambridge, MA, USAe Risk and Decision Science Team, Environmental Laboratory, Engineer Research and Development Center (ERDC), U.S. Army Corps of Engineers, Vicksburg, MS, USAf Department of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA

a r t i c l

Article history:Received 28 FeReceived in reAccepted 8 Oc

Keywords:Multi-criteria EnvironmentaEcosystem resMonitoringStakeholder p

1. Introdu

In the cmeasurablequantify thent life stagof Engineer

Corresponand Florida Cl32611-0570, U

E-mail add

1470-160X/$ http://dx.doi.o e i n f o

bruary 2012vised form 2 October 2012tober 2012

decision analysisl metricstoration

references, utility

a b s t r a c t

The selection of metrics for ecosystem restoration programs is critical for improving the quality andutility of design and monitoring programs, informing adaptive management actions, and characterizingproject success. The metrics selection process, that in practice is left to the subjective judgment of stake-holders, is often complex and should simultaneously take into account monitoring data, environmentalmodels, socio-economic considerations, and stakeholder interests. With limited funding, it is often verydifcult to balance the importance of multiple metrics, often competing, intended to measure differentenvironmental, social, and economic aspects of the system. To help restoration planners and practition-ers develop the most useful and informative design and monitoring programs, we propose the use ofmulti-criteria decision analysis (MCDA) methods, broadly dened, to select optimal ecosystem restora-tion metric sets. In this paper, we apply and compare two MCDA methods, multi-attribute utility theory(MAUT), and probabilistic multi-criteria acceptability analysis (ProMAA), for a hypothetical river restora-tion case study involving multiple stakeholders with competing interests. Overall, the MCDA results in asystematic, quantitative, and transparent evaluation and comparison of potential metrics that providesplanners and practitioners with a clear basis for selecting the optimal set of metrics to evaluate restorationalternatives and to inform restoration design and monitoring. In our case study, the two MCDA meth-ods provide comparable results in terms of selected metrics. However, because ProMAA can considerprobability distributions for weights and utility values of metrics for each criterion, it is most likely thebest option for projects with highly uncertain data and signicant stakeholder involvement. Despite theincrease in complexity in the metrics selection process, MCDA improves upon the current, commonly-used ad-hoc decision practice based on consultations with stakeholders by applying and presentingquantitative aggregation of data and judgment, thereby increasing the effectiveness of environmentaldesign and monitoring and the transparency of decision making in restoration projects.

2012 Elsevier Ltd. All rights reserved.

ction

ontext of ecosystem restoration projects, metrics are system properties that characterize the system ande impact of restoration activities, possibly at differ-es of restorations (Allen et al., 1997; U.S. Army Corpss, 1999; Nienhuis et al., 2002; Reichert et al., 2007;

ding author at: Department of Agricultural and Biological Engineeringimate Institute, Frazier Rogers Hall, Museum Road, PO box 110570,SA. Tel.: +1 781 645 6070; fax: +1 352 392 4092.ress: mconvertino@u.edu (M. Convertino).

Seager et al., 2007; Martine and Cockeld, 2008; McKay et al., 2011).Thoughtful, appropriate metrics selection is key to effectively char-acterizing the system, selecting a restoration strategy or a singlerestoration among a set of restoration alternatives, and under-standing the effects of project actions on the system (Ehrenfeld,2000). Appropriate, clearly dened metrics should reduce uncer-tainty, increase knowledge of the system and assess the usefulnessof applied restoration alternatives by creating a targeted, effectivemeans of evaluation. The evaluation of a restoration alternative canoccur both pre- and post-execution (Holmes, 1991), and it is cer-tainly important considering the variability of climate and otheranthropic factors (Palmer et al., 2008). For example, a monitor-ing plan based on sound metrics can demonstrate progress and

see front matter 2012 Elsevier Ltd. All rights reserved.rg/10.1016/j.ecolind.2012.10.005able ecosystem restorations

rtinoa,b,, K.M. Bakerc, J.T. Vogelc, C. Lud, B. Suedef Agricultural and Biological Engineering, University of Florida, Gainesville, FL, USAte Institute, c/o University of Florida, Gainesville, FL, USAologies contractor with the U.S. Army Corps of Engineers, Risk and Decision Science Td, MA, USA/ locate /eco l ind

for design and monitoring of

I. Linkove,f

Environmental Laboratory, Engineer Research and Development Center

M. Convertino et al. / Ecological Indicators 26 (2013) 7686 77

the degree to which objectives of a restoration are being met toleadership, stakeholders, and future project sponsors, increase thedepth and breadth of understanding about the effects of ecosystemrestoration practices, contribute to expanding knowledge aboutecosystemstive, efcienThom and Grootjans eations are trof environmecosystem r

The comgives rise toExtensive lioptions, ofttem charac2006). For eprovides fhabitat suplogical, geogothers (Thaonly be posa few metrclearly indiproject goa

Metrics choice of ming multiplcommunicauating thestask that rmethod. Thselection, indence, concsets, and AnDale and BeMoberg, 20we briey refer the remethodolog

The use sive and timmethod forselection vistakeholdercomplexityjudgment amakes the d(Dale and B

Historicabeen previo(e.g. those wical charactinvolve simmetrics oftcross-compinvestmenttion via thiwell-suitedmore famili

As a mofessional jumanagers magainst a sof metrics fand time-ef

a more structured metrics selection method than best professionaljudgment and historical precedence, but is generally not adequateas a standalone method. Screening does not facilitate formal consid-eration of a metrics utility within the total collection of its metrics

mosijer

interehenlyticsele

edgeecis

nmen et alrisond wevers007)imprringtion

thete alte fo

e impbergst uswouformrove

e band uatintion

MCDer. Mles ice aria a

2004ple

tandarent2009erentsion006;

and techor eed tot actihis ptem

a h Riveosyst in als avelopectioic anA). R

anaparattudy., and guide management decisions on the most effec-t, and cost-effective courses of action (Kondolf, 1995;

Wellman, 1996; U.S. Army Corps of Engineers, 1999;t al., 2002; Rohde et al., 2004). The same consider-ue for design plans that aim to change the congurationental systems at the initial or intermediate steps ofestorations.plexity of ecological systems and restoration objectives

a multitude of potential ecosystem monitoring metrics.sts of monitoring metrics provide hundreds of potentialen with numerous choices for just one specic ecosys-teristic (Thayer et al., 2005; Faber-Langendoen et al.,xample, NOAAs Tools for Monitoring Coastal Habitatsteen different metrics to monitor whether a mangroveports a complex trophic structure alone, including bio-raphical, hydrological, and chemical metrics as well as

yer et al., 2005). However, with limited funding, it maysible to effectively measure, estimate, or otherwise useics, so it is critical to select the metrics that can mostcate the state of the system and changes in relation tols.selection is thus a challenging process. The optimumetrics will depend on a number of factors includ-

e project objectives, technical feasibility, effectiveness,bility, and stakeholder preferences. Balancing and eval-e factors with respect to each metric choice is a difcultequires a comprehensive, practical metrics selectionere are a number of commonly used methods for metricscluding best professional judgment, historical prece-eptual modeling, screening using established criteriaalytic Hierarchy Process models (AHP) (Saathy, 1980;yeler, 2001; Niemeijer and de Groot, 2008; Linkov and11; Convertino et al., 2012; Mexas et al., 2012). Heredescribe only the most commonly used methods andader to more extended review papers for additionaly (see for example Linkov and Moberg, 2011).

of best professional judgment (BPJ) is generally inexpen-e-efcient and may be an appropriate metrics selection

small, well-understood projects. However, metricsa this method may exclude or place bias on specic

values, and becomes exceedingly difcult as project increases. Another weakness of both best professionalnd historical precedence is lack of transparency, whichecision-making more difcult to document and justifyeyeler, 2001; Niemeijer and de Groot, 2008).l precedence constitutes selection of metrics that haveusly utilized in similar ecosystem restoration programsith similar objectives, with similar regional or ecolog-eristics, that respond to similar disturbances, and/orilar stakeholders). Maintaining the use of historicalen allows for easy comparison to baseline data andarison among projects, and may involve lower initial

than developing new metrics. However, metrics selec-s method may encourage project planners to overlook

and site-specic metrics in favor of less appropriate butar metrics.re transparent alternative or supplement to best pro-dgment and historical precedence, restoration projectay sometimes evaluate or screen potential metrics

et of criteria to identify the most appropriate subsetor a given project. Screening is relatively inexpensivecient, and criteria are well-documented. Screening is

set, as(Niemetative compr

Ananative knowlof the denviroHuangcompamalizerank reet al., 2

To monitoapplicascienceevaluaity valurelativand Mothe mowhich well-into imptices. Wuseful in evalrestorajustifyconsidit enabers, sinof criteSigrid,the comunderstranspet al., of diffof deciet al., 2LinkovMCDAwork fdesignprojec

In tecosystion toBlackistic ecpresenMaterithe desame sministProMAinationa comcase st criteria are meant to apply to metrics individuallyand de Groot, 2008). In particular, there is no a quanti-nal structure for determining whether a metrics set issive.al Hierarchy Process is a controversial method for alter-ction developed by Saaty, 1980. To the best of our

it was never used in selection of metrics as alternativesion problem. However, AHP has been used in a variety oftal management problems (Linkov and Moberg, 2011;., 2011). Because AHP is based on a subjective pairwise

of criteria, rather than using value functions and nor-ights, it has been criticized for its measurement scale,al, and transitivity of preferences (Gass, 2005; Yatsalo.ove the efcacy of ecosystem restoration design and

programs (Linkov and Moberg, 2011), we suggest the of MCDA, a decision-making analysis based on decisionory (Keeney and Raiffa, 1976) that can quantitativelyernatives (i.e. metrics in our case) based on their util-r stakeholders with respect to dened criteria, and theortance of those criteria (Drechsler et al., 2003; Linkov, 2011). Applied correctly, MCDA methods will result ineful metric set for evaluating stated project priorities,ld enable project managers to make comprehensive,ed decisions, and allow researchers and practitioners

and update the principles that guide restoration prac-elieve that a formal MCDA-based method is largelyneeded for the selection of metrics that can be usedg restoration alternatives or monitoring alternatives ofs. Tsoutsos et al. (2009) provides several reasons thatA for use in complex decisions with similar factors toCDA is appropriate for complex decisions because: (a)ntegration of interests and objectives of multiple play-ll of this information can be accounted for in the formnd weight factors (Pohekar and Ramachandran, 2004;; Loken, 2007; Tsoutsos et al., 2009); (b) it deals with

xity of having multiple stakeholders by providing easilyable outputs, and, by virtue of working systematically, is

and user-friendly (Georgopoulou et al., 1997; Tsoutsos); and (c) it is well-documented and a large number

MCDA methods have been applied in a wide ranges (Phillis and Andriantiatsaholiniana, 2001; Kaminaris

Diakoulaki and Karangelis, 2007; Tsoutsos et al., 2009; Moberg, 2011). Here, we expand the application ofniques by developing and applying a MCDA frame-valuating and ranking ecosystem restoration metrics

characterize the system and quantify the effects ofons.aper, we introduce the framework for using MCDA forrestoration metrics selection and illustrate its applica-ypothetical restoration case study which we call ther Restoration Project. Our case study resembles a real-tem because we consider all the components typicallya river ecosystem. The paper is structured as follows.nd Methods describe the hypothetical case study andment of the components of the MCDA models. In then we introduce the theoretical background of the deter-d stochastic multi criteria decision models (MAUT andesults and Discussion present the results of the dom-lysis and metric alternative rankings. We also provideive assessment of both MCDA models applied to the

The Conclusions section discusses the benets and

78 M. Convertino et al. / Ecological Indicators 26 (2013) 7686

limitations of utilizing MCDA for metrics selection and the mostappropriate circumstances in which to apply this new methodol-ogy.

2. Materials and methods

This paper builds on metrics selection methodology proposed byLinkov and Moberg (2011) and tailors it specically for evaluationof ecosystem restoration monitoring metrics. Two MCDA methods,MAUT and ProMAA, were utilized to rank metrics for monitoringand evaluating a hypothetical river restoration project in which thecoupling between human and natural systems is very high.

2.1. Hypothetical aquatic ecological restoration

The Black River is a hypothetical perennial river with a broadoodplain consisting of impermeable surfaces, cottonwood forests,aquatic wetlands, bare soils, and the river network. Fig. 1 representsa schematic view of the hypothetical Black River and its land covercategories. Over the past several decades, the river and oodplainhave undergone signicant changes due to urbanization and damconstruction. The cumulative effect of these stressors is the disrup-tion of the original hydrologic regime, main stem channelization,and reduced river-oodplain interaction, which has increased reand ood hazards, reduced wildlife habitat quality and quantity,decreased biodiversity, and facilitated encroachment of harmfulexotic plants. In partnership with state authorities, federal gov-ernment institutions are planning an ecosystem restoration projectwith the goal of increasing ecosystem quality by restoring the struc-ture and fun

A multi-NGOs, andhydrologistbled to setapproach fouate restorRiver oodptrack systemdepended owell projec

a complex system with multiple objectives and stakeholders, theteam chose to use MCDA methodology to guide their selection ofthe optimal metric set.

2.2. Multi-criteria decision analysis (MCDA)

MCDA is a structured approach to decision-making that quan-titatively evaluates alternatives, in this case, metrics, based ondened project criteria, expert opinions, and stakeholder prefer-ences (Linkov and Moberg, 2011; Wood et al., 2012). It integratesa wide variety of information to evaluate project alternatives andrank them based on their aggregated value with respect to a setof criteria (Linkov and Moberg, 2011). It usually consists of fourstages. The project team, incorporating expert and stakeholderopinions, must dene: (1) the set of possible decision alternatives(in this case, metric alternatives) to be evaluated and ranked; (2)the criteria of the value tree that will inuence the decision thatthese alternatives will be evaluated against; (3) the importance ofeach criterion relative to the others or their weight followed bya normalization of weights performed separately for each order ofcriteria (criteria of order one, criteria of order two (or sub-criteria),etc.); and (4) the value of each alternative with respect to each cri-terion. Depending on the specic MCDA method, (3) and (4) mayalso include uncertainty estimates.

In MCDA methods that incorporate utility theory (Keeney andRaiffa, 1976) the values in (4) for each criterion are transformed intoutility values according to utility functions for each criterion. Utilityfunctions are expressions of stakeholder preferences of alternativesas a function of each criterion (Keeney and Raiffa, 1976) that are

asserect torat

andpproial al

withre caltern

withema

Fig. 1. Schem iver bidentied by t et al. A is the basiction of the Black River oodplain ecosystem.agency (federal, state, and local government, academia,

private consultants), multi-disciplinary (ecologists,s, geologists, engineers, economists) team was assem-

objectives, develop a conceptual model, identify anr assessing environmental benets, formulate and eval-ation alternatives to address degradation of the Blacklain, and develop an effective monitoring program to

changes and evaluate project success. The latter taskn selecting the most appropriate metrics to assess howt objectives were being met. As the project involves

usuallyor indi

Rescriteriawhile apotentnativesoftwathose anativethose r

atization of a hypothetical Black river. The main land cover categories within the rhe river network and river basin boundaries (dashed lines). The river basin in Settinn outlet.ssed by direct methods (e.g. interviews (Keeney, 1977)),methods (e.g. serious games (Braziunas, 2012)).ion planners and stakeholders should determine the

the relative importance (weighting) of each criterion,priate professionals and eld experts should create theternatives pool, and determine the value of each alter-

respect to each criterion. Using this information, MCDAn be used to rst eliminate dominated alternatives, oratives that are less valuable than at least one other alter-

respect to every decision criterion, and then to rankining. The rank is an ordinal number in the range [1,n],

asin (as described in Section 2.1). are represented. The river basin is(2007) is considered as the hypothetical Black river in this case study.

M. Convertino et al. / Ecological Indicators 26 (2013) 7686 79

where n is the number of metrics, assigned as a function of thedecreasing utility. The higher the utility, the lower the rank.

In the case of the Black River restoration, the MCDA analy-sis was used to narrow down and rank an initially large set ofaquatic ecoability to prto project othose objecthe people tered as sug

We utiliuation in CoSystem) (Yathe input dacomplex dutem compothese systemware was dDecision Scand Mobergcase study iFor further (2011).

The hypis twofold: to evaluateto which thproject impric alternatintended beent orders ocriteria untcriteria andbut generalpublic healtFrom the obcriteria andity of each pbest based be the mostmeasuring

The resurepresentedshowing thdominated methods, thcriteria of onormalizatisub-criteriacriteria but are normali

Chosen tives and shproject succinterest. Yetions shouland risks o2002; ClarkForum, 201case study wlogical, geocriteria (Tabbecause theria case by ecosystem 2012).

The general criteria selected give a good indication of the stateof the environment and the environmental effects of restorationmeasures. Some of these criteria included more detailed sub-criteria such as recreation and maintenance under economy. The

sub-criteria allows stakeholders to weight both the gen-tegory (e.g., economy) and more detailed aspects that relateic stakeholder concerns. Criteria formulation is an impor-ep in dening what is important to project success and what

be considered in the decision making process. More criteria,iteria, and criteria of higher order could have been considered

project, but the current set gives an adequate indication ofte of the system and progress toward objectives.e the taxonomy of criteria and sub-criteria was established,ject team then developed an initial, comprehensive set ofl aquatic ecosystem monitoring metrics related to each crite-able 1). The set of metrics to be selected should be as holistic

t of potential metrics for aquatic ecosystems, organized into functional cat-Each category of metrics can be thought of as an ecosystem service class.m services can be grouped in sustainability classes (Environmental, Social,nomical). Each initial potential metric is then considered dominated, non-ed, or equal to another metric according to a Pareto domination analysis

2.2). The 43 metrics constitute an exhaustive list according to the stake-(i.e. the authors in this case study) involved in the hypothetical Black riverion.

Metrics

logical Water Table LevelSoil MoistureBankfull DischargeHydroperiodFlooding Return Period 100-yearsFlooding Frequency 1-year RunoffFlooding Frequency 2-year RunoffMinimum Water FlowRiver Salinity

rphological Maximum Elevation GradientRiver Basin ExtensionFloodplain ExtensionAverage Riparian WidthHacks ExponentHillslope Stability FactorSediment Discharge

ical Species Area ExponentLocal Species RichnessRegional Species RichnessPairwise Species Richness SimilaritySpecies AbundanceAverage Landscape Connectivityp/A Patch RatioCanopy EvapotranspirationHabitat Area SongbirdGeographic Range SongbirdHabitat Area FishesNumber Invasive SpeciesMetapopulation Risk

ical

mical

econom

system monitoring metric alternatives based on theirovide information about system characteristics relatedbjectives (the criteria) and stakeholder preferences fortives (the weights). We consider stakeholders to be allhat have a stake in the environmental problem consid-gested in Wood et al. (2012).zed the MCDA software DECERNS-SDSS (Decision Eval-mplEx Risk Network SystemsSpatial Decision Supporttsalo, 2011) to model the problem space and analyzeta. The idea behind DECERNS-SDSS is that systems aree to the high degree of interconnections among sys-nents and because of the multiplicity of risks affectings (Linkov and Moberg, 2011; Yatsalo, 2011). The soft-

eveloped by Yatsalo (2011) supported by the Risk andience Team of the U.S. Army Corps of Engineers (Linkov, 2011). A demonstration version of the model and thes included in this paper in the Supplementary Material.information we refer the reader to Linkov and Moberg

othetical monitoring goal of the optimal set of metrics(i) to select the best restoration alternative; and, (ii)

restoration project success by measuring the degreee intended objectives have been achieved following thelementation period. To begin the MCDA analysis of met-ives, we rst dened the set of project objectives, ornets, in order to develop the model criteria at differ-f the value tree (Table 1). In this case study we developil the second order in the value tree; thus, we refer to

sub-criteria. Objectives vary from project to project,ly include environmental, economic, socio-political, andh and safety considerations (Linkov and Moberg, 2011).jectives of the hypothetical project, we derived a set of

sub-criteria against which we later evaluated the util-otential metric. The metric alternatives that performedon these criteria and sub-criteria were considered to

useful in characterizing important project aspects andfulllment of specic ecosystem restoration objectives.lts of the criteria and alternative development are

in DECERNS-SDSS by a structured value tree (Fig. 2)e overall objective, criteria, sub-criteria, and non-metric alternatives. Note that, on the contrary of AHPe tree can be asymmetric (e.g. a different number ofrder two for each criterion) and the assignment andon of weights is performed separately for criteria and. In this case, it is possible to compare the weights ofnot the weights of criteria and sub-criteria because theyzed at their respective level in tree.criteria depend on clear, well-dened project objec-ould be comprehensive, including all aspects relating toess as well as any additional system characteristics oft, we believe that a sustainable paradigm for restora-d be adopted considering that components, services,f ecosystems are highly interconnected (Pauly et al.,, 2007; Bettencourt and Kaur, 2011; World Economic1; National Academy of Sciences, 2012). Thus, in oure included environmental (hydrological, geomorpho-

logical, ecological, biochemical), social, and economicle 1). Health criteria have been considered separatelyy can be part of both social and environmental crite-case depending on the drivers of health issues in theanalyzed (Clark, 2007; National Academy of Sciences,

use oferal cato spectant stshouldsub-crfor thisthe sta

Oncthe progenerarion (T

Table 1Initial lisegories. Ecosysteand Ecodominat(Sectionholders restorat

Hydro

Geomo

Ecolog

Geolog

Bioche

Socio-

HealthGranulometric CurveConductivity

Total Maximum Daily LoadN %P %C %O %PHMicrobial BiomassBioaccumulation Potential

ical Number Visits/YearNumber of Trails

Number of Epidemics last 10 years

80 M. Convertino et al. / Ecological Indicators 26 (2013) 7686

Fig. 2. Multi-Cdeletion of thegure is a screare in Figure S

as possible are conside

The potein the eldbecomes avnd it beneproject-speconsideringtion of a spcan be also time.riteria Decision Analysis (MAUT and ProMAA) model architecture for the hypothetical Bl dominated metrics (see Table 1 for initial metrics and Table 2 for the analysis of non-denshot of the model in DECERNSSDSS (Supplementary Material). Dw and Uf are the crite1, S2 and S3 respectively). In the theoretical formulation of the model (Sections 2.3 and 2

in order to guarantee that the most important metricsred.ntial metrics pool can be adjusted as new information

of environmental and socio-economical managementailable or, in some projects, restoration managers maycial to adjust the potential metrics pool to account forcic objectives. This would certainly be necessary when

very specic objectives such as increasing the popula-ecic endangered species. Moreover the metrics pooladjusted when project-specic objectives are shifted in

In practexperts aretial metric terminologscore, or pasented in thknowledgementary Mhydroperio

Next, thand removack river case study. The value tree of the MCDA model is shown afterominated metrics using the Pareto-based domination analysis). Theria weights and partial utility score of criteria and metrics (examples.4) Dw and Uf are indicated as wj and Uj(ai) respectively.

ice, environmental practitioners, scientists and other then consulted to evaluate the utility of each poten-in measuring fulllment of each criterion. In MCDAy, this is to say that the analysts assign each metric artial utility, for each criterion. In the case study pre-is paper, the authors provided the scores using their best

of aquatic ecosystem function and structure. In Supple-aterial (Fig. S1) we report an example of scores for thed as a function of each criterion.e team ran an MCDA domination analysis to identifye metrics that were dominated by one or more others

M. Convertino et al. / Ecological Indicators 26 (2013) 7686 81

Table 2Pareto-based domination analysis. The table reports the non-dominated (whitelines), dominated (dark gray lines), and equivalent metrics (light gray lines). Equiv-alent metrics are characterized by the same utility for the objective of the decisionproblem that idominated alt(Sections 2.3 atable. The initi

Metric Index

1 234

5 6

7 8

9

10 11 12

13 1415 16 1718 19

20

2122 23 24 25

26

27

28 29 30

31

32

33 34 35 36 37 38 39 40 41 42 43

(Table 2). D(had lower ria. These munder any wset of nonranked.

Specically DECERNSDSS implements a Pareto dominancemethod (Yatsalo, 2011). A feasible combination of metrics for a col-lection of objectives is said to be Pareto dominated if there does

nother feasible combination of metrics under which eachve is

(Em a muof wr the

prefs the Black river restoration. The overall utility, U(ai), of the 25 non-ernatives is calculated by the MCDA methods (MAUT and ProMAA)nd 2.4 respectively). The metric index is for a better readability of theal list of metrics is in Table 1.

Metric name Domination

River Basin Extension Non-dominatedHacks Exponent Non-dominated

exist aobjectiter offpurelycation

AfteholderAverage Riparian Width Non-dominatedFloodplain Extension Dominated by River

Basin ExtensionLocal Species Richness Non-dominatedMetapopulation Risk Dominated by

Number of InvasiveSpecies

Number of Invasive Species Non-dominatedHabitat-area Fishes Dominated by Local

Species RichnessHabitat-area Songbird Dominated by Local

Species RichnessNumber of Trails Non-dominatedNumber of Visits/Year Non-dominatedFlooding Frequency 1-year Runoff Equals with

Flooding Frequency2-year Runoff

Flooding Frequency 2-year Runoff Non-dominatedWater Table Level Non-dominatedSoil-moisture Non-dominatedBankfull Discharge Non-dominatedHydroperiod Non-dominatedMinimum Water Flow Non-dominatedFloodplain Return Period 100-years Dominated by

Flooding Frequency1-year Runoff

River Salinity Dominated byWater Table Level

Maximum Elevation Gradient Non-dominatedHillslope Stability Factor Non-dominatedSediment Discharge Non-dominatedSpecies-area Exponent Non-dominatedRegional Species Richness Dominated by Local

Species RichnessPairwise Species Richness Similarity Dominated by Local

Species RichnessSpecies Abundance Dominated by Local

Species RichnessGeographic Range Songbird Non-dominatedAverage Landscape Connectivity Non-dominatedP/A Patch Ratio Dominated by Local

Species RichnessCanopy Evapotranspiration Dominated by

Flooding Frequency1-year Runoff

Total Maximum Daily Load Dominated byMicrobial Biomass

N % Equals with P %P % Dominated by O %C % Dominated by N %O % Non-dominatedPH Dominated by O %Microbial Biomass Non-dominatedD50 Non-dominatedGranulometric Curve Dominated by D50Conductivity Non-dominatedBioaccumulation Potential Non-dominatedNumber of Epidemics Last 10 Years Non-dominated

ominated metrics were those that were outperformedvalue scores) by at least one other metric in all crite-etrics were eliminated as they would not be selectedeighting scenario. The result of this step was a smaller

-dominated metrics which were then analyzed and

importanceto which mtives transland sub-cricase study,titioners anelicitation, ria and subaquatic ecowould be inences for eacontaining tion of the Bhydrologicato observe t

For prefmore sensihighest wehabitat impogy, 0.17 fohealth, 0.11malized wedependent is critical tosuite of prefweights repvalues of al(Keeney andby uncertaiis capable ois only capaassigned totainty relatdo not consgroup of sta

The inpuand sub-crvalues; anddecision su2007) whicutility to thranking wilbelow wheand ProMA

2.3. Multi-a

The Mupreferencesmetric altera given metproject objoverall utilwere combaspects of thria of differe at least as well off and some objective is strictly bet-merich and Deutz, 2006). Domination comparison is alti-objective metric comparison that gives some indi-

hich of two metric sets is closer to the Pareto front. domination analysis, restoration manager and stake-erences need to be elicited to establish the relative

of each criterion and sub-criterion. The relative extentsanagers and stakeholders value various project objec-ate to the relative weights of the corresponding criteriateria. Because this investigation utilized a hypothetical

we did not have access to a group of restoration prac-d stakeholders. Instead of practitioner and stakeholderwe assigned hypothetical weights to each of the crite--criteria, based on what we believed were reasonablesystem restoration priorities. In practice, stakeholdersterviewed or tested to determine their relative prefer-ch criterion. Supplementary Material reports the tablesthe weights of criteria and sub-criteria for the restora-lack River (Figs. S2 and S3 respectively; the example ofl sub-criteria is reported). In Figs. S2 and S3 it is possiblehe normalization of the weights in a [0,1] range.erence weighting, we assumed the stakeholders weretive to ecological problems and therefore assigned theights to ecology, and to those criteria with the largestact. The normalized weight values are: 0.22 for ecol-r geomorphology and hydrology, 0.14 for economy and

for geology, and 0.05 for biochemistry. These nor-ights always sum to one. These weights are highlyon which stakeholders views are incorporated, so it

involve a variety of stakeholders to capture the fullerences for project outcomes. In general the aggregatedresenting all the stakeholders involved are the averagel the individual stakeholder weights for each criterion

Raiffa, 1976). In reality these weights are characterizednty and may vary in time. The MCDA method, ProMAA,f handling weight uncertainty. On the contrary, MAUTble of evaluating the uncertainty related to the utility

each alternative for each criterion and not the uncer-ed to the stakeholders preferences (weights). Here weider the variability of stakeholder preferences amongkeholders and in time.t data are complete once the (1) taxonomy of criteria

iteria; (2) pool of potential metrics; (3) partial utility (4) weights, are formulated. An MCDA is then run usingpport software such as DECERNS-SDSS (Yatsalo et al.,h will rank the potential metrics in terms of their overalle set of weighted project objectives. Details of metricl be explained further in the specic method sectionsre we report the analytical characterization of MAUT,A methods used to rank metric alternatives.

ttribute utility theory (MAUT)

lti-Attribute Utility Theory (MAUT) resolves multiple and value scores into an overall utility value for eachnative, enabling comparison. In this case, the utility ofric for measuring fulllment of a specic aspect of theective was treated as a partial utility. To calculate anity for each potential metric, the partial utility valuesined based on the relative importance of the componente objectives to the stakeholders (the weighting of crite-nt order in the value tree). MAUT considers uncertainty

82 M. Convertino et al. / Ecological Indicators 26 (2013) 7686

related to the utility assigned to each metric for each criterion, butdoes not consider the uncertainty related to the stakeholders pre-ferences (weights) (von Winterfeldt and Edwards, 1986; Belton andStewart, 2002a,b).

To calculet the set oand the setmethod (Keranked base

U(ai) = f (Uwhere Uj(aicj of any ordmay take oKeeney andtial utilitiesis widely usof the overa

U(ai) = w1Uwith the no

m

j=1wj = 1,

where wj isterion cj inpartial utili

MAUT capartial utiland thus thbe consider

Despite is not univuse expecteVincke, 198Stewart, 202008). Howcharacteristutility valueple metricsto describe terion. Thisutilities forfunctions aimportant sRaiffa, 1976

For uncesidered for to 10 propoto each critsidered fromutility of eaand the stathis paper (gist (B.S.), ato reduce thmay bring i

2.4. Probab

Probabilan MCDA mvalues, for (Yatsalo, 20technical m

weighting uncertainties. The ProMAA algorithm utilizes the prob-ability distributions of alternative scores and of weight coefcientsfor assessing rank acceptability indices based on pair-wise compar-ison of alternatives (Linkov and Moberg, 2011; Yatsalo et al., 2007;

, 201l utili

p(w

p(wjolderrma

MAU weis andhe no

j)wj

ProMeria

of s in otivesre bales aaccep

indicnces004)

(Sik ),

Sik ink krms ). ThA is gs ar

set the

typi

1

wak

wkac

998)s, raA is

on the ailar thod), Preightlma f SMueirroxi

et al). Inof untivestionthetiilitylate the overall utility for each metric alternative, rstf potential metrics alternatives be A = {ai, i = 1, . . . , n}

of criteria be C = {cj, j = 1, . . . , m}. Using the MAUTeney and Raiffa, 1976), each metric alternative, ai, isd its overall utility, U(ai):

1(ai), . . . , Uj(ai)) (1)

) is the utility of alternative ai with respect to criterioner in the value tree. The generic MAUT function (Eq. (1))n multiple functional forms (Keeney and Raiffa, 1976;

Gregory, 2005). For this paper, we assume that the par- are independent, and utilize the additive form, whiched for practical MAUT applications. The functional formll utility of metric alternative ai, U(ai), is:

1(ai) + + wjUj(ai), (2)rmalization condition,

wj > 0, (3)

a weighting factor representing the importance of cri- the project. In DECERNSSDSS the weight wj and thety Uj(ai) are indicated as Dw and Uf respectively (Fig. 2).n use distributions for alternative utility scoring (i.e. theity, Uj(ai)), but can only use point values for weightse uncertainty related to stakeholder preferences cannoted.extensive use of the expected utility concept, its useersally accepted, and other approaches that do notd utility methods are often implemented (Brans and5; von Winterfeldt and Edwards, 1986; Belton and02a,b; Figueira et al., 2005; Tervonen and Figueira,ever, in metrics selection problems, like evaluating siteics and restoration project success, we believe that the

of a metric, rather than the expected value of multi- for a variety of scenarios and criteria, is the best waythe aspect of the ecosystem corresponding to each cri-

is because same values of criteria can have different different stakeholders. Thus, the elicitation of utilitynd the translation of values to utilities is an extremelytep for selecting and evaluating restoration (Keeney and).rtainty considerations, a normal distribution was con-the utility weight value with average ranging from 0rtionally to the importance of each metric with respecterion. A standard deviation from 0.1 to 0.01 was con-

the average value according to the uncertainty in thech metric in describing each criterion. The utility valuesndard deviations have been assessed by the authors ofenvironmental engineers (M.C., K.B., and C.L.), a biolo-n ecotoxicologist (I.L.), and an ecologist (J.V.)) in ordere subjective uncertainty and the bias that one expert

nto the decision problem.

ilistic multi-criteria acceptability analysis (ProMAA)

istic Multi-criteria Acceptability Analysis (ProMAA) isethod that can use distributions, rather than point

both weights and alternative utility scoring of criteria11; Linkov and Moberg, 2011; and see DECERNS-SDSSanual), allowing the user to account for both scoring and

Yatsalooveral

U(ai) =wherestakehThe nounlikefor theauthor0.05. T

m

j=1p(w

Theof critbilitiesmetricalternaMAA avariabRank abilityprefereet al., 2

Pik = Pwherewith raai in teeventsProMARankincriteria

Forsum is

Pi =n

k=

whereet al., 1

ThuProMAand/orwhich

Siming me(SMAAand w(Lahdetions oand Figcal appYatsalomanuatance alternarestora

Synprobab1; and see DECERNS-SDSS technical manual). Thus, thety of metric alternative ai, U(ai), is:

1)w1U1(ai) + + p(wj)wjUj(ai), (4)) is the probability to observe the weight wj expressings preference for criterion j of any order in the value tree.

lization condition for the weights still holds; however,T, ProMAA uses distributions instead of point values

ghts. In this case, the weights were determined by the to each weight was assigned a standard deviation ofrmalization condition is expressed analytically as:

= 1, wj > 0, 0 p(wj) 1, (5)

AA algorithm can also utilize probability distributionsutilities and weight coefcients for assessing proba-likely rank events (where events are associated withur decision problem) based on pairwise comparison of

in an integrated scale. In this case realizations of Pro-sed on numerical approximation of functions of randomnd numerical assessment of integrals (Yatsalo, 2011).tability indices are the output of ProMAA. Rank accept-es are probabilities that describe the variety of different

resulting in a certain rank for an alternative (Lahdelma, and can be expressed as Pik :

(6)

s the event characterized by the metric alternative ai and i, k = 1 . . . n (i.e., k 1 alternatives are better thanof a given criteria for a subset of a space of elementaryus, ranking or screening metrics {ai, i = 1,. . .,n} withinbased on the analysis of the matrix {Pik}, i,k = 1,. . .,n.e based on the weighted overall score of ai against theC.aggregation of the indicated probabilities, a weightedcally used:

cPik , (7)

are weights of relative importance of ranks (Lahdelma.nking or screening alternatives {ai, i = 1,. . .,n} withinbased on the analysis of the matrix {Pik}, i,k = 1,. . .,n,the holistic acceptability indices Pi, i = 1,. . .,n, fromverage rank of alternatives k can be assessed.to the more commonly used probabilistic outrank-, and stochastic multi-objective acceptability analysisoMAA accounts for uncertainty ranges in both criteria

values in its calculation of rank acceptability indiceset al., 2004). However, while software implementa-AA are based on Monte Carlo simulations (Tervonena, 2008), ProMAA implementation is based on numeri-mation of random variables (Linkov and Moberg, 2011;l., 2007; Yatsalo, 2011; and see DECERNS-SDSS technical

this paper, we apply ProMAA to assess the impor-certainty in stakeholder preferences in determining

ranking. In this case the alternatives are ecosystem metrics.cally generated values can be assigned as standard

distributions to criteria and preferences; thus, Monte

M. Convertino et al. / Ecological Indicators 26 (2013) 7686 83

Carlo simulations sample these distributions. In the majority ofcases stakeholder preferences are elicited from workshops or usingother methodologies (for example serious games (Nesloa andCooke, 2011)) and their value and distribution can vary con-siderably. Criteria distributions are inferred from data or fromstakeholder judgment. Thus, both preferences and criteria utilitiesare strongly case-specic and there is no standardized method-ology to gauge and assign their distributions. For further technicaldetails of ProMAA we direct readers to Yatsalo (2011) and to Linkovand Moberg (2011).

3. Results and discussion

The main objective of the paper was to provide methodologicalframework for a quantitative, structured, scalable, and transparentselection of metric alternatives. Metric selection is an extremelyimportant aspect of environmental management; however it isoften prone to high subjectivity that strongly affects both the selec-tion and the evaluation of restoration projects. In this paper, weaimed to illustrate the use of a quantitative, structured and trans-parent metrics selection methodology, MCDA, to rank potentialecosystem restoration metrics. The results of this case study arediscussed below.

The initial analysis, the Pareto-based domination analysis, elim-inated nearly half of the potential metrics based solely on theirpartial utility with respect to each criterion. Specically, thisresulted in the identication and elimination of 18 dominatedmetrics, narrowing the metric pool from the initial comprehensiveset of 43 aquatic ecosystem restoration metrics shown in Table 1to the 25 non-dominated metrics shown in Fig. 2. This greatly sim-plies the decision and provides a clear and logical justication forremoving dominated metrics independently of stakeholder prefer-ences as they are sub-optimal under any set of weights.

The results of the MAUT and ProMAA models, that is, the aver-age utility sFig. 3. The

[0,1] for both methods. This type of visualization allows analysts toeasily compare the utility of each metric as calculated by MAUTand ProMAA. For MAUT, the utility is calculated using Eq. (2),and for ProMAA using Equation 4. As many of the utility scoresare similar to each other, this ranking is not intended to explic-itly determine which metrics to use but it is an excellent guidefor decision makers and clearly indicates that some metrics aremore suitable than others (e.g. the metric local species richnessis clearly more useful than geographic range of songbird). ProMAAenhances the ecological metrics according to the stakeholders pref-erence for ecological criteria, but overall the differences betweenthe results of MAUT and ProMAA in terms of utilities of each ofthe metrics (Eqs. (2) and (4), respectively) are negligible. Despitethat the average utility is higher for ProMAA, the rank of themetrics as a function of the utility score for ProMAA and MAUT isvery similar.

The comparison of utilities from MAUT and ProMAA showsthat the sensitivity of the selected metric set to the stakeholderspreferences is low. This may be related to the small uncertaintygiven to the weights for each criterion, or to the existence ofan already well-dened set of metrics that were selected by avariety of experts in different elds. Certainly large variations instakeholder preferences that are much bigger than the uncertaintyassigned to the weights in this case study (Section 2.4) would causebigger variation in the utility of each metrics. Thus, such variationsin weights need to be considered in real practice. However, suchlarge variability of weights is observed when different groups ofstakeholders are considered for the same problem, or the samegroup of stakeholders is observed in time. Here, we consider uncer-tainty that is small and related to the assessment of stakeholderpreferences. The application of global sensitivity and uncertaintyanalysis (Saltelli et al., 2008) in utility-based MCDA methods forevaluating the effect of large uncertainties is an ongoing effort andnot the purpose of this paper.

e thow m

Fig. 3. Overall non-dProMAA respe AUT arank in a rangecore for each of the metric alternatives, are shown inaverage utility score is represented in the same range

Oncabout h

utility value calculated by the MAUT and ProMAA models. The utility U(ai) of the ctively. The utility is ordered from the highest to lowest value of utility according M

[1,25], where 25 is the number of non-dominated metrics.e metrics ranking is formulated (Fig. 3) the decisionany metrics to use should be a function of the available

ominated metrics is calculated using Equation 2 and 4 for MAUT andnd it is used to rank each metric. The higher the utility, the lower the

84 M. Convertino et al. / Ecological Indicators 26 (2013) 7686

Fig. 4. Probab n forcalculated usin the aindex (Eq. (7)) ssigne(Fig. 3). The low obabi

resources foconsider th

The predstakeholdercriterion arMetrics are(Eq. (6)) thaders preferfor ProMAAtainty) and are also calmetrics is etribution ofprobability This is usefuncertain inrank of metutility (Eq. rank (Eq. (6in both modcompromisthat the utiparative ansets of metvalue of infa metric in in this caseset arising prior to the

4. Conclus

Selectinand evaluatmanagemeupdating re

res dif

Groaseden inith mer harentncorreoveility distributions of the rank of metrics calculated by ProMAA. The rank k is showg Eq. (6). The overall rank of each metric alternative can be calculated by considering. The ranks based on the holistic acceptability indices are equivalent to the ranks aer the rank, the higher the overall utility of a metric. In the gure the value of the pr

r monitoring those metrics. In this case study we do notis aspect that is highly specic to each restoration.icted ranks of ProMAA considering the uncertainty in

preferences and in the utility of each metric for eache represented by the probability distributions in Fig. 4.

characterized by a probability distribution of their rankt is the result of the combined uncertainty of stakehol-ences (for MAUT the preferences are constant values,

preferences are a distribution that accounts for uncer-of utility values of metrics for each criterion. These ranks

limitedoften aand dethose bare ofttems wthe othtranspwhile i

Mo

led rank acceptability indices. The ranking order ofstablished considering the average value of the dis-

each rank (Eq. (7)). In Fig. 4 we show an example ofdistributions of ranks for the six most valuable metrics.ul in showing the reliability of rankings in the face ofputs and how uncertainty affects metric ranking. The

rics from ProMAA determined after calculating metrics(4)) or determined directly after calculating metrics)) is equivalent. This suggests that ProMAA can be usedes (probabilistic utility and outranking modes) withouting the metric selection process. However, we believelity is more useful information than the rank for a com-alysis of metrics utility. The difference in utility amongrics or between two metrics can be interpreted as theormation (Keeney and Raiffa, 1976) of a metric set or ofthe metric decision process. The value of information is

dened as the increase in the overall utility of a metricfrom the predicted additional information of a metric

metric selection.

ions and perspectives

g an appropriate and informative metrics set to monitore ecosystem restoration projects is critical for informingnt decisions of ecosystems, furthering the science, andstoration practices. With myriad metric choices and

incorporatitative modeand environport more s

In this sMAA, to deselection foof potentiaproject, descreened thnally appranked accoholder prefto each critter suited tothey translMoreover, Pferences thlacking or intain. Howevdid not shoranking of t

Comparin the introdence, concsets, and An the six highest ranked metrics. The rank acceptability index, Pik , isveraged value of the ranks after calculation of the holistic acceptabilityd to the metrics after the utility calculated using Eq. (4) with ProMAAlity for each rank (x-axis) is reported above each bar of the histograms.

ources for monitoring, selecting the best metric set iscult task (Noss, 1999; Dale and Beyeler, 2001; Niemeijerot, 2008). Current metrics selection methods, such as

on best professional judgment or historical precedenceadequate for decision-making involving complex sys-ultiple alternatives and evaluation criteria. MCDA, on

and, provides decision makers with a tool to clearly andly evaluate metric alternatives over a number of criteriaporating stakeholder values and expert opinions.r, MCDA limits the subjectivity in metric selection by

ng stakeholder preferences into structured and quanti-ls. By simultaneously considering the social, economic,mental components, the selected metrics aim to sup-ustainable restorations.tudy, we applied the MCDA methods, MAUT and Pro-monstrate how MCDA can be used to aid in metricsr aquatic restoration projects. We formulated a setl metrics for the hypothetical Black River restorationveloped a taxonomy of weighted project objectives,e initial metrics set to remove dominated metrics, andlied MAUT and ProMAA to develop a list of metricsrding to their importance as a function of a set of stake-

erences and utility functions of each metric with respecterion and sub-criterion. Utility-based methods are bet-

this type of analysis than value-based methods becauseate the value of metrics into utility to stakeholders.roMAA can incorporate uncertainty in stakeholder pre-

at can be useful when a real elicitation of preferences is cases where stakeholder preferences are highly uncer-er, in our hypothetical case study, MAUT and ProMAAw signicant differences in the predicted utility andhe selected metrics.ed to the common metrics selection methods presentedduction (best professional judgment, historical prece-eptual modeling, screening using established criteriaalytic Hierarchy Process (Saathy, 1980)) MCDA is more

M. Convertino et al. / Ecological Indicators 26 (2013) 7686 85

comprehensive and inclusive, incorporating expert opinion ona variety of subjects and stakeholder preferences from severalelds. This method allows planners to simplify complex situationswith varied and often conicting options, objectives, and opinions.New and chevaluated. Iincluding wclearly justThe MCDA project manThe quantitcompare ea

MCDA cby some liminput fromand more eods. Thoughcriteria andogy is involbasing critestakeholderjudgment tterion. Smaalternative input informalternativeswith respecmetrics werThis requirewith respecthe certaint

Usually concerns, aholder invoto participais designednot includeAnother methat will bebination of precedencethe original

An MCDholders areand/or highconsideratimanagemenated, for exathe monitoruse screeninlist of metrous methodpractice of and monito

Acknowled

The authBenet Ana(http://cw-acknowledgSDSS availaedged for tthe theoretand two antheir constr

project Decision and Risk Analysis Applications EnvironmentalAssessment and Supply Chain Risks for his research at the Riskand Decision Science Team, Engineering Research and Develop-ment Center, U.S. Army Corps of Engineers, in Concord, MA, USA.

as B, andhis rnt & . Armaterif thepons

dix A

plem in th0.00

nces

., Coviration

V., Steroach.V., Steroach.urt, LS, http.P., VMETHs, D.,

PhD rs/Da.C., 20://dx.dino, Mction ene://cw., Bey

catorski, D.,

lysis ow. Sur, M.,n extis. Ecold, J.G.or. Ecoch, Mciplesies, htngendomer,etlan

e of W, J., Gre of th., 2005312.oulouoach

J. Opes, A.Ptal du

N., 1nce w.B., Ketal sc, 3578is, S., tricity

R.L., 1. Perf

R.L., Gctives

R.L., Re Tradanging information can also easily be integrated andnterested parties can review components of the modeleights and alternative scores, and decision makers canify management choices according to model results.method for metrics selection thus enables restorationagers to make systematic, and transparent decisions.

ative results allow decision makers to clearly and easilych alternative and select the optimal metric set.an be extremely benecial, but it is also characterizeditations. Because it is so comprehensive and it includes

a variety of stakeholders, it can be time consumingxpensive than other, simpler metrics selection meth-, technically the project team alone could specify the

weights, one of the cornerstones of MCDA methodol-ving stakeholders in the decision process. This meansria and weighting partially on preferences elicited froms. It also takes a signicant amount of work and experto assign value scores to each alternative for every cri-ll increases in the amount of evaluation criteria andchoices translate to much larger increases in requiredation. The number of evaluation criteria and metric

is limited because each alternative must be evaluatedt to each criterion. For example, in this case study 25e evaluated with respect to 7 criteria and 6 sub-criteria.d 325 expert judgments of the value of each alternativet to each criterion as well as another 325 estimates ofy of those values.involving complex projects with serious stakeholder

successful MCDA evaluation often depends on stake-lvement, and is therefore limited by their willingnesste (Linkov and Moberg, 2011). Also, the methodology

to narrow down chosen metric alternatives and does guidance for choosing the original larger metric set.thod must be used to generate the metric alternatives

included in the MCDA analysis; perhaps one or a com-the previously mentioned methods, such as historical

or best professional judgment, can be used to assemble metric set.A technique should be used when many diverse stake-

interested in the project, and the situation is complex-prole with several objectives and alternatives underon. It is also useful with projects involving adaptivet as the situation can easily be updated and reevalu-mple by new ecosystem restoration design, consideringed metrics. Restoration managers may nd it useful tog before or after MCDA techniques to generate an initial

ics. We believe that overall this study provides a rigor-ological and computational advancement to the currentmetrics a utility-based selection for restoration designring, and in general for environmental management.

gements

ors acknowledge the support of the Environmentallysis Program of the U.S. Army Corps of Engineers

environment.usace.army.mil/eba/). Dr. B. Yatsalo ised for making a demonstration version of DECERNS-ble. D. Dokukin, and J.M. Keisler are kindly acknowl-heir support with DECERNS-SDSS and for the basis ofical framework respectively. The Editor, Dr. F. Mulleronymous reviewers are gratefully acknowledged foructive comments. M.C. acknowledges the funding of

C. Lu wnologywhen tronmethe U.Sthis mthose oother s

Appen

Supfound,2012.1

Refere

Allen, Eresto

Belton, App

Belton, App

BettencoPNA

Brans, JPRO

Braziunaties,pape

Clark, Whttp

ConvertSeletal B(http

Dale, V.Hindi

DiakoulaAnaRene

Drechslelatiosion

EhrenfelRest

EmmeriPrinStud

Faber-LaT., Cfor WOfc

FigueiraStat

Gass, S.I308

GeorgopapprEur.

Grootjancoas

Holmes,defe

Huang, Imen(19)

Kaminarelec

Keeney,Hum

Keeney,obje

Keeney,ValuSc and MSc student at Massachusetts Institute of Tech- research intern at the Risk and Decision Science Groupesearch was performed. C. Lu is currently at AMEC Envi-Infrastructure, Oakland, CA. Permission was granted byy Corps of Engineers Chief of Engineers to publish

al. The views and opinions expressed in this paper are individual authors and not those of the U.S. Army, ororing organizations.

. Supplementary data

entary data associated with this article can bee online version, at http://dx.doi.org/10.1016/j.ecolind.5.

ngton, W.W., Falk, D.A., 1997. Developing the conceptual basis for ecology. Restor. Ecol. 5 (4), 275276.wart, T., 2002a. Multiple Criteria Decision Analysis: An Integrated

Kluwer Academic Publishers, Dordrecht.wart, T., 2002b. Multiple Criteria Decision Analysis: An Integrated

Kluwer Academic Publishers, Boston..M.A., Kaur, J., 2011. Evolution and structure of sustainability science.://dx.doi.org/10.1073/pnas.1102712108.

incke, P., 1985. A preference ranking organisation method: theEE method for MCDM. Manag. Sci. 31 (6), 647656.

2012. Decision-theoretic elicitation of generalized additive utili-thesis, University of Toronto, http://www.cs.toronto.edu/darius/rius Braziunas PhDThesis.pdf07. Sustainability science: a room of its own. PNAS 104 (6), 17371738,oi.org/10.1073/pnas.0611291104.., Baker, K., Lu, C., Vogel, J., McKay, K., Linkov, I., 2012. Metricfor Ecosystem Restoration, ERDC Technical Note, Environmen-ts Analysis Program, US Army Corps of Engineering, forthcoming-environment.usace.army.mil/eba/index.cfm).eler, S.C., 2001. Challenges in the development and use of ecological. Ecol. Ind. 1, 310.

Karangelis, F., 2007. Multi-Criteria Decision Analysis and Cost-Benetf alternative scenarios for the power generation sector in Greece.stain. Energy Rev. 11/4, 716727.

Karin, F., Ilkka, H., OHara, R.B., Wissel, C., 2003. Ranking metapopu-nction risk: from patterns in data to conservation management deci-. Appl. 13, 990998, http://dx.doi.org/10.1890/1051-0761(2003)13., 2000. Dening the limits of restoration: the need for realistic goals.l. 8 (1), 29.

., Deutz, A., 2006. Multicriteria Optimization and Decision Making,, Algorithms and Case Studies, Leiden Institute of Advances Computertp://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.162.694oen, D., Rocchio, J., Schafale, M., Nordman, C., Pyne, M., Teague, J., Foti,

P., 2006. Ecological Integrity Assessment and Performance Measuresd Mitigation. Final Report to US Environmental Protection Agency etlands, Oceans and Watersheds, NatureServe, Arlington, VA.

eco, S., Ehrgott, M. (Eds.), 2005. Multiple Criteria Decision Analysis:e Art Surveys. Springer Science+Business Media, Boston, USA.. Model world: the great debateMAUT versus AHP. Interfaces 35 (4),

, E., Lalas, D., Papagiannakis, L., 1997. A multicriteria decision aidfor energy planning problems: the case of renewable energy option.rat. Res. 103, 483500.., Geelen, H.W.T., Jansen, A.J.M., Lammerts, E.J., 2002. Restoration ofne slacks in the Netherlands. Hydrobiologia 478, 181203.991. Post-project appraisal of conservation enhancements of oodorks. R&D Report 285/1/A, National Rivers Authority, Reading UK.isler, J., Linkov, I., 2011. Multi-criteria decision analysis in environ-iences: ten years of applications and trends. Sci. Total Environ. 4093594, http://dx.doi.org/10.1016/j.scitotenv.2011.06.022.Tsoutsos, T., Agoris, D., Machias, A., 2006. Assessing renewables-to-

systems: fuzzy expert system model. Energy Policy No 34, 13571366.977. The art of assessing multiattribute utility functions. Organ. Behav.orm. 19 (August (2)), 267310.regory, R.S., 2005. Selecting attributes to measure the achievement of. Operat. Res. 53 (1), 111.aiffa, H., 1976. Decision with Multiple Objectives: Preferences andeoffs. John Wiley & Sons, New York.

86 M. Convertino et al. / Ecological Indicators 26 (2013) 7686

Kondolf, G.M., 1995. Five elements for effective evaluation of stream restoration.Restor. Ecol. 3 (2), 133136.

Lahdelma, R., Hokkanen, J., Salminen, P., 1998. SMAAstochastic multi-objectiveacceptability analysis. Eur. J. Operat. Res. 106, 137143.

Lahdelma, R., Makkonen, S., Salminen, P., 2004. Treating dependent uncertaintiesin multi-criteria decision problems. In: Rao, M.R., Puri, M.C. (Eds.), OperationalResearch and Its Applications: Recent Trends, Vol. II, New Delhi. , pp. 427434.

Linkov, I., Moberg, E., 2011. Multi-Criteria Decision Analysis: Environmental Appli-cations and Case Studies. CRC Press.

Loken, E., 2007. Use of multicriteria decision analysis methods for energy planningproblems. Renew. Sustain. Energy Rev. 11, 15841595.

Martine, M., Cockeld, G., 2008. Managing trade-offs in landscape restoration andrevegetation projects. Ecol. Appl. 18, 20412049, http://dx.doi.org/10.1890/07-1328.1.

McKay S.K., Linkov I., Fischenich J.C., Miller S.J., Valverde L.J. Jr., 2011. EcosystemRestoration Objectives and Metrics, USACE ERDC Technical Note.

Mexas, M.P., Goncalves Quelhas, O.L., Gomes Coasta, L., 2012. Prioritization of enter-prise resource planning systems criteria: Focusing on construction industry. Int.J. Product. Econ., http://dx.doi.org/10.1016/j.ijpe.2012.05.025.

National Academy of Sciences. 2012. Ecosystem Services: Charting a Pathto Sustainability, Interdisciplinary Research Team Summaries, NationalAcademy of Sciences Press, http://download.nap.edu/cart/download.cgi?&record id=13331&free=1

Nesloa, R.E.J., Cooke, R.M., 2011. Modeling and validating stakeholder pre-ferences with probabilistic inversion. Appl. Stoch. Model. Bus. Ind.,http://dx.doi.org/10.1002/asmb.888.

Niemeijer, D., de Groot, R.S., 2008. A conceptual framework for selecting environ-mental indicator sets. Ecol. Indicat. 8, 1425.

Nienhuis, P.H., Bakker, J.E., Grootjans, A.P., Gulati, R.D., De Jonge, V.N., 2002. Thestate of the art of aquatic semi-aquatic ecological restoration projects in theNetherlands. Hydrobiologia 478, 219233.

Noss, R.F., 1999. Assessing and monitoring forest biodiversity: a suggested frame-work and indicators. Forest Ecol. Manage. 115, 135146.

Palmer, M.A., Reidy Liermann, C.A., Nilsson, C., Florke, M., Alcamo, J., Lake, P.S., Bond,N., 2008. Climate change and the Worlds River Basins: anticipating managementoptions. Front. Ecol. Environ. 6, 8189.

Pauly, D., Christensen, V., Guenette, S., Pitcher, T.J., Sumaila, U.R., Walters, C.J., Wat-son, R., Zeller, D., 2002. Towards sustainability in world sheries. Nature 418,689695.

Phillis, Y.A., Andriantiatsaholiniana, L.A., 2001. Sustainability: an ill-dened conceptand its assessment using fuzzy logic. Ecol. Econ. 29, 235252.

Pohekar, S.D., Ramachandran, M., 2004. Application of multi-criteria decision mak-ing to sustainable energy planninga review. Renew. Sustain. Energy Rev. 8 (4),365381.

Reichert, P., Borsuk, M., Hostmann, M., Schweizer, S., Sprri, C., Tockner, K., Truffer,B., 2007. Concepts of decision support for river rehabilitation. Environ. Model.Software 22, 188201.

Rohde, S., Kienast, F., Brgi, M., 2004. Assessing the restoration success of riverwidenings: a landscape approach. Environ. Manage. 34, 574589.

Saathy, T.L., 1980. The Analytic Hierarchy Process: Planning, Priority Setting,Resource Allocation. McGraw-Hill, New York.

Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., 2008. GlobalSensitivity Analysis: The Primer. John Wiley & Sons Ltd.

Seager, T.P., Satterstrom, F.K., Linkov, I., Tuler, S.P., Kay, R., 2007. Typological reviewof environmental performance metrics (with illustrative examples for oil spillresponse). Integr. Environ. Assess. Manag. 3 (3), 310321.

Settin, T., Botter, G., Rodriguez-Iturbe, I., Rinaldo, A., 2007. Numerical studies onsoil moisture distributions in heterogeneous catchments. Water Resour. Res.43, W05425.

Sigrid, S., 2004. Valuation for sustainable development the role of multicriteriaevaluation. Vierteljahreshefte zur Wirtschaftsforschung (Quart. J. Econ. Res.) 73(1), 110.

Tervonen, T., Figueira, J.R., 2008. A survey on stochastic multicriteria acceptabilityanalysis methods. J. Multi-Criteria Decision Anal. 15 (12), 114.

Thayer G.W., Merkey D.H., McTigue T.A., Burrows F.M., Salz R.J., Gayaldo P.F., 2005.Science based restoration monitoring of coastal habitats. Decision AnalysisSeries, No. 23 Vol. 2. NOAA.

Thom, R.M., Wellman, K.F., 1996. Planning aquatic ecosystem restoration monitoringprograms. Pacic Northwest National Laboratory, Richland, Washington; nalreport prepared for US Army Corps of Engineers, Institute for Water Resources,Alexandria, VA, IWR Report 96-R-23.

Tsoutsos, T., Tsouchlaraki, A., Tsiropoulos, M., Serpetsidakis, M., 2009. Visual impactevaluation of a wind park in a Greek island. Appl. Energy 86 (5), 15871600.

U.S. Army Corps of Engineers (USACE). 1999. Civil Works Ecosystem RestorationPolicy. ER 1165-2-501.

von Winterfeldt, D., Edwards, W., 1986. Decision Analysis and Behavioral Research.Cambridge University Press, Cambridge.

Wood, M.D., Bostrom, A., Bridges, T., Linkov, I., 2012. Cognitive mapping tools: reviewand risk management needs. Risk Anal. 32 (8), 13331348.

World Economic Forum. 2011. Global Risks 2011 Sixth Edition: An initiative ofthe Risk Response Network, World Economic Forum Press, http://reports.weforum.org/global-risks-2011/

2011. DECERNSSDSS: Decision Evaluation in Complex Risk Network Systems.www.decerns.com

Yatsalo, B.I., Gregory, A., Kiker, A., Kim, J., Bridges, T.S., Seager, T.P., Gardner, K., Sat-terstrom, F.K., Linkov, I., 2007. Application of multicriteria decision analysis toolsto two contaminated sediment case studies. Integr. Environ. Assess. Manag. 3(2), 223233.

Multi-criteria decision analysis to select metrics for design and monitoring of sustainable ecosystem restorations1 Introduction2 Materials and methods2.1 Hypothetical aquatic ecological restoration2.2 Multi-criteria decision analysis (MCDA)2.3 Multi-attribute utility theory (MAUT)2.4 Probabilistic multi-criteria acceptability analysis (ProMAA)

3 Results and discussion4 Conclusions and perspectivesAcknowledgementsAppendix A Supplementary dataAppendix A Supplementary data