Eikos a Simulation Toolbox for Sensitivity Analysis

8/2/2019 Eikos a Simulation Toolbox for Sensitivity Analysis

1/57

UPTEC IT05015

Degree Project 20 p

February 2005

Eikos

A Simulation Toolbox for Sensitivity Analysis

Per-Anders Ekstrom


2/57

This page intentionally left blank


3/57

Faculty of Science and Technology

The UTH-unit

Visiting address:

Angstromlaboratoriet

Laderhyddsvagen 1

House 4, Level 0

Postal address:

Box 536

SE-751 21 Uppsala

SWEDEN

Telephone:+46 (0)18471 30 03

Telefax:

+46 (0)18471 30 00

Homepage:

http://www.teknat.uu.se/student

Abstract

Eikos

A Simulation Toolbox for Sensitivity Analysis

Per-Anders Ekstrom

Computer-based models can be used to approximate real life processes. These mod-els are usually based on mathematical equations which are dependent on severalvariables. The predictive capability of models is therefore limited by the uncer-tainty in the value of these. Sensitivity analysis is used to apportion the relativeimportance each uncertain input factor has on the output variation. Sensitivityanalysis is therefore an essential tool in simulation modeling and for performing riskassessments.Simple sensitivity analysis techniques based on fitting the output to a linear equationare often used, for example correlation or linear regression coefficients. These meth-ods work well for linear models, but for non-linear models their sensitivity estimationsare not accurate. Usually models of complex natural systems are non-linear.Various software products exist for sensitivity analysis; unfortunately these productscannot efficiently be linked to the Matlab environment. The algorithms used in

these products are also hidden. Many model builders (in the radioecological field)develop their models in the Matlab/Simulink environment. A Matlab toolbox forsensitivity analysis is therefore interesting.Within the scope of this work, various sensitivity analysis methods, which can copewith linear, monotone as well as non-monotone problems, have been implemented,in Matlab, in a generic way. A graphical user interface has also been developed,from which the user easily can load or call their model and perform a sensitivityanalysis.The sensitivity analysis methods has been benchmarked with well-known test func-tions and compared with other sensitivity analysis software, with successful results.

With this toolbox, model builders can easily perform simple as well as advanced

sensitivity analysis on their models.

Supervisors: Rodolfo Avila Moreno and Robert Broed

Subject controller: Virgil Stokes

Examiner: Lars-Henrik Eriksson

UPTEC IT05015

Sponsors: Facilia AB, The Norwegian Radiation Protection Authority and

Posiva Oy


4/57



5/57

Sammanfattning

EikosEn verktygslada for kanslighetsanalys

Mitt examensarbete, som avslutar mina civilingenjors-studier vid Uppsala universitet, sammanfattas i dettadokument. Arbetet har utforts pa ett konsultforetag iStockholm som heter Facilia AB. Uppdrag som FaciliaAB far kan exempelvis vara att utfora riskanalyser vid

stralskydds- och karnavfallsfragor.

Vid genomforandet av en riskanalys anvands dator-baserade modeller for att uppskatta faktiska processeri naturen. Modellerna bygger oftast pa matema-tiska ekvationer, vilka i sin tur ar beroende av enstor mangd faktorer. Exempel pa sadana faktorerar forangningshastighet, markens vattenkoncentration,tjockleken hos markens rotlager och tidsfordrojning.Modellernas mojlighet att forutspa trovardiga resultatar darmed kraftigt begransad av osakerheten hos dessafaktorer.

Kanslighetsanalys anvands till att hitta de osakra fak-torer som paverkar modellens resultat i storst omfat-tning. En sadan sanalys ar darfor en viktig komponenti arbetet med att skapa modeller och att genomf orariskanalyser.

Malet med mitt arbete pa Facilia AB var ursprungligenuppdelat i tre delar:

Forst och framst skulle jag implementerakanslighetsanalysmetoder som kan hanterade typer av modeller som oftast anvands forriskanalyser av ekosystem.

Darefter skulle dessa modeller testas ochriktigheten sakerstallas med hjalp av kandatestfunktioner och modeller utvecklade paFacilia AB.

Slutligen skulle aven programvara med ettanvandarvanligt grafiskt granssnitt utveck-las, dvs. det skulle vara latt for anvandarenatt utnyttja.

Facilia har utvecklat en plattform, kallad Ecolego,som ar sarskilt anpassad for simulering och modeller-

ing av ekologiska system. Denna plattform anvander

sig av Matlab och da speciellt verktyget Simulink sommatematisk motor. Av den anledningen har det varitnaturligt att utfora arbetet i Matlab-miljon.

Kanslighetsanalysmetoderna som inforts har jag delatupp i tre olika kategorier. Forsta typen ar screening-metoder, vilka kan anvandas i ett tidigt stadium da mankanske har hundratals med osakra faktorer, eller om endatabehandling av modellen tar mycket lang tid. Dessametoder kan med relativt fa antal databehandlingaridentifiera modellens viktigaste osakerhetsfaktorer.Den andra typen av metoder ar samplings-baserademetoder. Dessa metoder bygger pa Monte Carlo-simuleringar och forutsatter ett linjart samband mel-lan vardena pa faktorerna och modellens resultat. Omdet linjara sambandet ar uppfyllt kan dessa metoderval uppskatta hur stor del av osakerheten hos model-lens resultat en viss faktor bidrar med. Den sista typen

av metoder har kallats varians-baserade metoder. Avendessa metoder anvander sig av Monte Carlo-tekniker,men till skillnad mot de samplings-baserade metodernaar de relativt komplicerade, och kraver manga modell-databehandlingar. Fordelen med de varians-baserademetoderna ar att de ar modelloberoende, dvs. de haringa krav pa att modellens resultat ska ha ett linjartberoende av faktorerna.

Diverse programvara for kanslighetsanalys finns idagpa marknaden, men tyvarr sa kan ingen av dessa paett enkelt satt kopplas till Matlab-miljon. Algorit-merna som anvands i dessa produkter ar ocksa gomda

for anvandaren. Manga modellbyggare (inom det ra-dioekologiska omradet) utvecklar sina modeller i enMatlab/Simulink-miljo. En verktygslada for kans-lighetsanalys utvecklad i Matlab ar darfor av stortintresse.

De implementerade kanslighetsanalysmetoderna hartestats med valkanda testfunktioner och jamforts medannan programvara for kanslighetsanalys, med lyckaderesultat.

Med denna verktygslada, dopt till Eikos, kan modell-byggare utfora enkel saval som avancerad kanslighets-

analys pa sina modeller.


6/57



7/57

TABLE OF CONTENTS

Table of Contents

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Uncertainty analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.3 Sensitivity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.4 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.5 Outline of the report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Monte Carlo simulations 3

2.1 Simple random sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 Latin hypercube sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.3 Rank order correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Probability distributions 6

3.1 Normal (Gaussian) distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.2 Log-normal distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.3 Uniform (Rectangular) distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.4 Log-uniform distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.5 Triangular distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.6 Log-triangular distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.7 Exponential distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.8 Extreme value (Gumbel) distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.9 Beta distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.10 Gamma distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.11 Chi-square distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.12 Weibull distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Model evaluations 15

5 Screening methods 16

5.1 Morris method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

6 Sampling-based methods 18

6.1 Graphical methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6.2 Regression analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6.3 Correlation coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

6.4 Rank transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

6.5 Two-sample tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

i


8/57

TABLE OF CONTENTS

6.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

7 Variance-based methods 21

7.1 Sobol indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

7.2 Jansen (Winding Stairs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

7.3 Fourier Amplitude Sensitivity Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

7.4 Extended Fourier Amplitude Sensitivity Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

7.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

8 EikosThe application 26

8.1 The main window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

8.2 Using a Simulink model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

8.3 Using a standard Matlab function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

8.4 The distribution chooser window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

8.5 The correlation matrix window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

8.6 The uncertainty analysis window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

8.7 The sensitivity analysis window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

8.8 The scatter plot window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

8.9 The variance-based methods window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

8.10 The Morris screening window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

8.11 The Monte Carlo filtering window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

8.12 Exporting and saving results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

9 Results of benchmarking 31

9.1 Test of code for inducing rank correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

9.2 Benchmarking of procedure for generating probability distributions . . . . . . . . . . . . . . . . . . 31

9.3 Benchmarking of SA methods for linear models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

9.4 Benchmarking of SA methods for non-linear monotonic models . . . . . . . . . . . . . . . . . . . . 32

9.5 Benchmarking of SA methods for non-monotonic models . . . . . . . . . . . . . . . . . . . . . . . . 33

9.5.1 The Sobol g-function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

9.5.2 The Ishigami function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

9.5.3 The Morris function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

10 Example of practical application 36

11 General discussion 40

12 Acknowledgements 41

A Appendix I

ii


9/57

TABLE OF CONTENTS

A.1 Computer code for Iman & Conover rank correlation . . . . . . . . . . . . . . . . . . . . . . . . . . I

A.2 Computer code for the inverse of the CDF of the gamma distribution . . . . . . . . . . . . . . . . I

A.3 Computer code for Morris design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IIA.4 Computer code for FAST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I II

A.5 Computer code for EFAST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV

iii


10/57



11/57

1.2 Uncertainty analysis 1 INTRODUCTION

1 Introduction

This work has been carried out as a Degree Project to

obtain a M.Sc. in Information Technology Engineeringat Uppsala University. The work has taken place at,and been organized by, Facilia AB, Stockholm. Exter-nal funding has been provided by the Norwegian Ra-diation Protection Authority (NRPA) and the FinnishNuclear Waste Management Company, Posiva Oy. Su-pervisors for the project have been Dr. Rodolfo AvilaMoreno, director of Facilia AB and Robert Broed, Ph.D.student at the Department of Nuclear Physics, Stock-holm University.

1.1 BackgroundFacilia AB is a scientific consultant company activein the fields of environmental and health risk assess-ments, safety assessments in radioactive waste manage-ment and radiation protection.

Facilia AB has developed a platform (Ecolego) for simu-lation modeling and environmental risk assessment basedon Matlab/Simulink and Java. This platform shouldinclude procedures for sensitivity analysis, which can beused to assess the influence of different model param-eters on simulation endpoints. Such assessments are

needed for ranking the parameters in order of impor-tance serving various purposes, for example:

(a) to identify major contributors to the uncer-tainties in the predictions with a model, or

(b) to identify research priorities to improve riskassessments with a model.

Simulations with computer models are used for the riskassessment of systems that are too complex to analysedirectly.

In the following text, the model is seen as a black boxfunction f with k input factors x = (x1, x2, . . . , xk) andone single scalar output y, i.e.

y = f(x1, x2, . . . , xk). (1.1)

In an application the dimensionality of y may of coursebe a vector instead of a single scalar value. That willbe determined by the function or program of interest.

The term input factor is used to denote any quantitythat can be changed in the model prior to its execution.It may be, for example, a parameter, an initial value, a

variable or a module of the model.

During this project, a Matlab toolbox has been de-veloped, the name of the toolbox is Eikos1. In Eikos amodel can be a Matlab function, a general Simulink

model or any type of stand-alone executable code thathas been compiled with a Matlab wrapper. Matlabsupports C/C++, FORTRAN and Java.

1.2 Uncertainty analysis

When pursuing risk assessments, the general features ofthe modeled systems are usually quite well understood,but problems may arise as there exist uncertainties insome important model inputs. Thus the input factorsof models are unfortunately not always known with asufficient degree of certainty. Input uncertainty can be

caused by natural variations as well as by errors anduncertainties associated with measurements.

In this context, the model is assumed to be determin-istic, i.e. the same input data would produce the sameoutput if the model is run twice. Therefore, the in-put uncertainties are the only uncertainty propagatedthrough the model affecting the output uncertainty.

The uncertainty of input factors is often expressed interms of probability distributions; it can also be speci-fied by samples of measured values, i.e. empirical prob-ability distributions. The uncertainties of the different

input factors may have dependencies on each other, i.e.they may be correlated2.

Generally, the main reason of performing an uncertaintyanalysis is to assess the uncertainty in the model outputthat derives from uncertainty in the inputs. A questionto be investigated is How does y vary when x variesaccording to some assumed joint probability distribu-tions?.

1.3 Sensitivity analysis

Andrea Saltelli [Sal00] states that Sensitivity analysis(SA) is the study of how the variation in the output ofa model (numerical or otherwise) can be apportioned,qualitatively or quantitatively, to different sources ofvariation, and how the given model depends upon theinformation fed into it.. Saltelli also list a set of reasonswhy modelers should carry out a sensitivity analysis,these are to determine:

(a) if a model resembles the system or processes

1Eikos Ancient Greek word for To be expected with some de-gree of certainty.2Correlation A term used to describe the degree to which onefactor is related to another.

1


12/57

1 INTRODUCTION 1.4 Objective

under study;

(b) the factors that mostly contribute to theoutput variability;

(c) the model parameters (or parts of the modelitself) that are insignificant;

(d) if there is some region in the space of in-put factors for which the model variation ismaximal;

(e) the optimal regions within the space of thefactors for use in a subsequent calibrationstudy;

(f) if and which (group of) factors interact with

each other.

Sensitivity analysis aims at determining how sensitivethe model output is to changes in model inputs. Wheninput factors are relatively certain, we can look at thepartial derivative of the output function with respect tothe input factors. This sensitivity measure can easily becomputed numerically by performing multiple simula-tions varying input-factors around a nominal value. Wewill find out the local impact of the factors on the modeloutput and therefore techniques like these are called lo-cal SA. For environmental and health risk assessments,

input factors will often be uncertain and therefore localSA techniques will not be usable for a quantitative anal-ysis. We want to find out which of the uncertain inputfactors are more important in determining the uncer-tainty in the output of interest. To find this we need toconsider global SA, which are usually implemented us-ing Monte Carlo simulations and are, therefore, calledsampling-based methods.

Different SA techniques will do well on different typesof model problems. At an initial phase, for models witha large amount of uncertain input factors, a screeningmethod could be used to qualitatively find out which

the most important factors are and which are not im-portant. The screening method implemented in Eikosis the Morris design [Mor91]. A natural starting pointin the analysis with sampling-based methods would beto examine scatter plots. With these, the modeler cangraphically find out nonlinearities, nonmonotonicity andcorrelations between the input-output factors. For lin-ear models, linear relationship measures like Pearsonproduct moment correlation coefficient (CC), PartialCorrelation Coefficients (PCC) and Standardized Re-gression Coefficients (SRC) will perform well. For non-linear but monotonic models, measures based on ranktransforms like Spearman Rank Correlation Coefficient

(RCC), Partial Rank Regression Coefficient (PRCC)

and Standardized Rank Regression Coefficients (SRRC)will perform well. For non-linear non-monotonic mod-els, methods based on decomposing the variance are

the best choice. Examples of these methods are theSobol method, Jansens alternative, the Fourier Am-plitude Sensitivity Test (FAST) and the Extended Fou-rier Amplitude Sensitivity Test (EFAST). Methods ofpartitioning the empirical input distributions accord-ing to quantiles3 (or other restrictions) of the outputdistribution are called Monte Carlo filtering. Measuresof their difference are called two-sample-tests. Twonon-parametric4 two-sample-tests are implemented,the Smirnov test and the Cramer-von Mises test.

1.4 Objective

This work deals with implementing SA methods thatcan cope with the type of models commonly used inenvironmental risk assessment. The work also includetesting the implemented methods with well known testfunctions and environment risk assessment models de-veloped at Facilia AB, as well as providing user friendlysoftware written in Matlab that later-on can be incor-porated into the Ecolego-platform.

1.5 Outline of the report

The remainder of this report begins with a section de-scribing the Monte Carlo method in Section 2. Samplegeneration and how to induce dependencies on the in-put factor distributions are subjects considered.

Elementary probability theory and statistics is subjectof Section 3. Also, the properties of various probabilitydistributions offered in Eikos are described.

Section 4 describes how the input-output mapping isdone in Eikos, i.e. how to evaluate the models withdifferent inputs to obtain outputs.

The implemented methods of sensitivity analysis aredescribed in Section 5-7, including screening designs,graphical techniques and the usual global SA meth-ods. The last of these sections (Section 7) describesthe state-of-the-art variance-based methods.

A presentation of the graphical user interface (GUI) ofEikos comes in Section 8. The GUI has been imple-mented so that the SA process can be performed easily.

Results of benchmarking the probability distributions

3The q-quantile of a random variable X is any value x such thatthe probability P(X x) = q.4Non-parametric minimal or no assumptions are made aboutthe probability distributions of the factors being assessed.

2


13/57

2 MONTE CARLO SIMULATIONS

generated in Eikos with commercial systems are shownin Section 9. Comparisons of results from running knowntest-functions as well as a test of the codes for induc-

ing rank correlation are presented. Section 10 presentsa sensitivity study of a risk assessment model imple-mented in Ecolego.

Finally in Section 11 a general discussion is given.

Throughout the report, Matlab code snippets imple-menting parts of the algorithms, has been included tofacilitate the understanding by the reader and make thetheory easier to understand. In Appendix, larger, morecomplete Matlab codes for some of the implementedmethods are collected.

2 Monte Carlo simulations

Monte Carlo (MC) methods are algorithms for solvingvarious kinds of computational problems by using ran-dom numbers [tfe05]. It includes any technique of sta-tistical sampling employed to approximate solutions toquantitative problems. MC sampling refers to the tech-nique of using random or pseudo-random5 numbers tosample from a probability distribution, it is a conven-tional approach to solve uncertainty analysis problems.

Figure 1: A schematic view of sampling-based sensitiv-ity analysis, Saltelli [Sal00].

A MC simulation is based on performing multiple modelevaluations with probabilistically selected model inputs.The results of these evaluations can be used to deter-

5The prefix pseudo- is used since the random number is generatedby a computational algorithm and not from a truly randomevent such as radioactive decay.

mine the uncertainty in the model output (prediction)and to perform SA.

Eikos is a Matlab utility designed to assist the user

in the implementation of a MC simulation, followedby a sensitivity analysis based on the MC simulation(sampling-based SA). The main steps of a SA based ona MC simulation are:

(1) For the defined model f with input factorsxi, i = 1, . . . , k and output y, ranges and dis-tributions are selected for each input factor.These k distributions D1, D2, . . . , Dk charac-terizes the uncertainty in the input factors.An expert in the risk assessment field mustdefine these distributions.

(2) Samples, X = X1, X2, . . . , Xk, correspond-ing to the ranges and distributions definedin the previous step are generated. Thesesamples can be thought of as a matrix whichcontains, for N model runs, the sampled val-ues of each of the k input factors under ex-amination.

X =

x11 x12 x1kx21 x22 x2k

......

. . ....

xN1 xN2 xNk

(2.1)

(3) The model is evaluated N times, once foreach element of the sample, creating a map-ping in the input-output space. The pro-duced output is Y = f(X1, X2, . . . , Xk).

(4) The output of the previous step is used asthe basis for the uncertainty analysis. Theuncertainty can for example be character-ized with a mean value and a variance.

(5) In the last step the mapping between Y andX is used by various sensitivity analysis tech-

niques to rank the importance of the inputfactors.

To recreate the input factor distributions through sam-pling by the MC method, a large number of iterationsmay be required. If too few iterations are made, thevalues in the outer ranges of the distributions may notbe represented in the samples and thus their impact onthe results will not be included in the simulation out-put. To address this problem the sampling techniquecalled Latin Hypercube Sampling (LHS) was developed.LHS is a stratified sampling technique that guaranteessamples to be drawn over the whole range of the distri-

bution. LHS is described in Section 2.2.

3


14/57

2 MONTE CARLO SIMULATIONS 2.2 Latin hypercube sampling

2.1 Simple random sampling

To generate random samples from any distribution one

usually starts with uniform random numbers. With uni-form random numbers one can produce random num-bers from other distributions, either directly or by usinginversion or rejection methods.

In Eikos, the inversion method is used to replicate theexpected distributions. The sampling procedure of adistribution can be divided into two steps:

(1) generate uniformly distributed numbers inthe interval [0, 1], and then

(2) deterministically transform these numbersinto the wanted probability distributions us-

ing the inversion method.

The inversion method works due to a fundamental the-orem that relates the uniform distribution to other con-tinuous distributions:

Theorem 1 If is a continuous distribution with in-

verse 1, and x is a uniform random number, then

1(x) has distribution .

So, to generate a random sample from any probabil-ity distribution, a random number r is first generated,

drawn with equal probability in the interval [0, 1]. Thisvalue is then used in the equation of the inverse distri-bution to determine the value to be generated for thedistribution:

1(r) = x (2.2)

Unfortunately, this approach is usually not the mostefficient.

With simple random sampling the samples are entirelyrandom in the sense that each sample is drawn indepen-dently from all other samples. Too few samples may notgive a good coverage of the entire range space.

If the computer code of the model is not time consumingand a large number of iterations can be performed, thenrandom sampling is the best technique to use, since ityields un-biased estimates of the desired output char-acteristics, for example the mean and the variance.

In Matlab there exists a very efficient uniform pseudo-random number generator named rand. This uniformrandom generator is state-of-the-art and extremely fastsince it does not use any multiplications or divisions atall. It is based on the work of George Marsaglia [Mar68]and has been shown to have a period as big as 21492.

The function call to generate uniform samples in Mat-

lab for the N k input factor matrix X would be:

X = rand(N,k);

2.2 Latin hypercube samplingLatin hypercube sampling (LHS) was introduced byMcKay, Conover and Beckman [MCB79] and is todayaccording to David Vose [Vos96] an essential feature inany risk analysis software package.

LHS is designed to accurately recreate the input distri-bution through sampling in fewer iterations comparingwith the simple random sampling. LHS is a so-calledstratified sampling technique where the random vari-able distributions are divided into equal probability in-tervals. The technique being used is known as stratified

sampling without replacement. This procedure ensuresthat each sub-interval for each variable is sampled ex-actly once, in such a way that the entire range of eachvariable is explored.

A LHS proceeds as follows:

(1) The probability distribution of an input fac-tor is split into N non-overlapping intervalsof equal marginal probability 1/N.

(2) In the first iteration, one of these intervalsis selected randomly.

(3) Within this interval one observation is cho-sen randomly and are now the first sampleof this specific input factor.

(4) In the second and the rest of the N iter-ations, one interval which has not alreadybeen chosen is randomly selected and theprocedure is repeated from step number (3)to obtain the N samples. Since there are Nsamples generated, each of the N intervalswill have been sampled-from only once.

(5) The whole procedure is repeated for each ofthe k input factors.

Hence, LHS is equivalent to uniformly sample from thequantiles of the distribution (equivalent to samplingthe vertical axis of the cumulative distribution functionCDF6) and then inverting the CDF to obtain the ac-tual distribution values that those quantiles represent[WJ98]. Therefore the LHS procedure can always startwith generating uniformly distributed samples in theinterval (0,1) and then the inverse of the CDF can beapplied to obtain the sought sample.

6The cumulative distribution function is described in Section 3.

4


15/57

2.3 Rank order correlation 2 MONTE CARLO SIMULATIONS

Figure 2: Division of the normal probability distribu-tion into 10 equiprobable intervals.

LHS allows an un-biased output mean with a smallernumber of runs, i.e. it will require fewer samples thansimple random samples for similar accuracy. One draw-back of LHS is that it can yield biased estimates of thesample variance.

To generate uniform LHS samples in Matlab for theNk input factor matrixX the following code is enough:

for i=1:kX(:,i) = (randperm(N)-rand(N,1))/N;

end

The built-in Matlab-function randperm(N) returns arandom permutation of the integers from 1 to N.

2.3 Rank order correlation

In the generation of samples we have assumed that theinput factors are independent of each other and shouldtherefore be uncorrelated. But since input factors could

be related to each other there is a need to be able toinduce a desired correlation structure onto the sample.

The technique to correlate input samples used in Eikoswas developed by Iman and Conover [IC82]. It is arestricted pairing technique that is based on rank cor-relations rather than sample correlations. When usingthe ranks the data is replaced with their correspondingranks. The ranks are defined as follow; the smallestvalue is assigned 1, the second smallest 2 and so on tothe largest value that is assigned rank N. The tech-nique involves a re-permutation of the columns of theN

k matrix X so that a desired rank-correlation struc-

ture results between the individual input factors. Iman

and Conover [IC82] lists the following desirable featureswith their method:

(a) It is distribution-free. That is, it may beused with equal facility on all types of dis-tribution functions.

(b) It is simple. No unusual mathematical tech-niques are required to implement the method.

(c) It can be applied to any sampling schemefor which correlated input variables can logi-cally be considered, while preserving the in-tent of the sampling scheme. That is, thesame number originally selected as input val-ues are retained; only their pairing is af-

fected to achieve the desired rank correla-tions. This means that in Latin Hypercubesampling the integrity of the intervals is main-tained. If some other structure is used forselection of values, that same structure isretained.

(d) The marginal distributions7 remain intact.

David Vose [Vos96] says that most risk analysis prod-ucts now offer a facility to correlate probability distri-butions within a risk analysis model using rank order

correlation.In Eikos the user has the possibility of defining a k k-dimensional correlation matrix C with the wanted cor-relation structure. The following 4-dimensional corre-lation matrix C tells us that the input factors one andtwo are positively correlated to a degree 0.5, the inputfactors two and four are negatively correlated to a de-gree 0.3 and the rest of the factors are fully uncorrelatedwith each other.

C =

1.0 0.5 0.0 0.0

0.5 1.0 0.0 0.3

0.0 0.0 1.0 0.00.0 0.3 0.0 1.0

(2.3)

If two factors are fully correlated, it corresponds to a 1in the correlation matrix C (that is why the diagonalconstitutes of ones). There are ways to define a corre-lation structure that is impossible, for example, if X1and X2 are highly positively correlated as well as X2and X3. Then X1 and X3 cannot be highly negativelycorrelated. In general, C is valid only if it is positive

7Given N random variables X1, . . . ,XN with joint probabilitydensity function (x1, . . . , xN), the marginal distribution ofxr isobtained by integrating the joint probability density over all vari-ables but xr: gr(xr) =

. . .

(x1, . . . , xN)dx1 . . . dxN.

5


16/57

3 PROBABILITY DISTRIBUTIONS

semi-definite. The Hermitian part8 of a positive semi-definite matrix has all positive eigenvalues. The codeto check for validity used in Eikos is:

% Hermitian part.

HP = 0.5*(C+C);

% Get eigenvalues.[V,D] = eig(HP) ;

eigenvals = diag(D);

% Look for non-positive values.if any(real(eigenvals) < 0)

error(Matrix is not positive semi-definite.);

end

Theoretical details of the rank correlation method canbe found in the original article by Iman and Conover[IC82]. The algorithm can be summarized by the fol-lowing seven steps, where R is the N

k input factor

matrix and C is the k k matrix corresponding to thewanted correlation matrix:

(1) Calculate the sample correlation matrix TofR.

(2) Calculate lower triangular cholesky decom-position P ofC, i.e. C = PPT.

(3) Calculate lower triangular cholesky decom-position Q ofT, i.e. T = QQT.

(4) Obtain S such that C = STST, can be cal-

culated as S = PQ1.

(5) Obtain Rscore by rank-transforming R andconvert to van der Waerden scores9.

(6) Calculate target correlation matrix R =RscoreS

T.

(7) Match up the rank pairing in R accordingto R.

A Matlab implementation of the rank correlation pro-cedure can be found in Appendix A.1.

8The Hermitian part of a real matrix A is defined as AH 1

2(A+AT).

9The van der Waerden scores are 1(i/(N + 1)), where 1

is the inverse CDF of the standard normal distribution, i is theassigned rank and N is the total number of samples.

3 Probability distributions

Some basic theory of probability and statistics is essen-

tial to understand for example the inversion techniqueused to generate samples of general distributions men-tioned in the previous section. In this section some el-ementary theory about distributions and few essentialmoments are therefore described. A list and some de-tails of available probability distributions in Eikos arelisted after the theory in this section.

If we consider the distribution of the uncertain inputvariable x. The cumulative distribution function (x)(CDF) gives the probability P that the variable X willbe less than or equal to x, i.e.

(x) = P(X x) (3.1)

The CDF has the following two theoretical properties:

(1) It is always non-decreasing.

(2) (x) = 0 at x = and (X) = 1 at x = .

The CDF therefore ranges from zero to one. Fig. 3shows the CDF of a normal distribution with = 0 and = 1.

Figure 3: CDF of a normal distribution, with = 0 and = 1.

The inverse of the CDF gives the value of (x) for agiven value of x. This inverse function 1((x)) iswritten as

1((x)) = x (3.2)

Fig. 4 shows the inverse of the CDF of a normal distri-

bution with = 0 and = 1.

6


17/57

3 PROBABILITY DISTRIBUTIONS

Figure 4: The inverse of the CDF of a normal distribu-tion, with = 0 and = 1.

The uncertainty of the input factors is described withprobability density functions (x) (PDF). A PDF hastwo theoretical properties:

(1) The PDF is zero or positive for every possi-ble outcome.

(2) The integral of a pdf over its entire range ofvalues is one.

The PDF of a continuous random variable X is the func-tion (x) such that

P(a < X b) =ba

(x)dx, (3.3)

for all real a and b, a < b. Fig. 5 shows the PDF of anormal distribution with = 0 and = 1.

Figure 5: PDF of a normal distribution, with = 0 and

= 1.

Since the random variable X is continuous the proba-bility of observing any particular value becomes zero.Therefore, the PDF needs to be integrated over the in-

terval of interest. The PDF could also be described asthe rate of change (the gradient) of the CDF:

(x) =d

dx(x). (3.4)

From this, it is easy to understand that the first ofthe theoretical properties of a PDF listed above is true.The PDF is zero or positive since the gradient of a non-decreasing curve is always non-negative. The probabil-ity of lying between any two exact values (a, b):

P(a x b) = (b) (a), where b > a. (3.5)

The expected value of a random variable X is denoted aseither or E(X). Other names for E(X) are the expec-tation, mean or average ofX. The expectation indicatesthe position of the center of mass of the distribution.Expectation is given by:

E(X) = =

+

x(x)dx, (3.6)

where (x) is the probability density function of X.

The mode of a random variable X with probability den-sity function (x) is the x-value at which (x) is the

largest, i.e. it is the most likely outcome. The mode isdenoted Mo.

The variance of a random variable X is denoted as ei-ther 2 or V(X). It is the average of the squared de-viations about the mean and measures how widely dis-persed the values are in the distribution. Variance isgiven by:

V(X) = 2 =

+

(x )2(x)dx, (3.7)

where = E(X). Variance can also be defined in justterms of expectations of the random variable X

V(X) = 2 = E[(X E(X))2] = E(X2) E(X)2. (3.8)

The positive square root of the variance ( =

V(X))is called the standard deviation.

In Eikos, distributions with infinite tails can be boundedor limited by truncation. If a probability distributionfunction is truncated, the characteristics of the PDFwill change. For example, if a normal distribution with = 0 and = 1 is truncated such that the probabil-ity for values smaller than -6 is zero, then the actualmean will be slightly larger than zero. The truncation

is implemented as follows: when generating samples, if

7


18/57

3 PROBABILITY DISTRIBUTIONS 3.2 Log-normal distribution

for example a left bound on the PDF is supposed tobe at -6, then the random uniform samples are drawnfrom the range [(6), 1] instead of [0, 1], these samples

are then used to get the needed ones by the inversionmethod.

3.1 Normal (Gaussian) distribution

The normal distribution is probably the most frequentlyused distribution in probability theory. Its popularitycomes from that many natural phenomena follow a sta-tistically normal distribution.

The normal distribution is specified with a mean value and a standard deviation . The PDF is defined as:

(x) = 1

2e

12 ( x

)2 (3.9)

Fig. 5 shows the normal PDF using a mean value of = 0 and standard deviation = 1. In Matlab thePDF can be computed as (x being the vector of X-values):

y = exp(-0.5*((x-mu)/sigma).^2)/(sqrt(2*pi)*sigma);

There is no analytic formula for the CDF () of a nor-mal distribution. The function exists, but cannot beexpressed in terms of other standard functions. There-fore both the CDF and its inverse is approximated bycomputer algorithms. In Matlab the CDF and its in-verse can be computed with the help of the error func-tion erf and its inverse erfinv respectively (or theircomplements). CDF can be computed with:

p = 0.5*erfc(-((x-mu)/sigma)/sqrt(2));

The inverse of CDF can be computed as (p being avector of uniform random samples):

p(p1) = NaN;r = (-sqrt(2)*sigma)*erfcinv(2*p)+mu;

Range: X Parameters: , 0 < E(X) =

Mo =

V(X) = 2

3.2 Log-normal distribution

As with the normal distributions many natural phenom-

ena occurs according to the log-normal distribution. It

is often used when most values occur near the minimum,i.e. the values are positively skewed10.

If the natural logarithm of a random variable X has

a normal distribution with mean and standard devi-ation , then its distribution is log normal. The log-normal PDF is given by:

(x) =1

x

2e

12 (

log x )

2

(3.10)

Fig. 6 shows the PDF of a log-normal distribution witha mean value = 0 and a variance = 1.

Figure 6: PDF of a log-normal distribution, with = 0and = 1.

In Matlab the PDF of the log-normal distribution iscomputed as:

y = exp(-0.5*((log(x)-mu)/sigma).^2)./ ...

(x*sqrt(2*pi)*sigma);

As with the normal distribution, no formula for theCDF and the inverse of the CDF exist for the log-normal distribution. Eikos uses function calls to thenormal distribution function to obtain the CDF and its

inverse. In Matlab the CDF of the log-normal distri-bution is computed as:

x(x


19/57

3.4 Log-uniform distribution 3 PROBABILITY DISTRIBUTIONS

Where invnorm is the function that returns the inverseof the CDF of a normal distribution.

Range: 0

X

Parameters: 0 < , 0 < E(X) = e+0.5

2

Mo = e2

V(X) = e2+22 e2+2

3.3 Uniform (Rectangular) distribution

If it is only possible to define a minimum and maximumvalue for a factor, a uniform distribution is appropriate.For a uniform distribution all values between the mini-mum and maximum value are equally likely to be sam-

pled. It is the simplest of all continuous distributions.Parameters are the upper and lower bounds, a and b.The uniform PDF is given as:

(x) =

0, x / (a, b)1

ba , x (a, b).(3.11)

Fig. 7 shows the PDF of an uniform distribution witha minimum value a = 0 and a maximum value b = 1.

Figure 7: PDF of an uniform distribution, with a = 0and b = 1.

In Matlab the uniform PDF can be computed as:

y = zeros(size(x));k = find(x>=a & x


20/57

3 PROBABILITY DISTRIBUTIONS 3.5 Triangular distribution

In Matlab the log-uniform PDF can be computed as:

y = zeros(size(x));

k = find(x>=a & xc & x b.

(3.18)

In Matlab the CDF of a triangular distribution canbe computed as:

p = zeros(size(x));

k = find(x>=a & xc & xb) = 1;

The inverse of the CDF of a triangular distribution isdefined as:

1(p) =

a +

p(b a)(c a), p < cabc

b

(1 p)(b a)(b c), p cabc .(3.19)

In Matlab the inverse of the CDF of a triangular dis-tribution can be computed as:

p(p>1) = NaN;

p(p


21/57

3.7 Exponential distribution 3 PROBABILITY DISTRIBUTIONS

Range: a X bParameters: < a, a < b, a c bE(X) = a+b+c3

Mo = cV(X) = a2+b2+c2abacbc

18

3.6 Log-triangular distribution

The log-triangular distribution is like the triangular dis-tribution except that it is sampled on a logarithmicscale instead of a linear scale. The upper and lowerbounds are still entered in linear space. The PDF of alog-triangular distribution is defined as:

(x) =

2(log(x)log(a))(log(b)log(a))(log(c)log(a))x

, a x c2(log(b)log(x))

(log(b)log(a))(log(b)log(c))x, c < x b

0, x / (a, b)(3.20)

Fig. 10 shows the PDF of a log-triangular distributionwith a minimum value a = 1, a maximum value b = 10and a nominal value c = 4.

Figure 10: PDF of a log-triangular distribution, witha = 1, b = 10 and c = 4.

In Matlab the PDF of a triangular distribution can becomputed as:

y = zeros(size(x));k = find(x>a & xa & x>c & x b.

(3.21)In Matlab the CDF of a log-triangular distributioncan be computed as:

p = zeros(size(x));

k = find(x>=a & xc & xb) = 1;

The inverse of the CDF of a triangular distribution isdefined as:

1(p) =elog(a)+

p(log(b)log(a))(log(c)log(a)), p < cabc

elog(b)(1p)(log(b)log(a))(log(b)log(c)), p cabc .

(3.22)

In Matlab the inverse of the CDF of a triangular dis-

tribution can be computed as:

p(p>1) = NaN;p(p


22/57

3 PROBABILITY DISTRIBUTIONS 3.8 Extreme value (Gumbel) distribution

Fig. 11 shows the PDF of an exponential distributionwith = 1.

Figure 11: PDF of an exponential distribution, with = 1.

In Matlab the PDF of an exponential distribution canbe computed as:

y = exp(-x/alpha)/alpha;

y(x


23/57

3.10 Gamma distribution 3 PROBABILITY DISTRIBUTIONS

p(p1) = NaN;r = mu-sigma*log(-log(p));

Range: X +Parameters: +, 0 < E(X) = + , where =Eulers constant 0.577216Mo =

V(X) = 2

6 2

3.9 Beta distribution

The beta distribution is commonly used to representvariability over a fixed range. Another common useof this distribution is to describe empirical data and

predict random behavior of percentages and fractions[Dec04].

The beta distribution has two parameters and andits PDF is given by:

(x) =x1(1 x)1

B[, ], (3.29)

where B[, ] is the beta function with parameters and , given by

B[, ] =

10

x1(1 x)1dx. (3.30)

Fig. 13 shows the PDF of a beta distribution with = 2and = 2.

Figure 13: PDF of the beta distribution, with = 2and = 2.

In Matlab the PDF of the beta distribution can becomputed as a and b is parameter and respectively:

y = x.^(a-1).*(1-x).^(b-1)/beta(a,b);

Where beta(a,b) is the beta function in Eq. 3.30 whichis built-in in Matlab. The CDF of the beta distribu-tion is defined as:

(x) =

x0 x

1(1 x)1dxB[, ]

(3.31)

In Matlab the CDF of the beta distribution can becomputed by using the incomplete beta function whichis a built-in function in Matlab:

p = betainc(x,a,b);

The formula for the inverse of the CDF of the betadistribution does not exist in a simple closed form. It

is computed numerically. InMatlab

the inverse of theCDF of the beta distribution can be computed as:

b = min(b,100000);

r = a/(a+b);dr = 1;

while any(any(abs(dr)>256*eps*max(r,1)))

dr = (betainc(r,a,b)-p)./pdfbeta(r,a,b);r = r-dr;

r = r+(dr-r)/2.*(r1);

end

The above code has been adapted from codes developed

by Anders Holtsberg [Hol00].

Range: X +Parameters: +, 0 < E(X) = +Mo =

1+1 , > 1 and > 1

V(X) = (+)2(++1)

3.10 Gamma distribution

The Gamma distribution applies to a wide-range of

physical quantities. It is used in meteorological pro-cesses to represent pollutant concentrations and mois-ture quantities. The gamma distribution is also usedto measure the time between the occurrence of eventswhen the event process is not completely random [Dec04].

The gamma distribution is defined by two parameters and , its PDF is given by:

(x) =1

[]

x

1ex/ (3.32)

Fig. 14 shows the PDF of a gamma distribution with

= 3 and = 1.

13


24/57

3 PROBABILITY DISTRIBUTIONS 3.12 Weibull distribution

Figure 14: PDF of the gamma distribution, with = 3and = 1.

In Matlab the PDF of the gamma distribution can becomputed as:

x(x


25/57

4 MODEL EVALUATIONS

The Weibull distribution is defined by a shape param-eter and a scale parameter , its PDF is defined as:

(x) =X1

e( x

)

(3.34)

Fig. 16 shows the PDF of an Weibull distribution with = 1 and = 2.

Figure 16: PDF of the Weibull distribution, with = 1and = 2.

InMatlab

the PDF of the Weibull distribution can becomputed as:

y = alpha*(x/beta).^(alpha-1)/beta.* ...

exp(-((x/beta).^alpha));y(x


26/57

5 SCREENING METHODS 5.1 Morris method

5 Screening methods

When dealing with computationally expensive models,

containing large amounts of uncertain input factors,screening methods can be used to isolate the set of fac-tors that has the strongest effect on the output variabil-ity with very few model evaluations. It is valuable toidentify the most important input factors in an initialphase of the analysis. The number of uncertain inputfactors to examine might be reduced. It is often thecase that the number of important input factors is quitesmall compared to the total number of input factors ofa model.

The most appealing property of the screening methodsis their low computational costs i.e. the required num-

ber of model evaluations. A drawback of this featureis that the sensitivity measure is only qualitative. It isqualitative in the sense that the input factors are rankedin order of importance, but they are not quantified onhow much a given factor is more important than others.

There are various screening techniques described in theliterature. One of the simplest is the one-factor-at-a-time design (OAT). In OAT designs the input factorsare varied in turn and the effect each has on the outputis measured. Normally, the factors that are not variedare fixed at nominal values, which are the values thatbest estimates the factors. A maximum and minimum

value is often used representing the range of likely valuesfor each factor. Usually the nominal value is chosen tobe midway of these extremes.

The OAT designs can be used to compute the localimpact of the input factors on the model outputs. Thismethod is often called local sensitivity analysis. It isusually carried out by computing partial derivatives ofthe output functions with respect to the input variables.The approach often seen in the literature is, instead ofcomputing derivatives, to vary the input factors in asmall interval around the nominal value. The intervalis usually a fixed (e.g. 5%) fraction of the nominal value

and is not related to the uncertainty in the value of thefactors.

In general, the number of model evaluations requiredfor an OAT design is of the order O(k) (often, 2k + 1),k being the number of factors examined.

Elementary OAT designs are implemented in Eikos, withthe possibility of choosing a nominal value and extremesfor each input factor and then to vary each factor in turnto compute local sensitivities (partial derivatives) if theextremes are not too far away from the nominal values.

5.1 Morris method

Morris [Mor91] came up with an experimental plan that

is composed of individually randomized OAT designs.The data analysis is then based on the so called elemen-tary effects, the changes in an output due to changesin a particular input factor in the OAT design. Themethod is global in the sense that it does vary over thewhole range of uncertainty of the input factors. TheMorris method can determine if the effect of the inputfactor xi on the output y is negligible, linear and ad-ditive, nonlinear or involved in interactions with otherinput factors xi

11.

According to Morris the input factor xi may be impor-tant if:

(a) f(xi + ,xi) f(x) is nonzero, then xi af-fects the output.

(b) f(xi + ,xi) f(x) varies as xi varies, thenxi affects the output nonlinearly.

(c) f(xi+,xi)f(x) varies as xi varies, thenxi affects the output with interactions.

is the variation size.

The input factor space is discretized and the possi-ble input factor values will be restricted to be inside aregular k-dimensional p-level grid, where p is the num-ber of levels of the design. The elementary effect ofa given value xi of input factor Xi is defined as a finite-difference derivative approximation

eei(x) = [f(x1, x2, . . . , xi1, xi + ,

xi+1, . . . , xk) f(x)]/, (5.1)

for any xi between 0 and 1 where x {0, 1/(p 1), 2/(p 1), . . . , 1}, and is a predetermined multipleof 1/(p 1). The influence of xi is then evaluated bycomputing several elementary effects at randomly se-

lected values of xi and xi.If all samples of the elementary effect of the ith inputfactor are zero, then xi doesnt have any effect on theoutput y, the sample mean and standard deviation willboth be zero. If all elementary effects have the samevalue, then y is a linear function of xi. The standarddeviation of the elementary effects will then of coursebe zero. For more complex interactions, due to inter-actions between factors and nonlinearity, Morris statesthat if the mean of the elementary effects is relativelylarge and the standard deviation is relatively small, the

11xi is the input factor under consideration; xi is all input factors,except the input factor under consideration.

16


27/57

5.1 Morris method 5 SCREENING METHODS

effects of xi on y is mildly nonlinear. If the opposite,the mean is relatively small and the standard devia-tion is relatively large, then the effect is supposed to be

strongly nonlinear. As a rule of thumb:

(a) a high mean indicates a factor with an im-portant overall influence on the output and

(b) a high standard deviation indicates that ei-ther the factor is interacting with other fac-tors or the factor has nonlinear effects onthe output.

To compute r elementary effects of the k inputs we needto do 2rk model evaluations. With the use of Morrisrandomized OAT design the number of evaluations are

reduced tor(k + 1)

.The design that Morris proposed is based on the con-struction of a (k + 1) k orientation matrix B. Rowsin B represent input vectors x = x(1), x(2), . . . , x(k+1)

that defines a trajectory 12 in the input factor space, forwhich the corresponding experiment provides k elemen-tary effects, one for each input factor. The algorithmof Morris design is

(a) Randomly choose a base value x for x,sampled from the set {0, 1/(p1), . . . , 1}.

(b) One or more of the k elements in x are in-

creased by .

(c) The estimated elementary effect of the ithcomponent ofx(1) is (if changed by )

eei(x(1)) = [f(x

(1)1 , x

(1)2 , . . . , x

(1)i1, x

(1)i + ,

x(1)i+1, . . . , x

(1)k ) f(x(1))]/,

ifx(1) has been increased by , or

eei(x(1)) = [f(x(1)) f(x(1)1 , x

(1)2 , . . . , x

(1)i1,

x(1)i , x

(1)i+1, . . . , x

(1)k )]/,

ifx(1) has been decreased by

(d) Let x(2) be the new vector (x(1)1 , x(1)2 , . . . , x

(1)i1, x

(1)i

, x(1)i+1, . . . , x

(1)k )) defined above. Select a

new vector x(3) such that x(3) differs fromx(2) for only one component j: either x(3)j =

x(2)j + or x

(3)j = x

(2)j , j = i. Estimated

elementary effect of factor j is then

eej(x(2)) =

f(x(3)) f(x(2))

12Trajectory A sequence of points starting from a random basevector in which two consecutive elements differ only for one com-ponent.

if > 0, or

eej(x(2)) =

f(x(2)) f(x(3))

otherwise.

(e) The previous step is then repeated such thata succession ofk+1 input vectors x(1), x(2), . . . , x(k+1)

is produced with two consecutive vectors onlydiffering in one component.

To produce the randomized orientation matrix B con-taining the k + 1 input vectors, Morris proposed:

B

= (Jk+1,1x

+ (/2)((2B Jk+1,k)D

+ Jk+1,k)P

),(5.2)where Jk+1,k is a (k+1)k matrix of ones, B is a (k+1)k sampling matrix which has the property that for everyof its column i = 1, 2, . . . , k, there are two rows of B thatdiffer only in their ith entries. D is a k-dimensionaldiagonal matrix with elements 1 and finally P is akk random permutation matrix, in which each columncontains one element equal to 1 and all the others equalto 0, and no two columns have ones in the same position.

B provides one elementary effect per factor that israndomly selected, r different orientation matrices B

have then to be selected in order to provide a r k-dimensional sample.The main advantage of the Morris design is the rela-tively low computational cost. The design requires onlyabout one model evaluation per computed elementaryeffect.

One drawback with the Morris design is that it onlygives an overall measure of the interactions, indicatingwhether interactions exists, but it does not say whichare the most important. Also it can only be used with aset of orthogonal input factors, i.e. correlations cannotbe induced on the input factors.

In an implementation, there is a need to think about thechoice of the p levels among which each input factor isvaried. In Eikos these levels correspond to quantiles ofthe input factor distributions, if the distributions arenot uniform. For uniform distributions, the levels areobtained by dividing the interval into equidistant parts.

The choice of the sizes of the levels p and realizationsr is also a problem; various experimenters have demon-strated that the choice of p = 4 and r = 10 producesgood results [STCR04].

The Matlab code for the Morris method can be found

in Appendix A.3.

17


28/57

6 SAMPLING-BASED METHODS 6.2 Regression analysis

6 Sampling-based methods

The sampling-based methods are among the most com-

monly used techniques in sensitivity analysis. Theyare computed on the basis of the mapping betweenthe input-output-relationship generated by the MonteCarlo simulation. Sampling-based methods are some-times called global since these methods evaluate theeffect of xi while all other input factors xj , j = i, arevaried as well. All input factors are varied over theirentire range. This in contrast to local perturbationalapproaches where the effect of xi is evaluated when theothers xj , j = i, are kept constant at a nominal value.In the rest of this section various common sampling-based sensitivity analysis methods are described.

6.1 Graphical methods

Providing a means of visualizing the relationships be-tween the output and input factors, graphical methodsplay an important role in SA.

A plot of the points [Xji , Yj ] for j = 1, 2, . . . , N , usuallycalled a scatter plot can reveal nonlinear or other unex-pected relationships between the input factor xi and theoutput y. Scatter plots are undoubtedly the simplestsensitivity analysis technique and is a natural starting

point in the analysis of a complex model. It facilitatesthe understanding of model behavior and the planningof more sophisticated sensitivity analysis. When onlyone or two inputs dominate the outcome, scatter plotsalone often completely reveal the relationships betweenthe model input X and output Y. Using Latin Hyper-cube sampling can be particularly revealing due to thefull stratification over the range of each input variable.

In Matlab the function scatter(X,Y) produces a scat-ter plot of the vectors X and Y.

A tornado graph is another type of plot often used to

present the results of a sensitivity study. It is a simplebar graph where the sensitivity statistics is visualizedvertically in order of descending absolute value. All ofthe global sensitivity measures presented in this sectioncan be presented with a tornado graph.

6.2 Regression analysis

A sensitivity measure of a model can be obtained usinga multiple regression to fit the input data to a theoreti-cal equation that could produce the output data with assmall error as possible. The most common technique of

regression in sensitivity analysis is that of least squares

linear regression. Thus the objective is to fit the inputdata to a linear equation (Y = aX + b) approximatingthe output Y, with the criterion that the sum of the

squared difference between the line and the data pointsin Y is minimized. A linear regression model of theN k input sample X to the output Y takes the form:

Yi = 0 +k

j=1

jXij + i, (6.1)

where j are regression coefficients to be determinedand i is the error due to the approximation, i.e. i =Yi Yi.A measure of the extent to which the regression modelcan match the observed data is called the model coeffi-

cient of determination, R2

, which is defined as:

R2 =

Ni=1 (Yi Y)2Ni=1 (Yi Y)2

, (6.2)

where Yi is the approximated output obtained from theregression model and Yi and Y are the original valuesand their mean respectively. If R2 is close to 1, thenthe regression model is accounting for most of the vari-ability in Y. If on the other side R2 is low, nonlinearbehavior is implicated and a linear approximation istherefore no good. Another method of analysis shouldtherefore be used. In Matlab (y is the original val-

ues and yhat is the approximation from the regressionmodel):

R2 = norm(yhat-mean(y)) 2/norm(y-mean(y))^2;

Alternatively, R2 could be obtained in Matlab by thefollowing code (X is the input factor matrix and y is theoutput vector):

% compute correlation matrixcorr = corrcoef([X y]);

% invert the correlation matrix

corr = inv(corr);

% compute the model coefficient of determinationR2 = 1-1/corr(end,end);

The regression coefficients j , j = 1, . . . , k, measuresthe linear relationship between the input factors andthe output. Their sign indicates whether the output in-creases (positive coefficient) or decreases (negative co-efficient) as the corresponding input factor increases.Since the coefficients are dependent on the units inwhich X and Y are expressed, the normalized form ofthe regression model is used in sensitivity analysis:

Yi

Yi

s=

k

j=1

j sj

s

Xij Xjsj

, (6.3)

18


29/57

6.3 Correlation coefficients 6 SAMPLING-BASED METHODS

where

s = N

i=1

(Yi Y)2

N 1 1/2

, sj = N

i=1

(Xij Xj)2

N 1 1/2

(6.4)In sensitivity analysis, the standardized coefficients j sj/sin Eq. (6.3), called standardized regression coefficients(SRCs), are used as a sensitivity measure.

IfXj are independent, SRCs provide a measure of theimportance, based on the effect of moving each variableaway from its expected value by a fixed fraction of itsstandard deviation while retaining all other variables attheir expected values. Calculating SRCs is equivalentto performing the regression analysis with the input andoutput variables normalized to mean zero and standard

deviation one.In Eikos, SRCs are computed with the least squaresmethod (X is the N k-dimensional input matrix and yis the N-dimensional output vector):

% Add a constant term.X = [ones(N,1) X];

% Find the least squares solution by the use% of Matlab backslash operator.

% b is the vector of regression coefficients.

b = X\y;

% "Standardize" the regression coefficients.

r = b.*std(X)/std(y);

% Remove the constant term.

SRC = r(2:end);

An alternative approach to compute the SRCs in Mat-lab is:

% compute correlation matrix

corr = corrcoef([X y]);% invert the correlation matrix

corr = inv(corr);

for i=1:length(corr)-1% compute standardized regression coefficients

SRC(i,1) = -corr(i,end)/corr(end,end);end

6.3 Correlation coefficients

The Correlation coefficients (CC) usually known as Pear-sons product moment correlation coefficients, providea measure of the strength of the linear relationship be-tween two variables. CC between two N-dimensionalvectors x and y is defined by:

xy = Nk=1 (xk x)(yk y)

Nk=1 (xk x)21/2 Nk=1 (yk y)21/2, (6.5)

where x and y are defined as the mean of x and y re-spectively. CC could be reformulated as:

xy =

cov(x, y)

(x)(y) , (6.6)

where cov(x, y) is the covariance between the data setsx and y and (x) and (y) are the sampled standarddeviations.

The correlation coefficient is then the normalized co-variance between the two data sets and (as SRC) pro-duces a unitless index between -1 and +1. CC is equalin absolute value to the square root of the model co-efficient of determination R2 associated with the linearregression.

In Matlab there is a function corrcoef(x,y) thatcomputes the correlation coefficient, but since it returnsthe whole correlation matrix an alternative code is givenby:

x_m = x-mean(x); %Remove mean of xy_m = y-mean(y); %Remove mean of y

SSxx = x_m*x_m;

SSyy = y_m*y_m;SSxy = x_m*y_m;

rho = SSxy/sqrt(SSxx*SSyy);

CC only measures the linear relationship between twovariables without considering the effect that other possi-ble variables might have. So when more than one inputfactor is under consideration, as it usually is, partialcorrelation coefficients (PCCs) can be used instead toprovide a measure of the linear relationships betweentwo variables when all linear effects of other variablesare removed. PCC between an individual variable Xiand Y can be obtained from the use of a sequence ofregression models. Begin by constructing the followingregression models:

Xi = c0 +

kj=1,j=i

cjXj , Yi = b0 +

kj=1,j=i

bjXj . (6.7)

Then PCC is defined by the CC of Xi Xi and Y Y.PCC can also be written in terms of simple correlationcoefficients. Denote the PCC of X1 and Y holding Z =X2, . . . , Xk fixed, X1Y|X2,...,Xk . Then

X1Y|Z =X1Y X1ZY Z(1 2X1Z)(1 2Y Z)

. (6.8)

The following Matlab code computes PCC for everyinput factor in the input matrix:

19


30/57

6 SAMPLING-BASED METHODS 6.5 Two-sample tests


corr = corrcoef([X y]);for i=1:length(corr)-1

% 2 by 2 correlation matrix of Xi and y.

S11 = [1 corr(i,end);corr(i,end) 1];% k-1 by k-1 correlation matrix of X~i.S22 = corr;

% Remove effects of Xi and y.

S22(:,i)=[];S22(i,:)=[];S22(:,end)=[];S22(end,:)=[];% 2 by k matrix of all pairwise

% correlations between (Xi,y) and X~i.

S12 = [corr(:,i); corr(:,end)];

% Remove effects of Xi and y.S12(:,i)=[];S12(:,end)=[];

% Form the condition correlation matrix of (Xi,y)

% given X~i.S112 = S11-S12*inv(S22)*S12;

% PCC is the off-diagonal of S112 divided by the

% geometric mean of the two diagonals.PCC(i,1) = S112(1,2)/sqrt(S112(1,1)*S112(2,2));

end

An alternative approach to compute the PCCs in Mat-lab is:


corr = corrcoef([X y]);% invert the correlation matrix

corr = inv(corr);

for i=1:length(corr)-1

% compute partial correlation coefficientPCC(i,1) = -corr(i,end)/sqrt(corr(i,i)*corr(end,end));

end

PCC characterizes the strength of the linear relation-ship between two variables after a correction has beenmade for the linear effects of the other variables in theanalysis. SRC on the other hand characterizes the ef-fect on the output variable that results from perturbingan input variable by a fixed fraction of its standard de-viation. Thus, PCC and SRC provide related, but notidentical, measures of the variable importance. Wheninput factors are uncorrelated, results from PCC andSRC are identical.

6.4 Rank transformations

Since the above methods are based on the assumptionof linear relationships between the input-output factors,they will perform poorly if the relationships are non-linear. Rank transformation of the data can be usedto transform a nonlinear but monotonic13 relationshipto a linear relationship. When using rank transforma-tion the data is replaced with their corresponding ranks.Ranks are defined by assigning 1 to the smallest value,2 to the second smallest and so-on, until the largestvalue has been assigned the rank N. If there are ties

13A function is monotone if its output only increases or decreaseswith the inputs, i.e. it preserves the order.

in the data set, then the average rank is assigned tothem. The usual regression and correlation proceduresare then performed on the ranks instead of the original

data values. Standardized rank regression coefficients(SRRC) are SRC calculated on ranks, spearman rankcorrelation coefficient (RCC) are corresponding CC cal-culated on ranks and partial rank correlation coefficients(PRCC) are PCC calculated on ranks.

The model coefficient of determination R2 in Eq. (6.2)can be computed with the ranked data, and measuresthen how well the model matches the ranked data.

Rank transformed statistics are more robust, and pro-vide a useful solution in the presence of long tailedinput-output distributions. A rank transformed modelis not only more linear, it is also more additive. Thus

the relative weight of the first-order terms is increasedon the expense of higher-order terms and interactions.

The following Matlab code is used in Eikos to acquireranks r of a vector x:

[foo,i ] = sort(x);

[foo,r1] = sort(i);[foo,i ] = sort(flipud(x));

[foo,r2] = sort(i);

r = (r1+flipud(r2))/2;

6.5 Two-sample tests

Two-sample tests were originally designed to check thehypothesis that two different samples belong to the samepopulation [Con99]. In sensitivity analysis, two-sampletests can be used together with Monte Carlo filtering.In Monte Carlo filtering you partition the input samplesinto two sub-samples according to restrictions on theoutput distribution. The two-sample test is then per-formed on the two empirical cumulative distributionsof the sub-samples, and measures if their difference issignificant or not, i.e. if the two distributions are dif-ferent, then it can be said that the factor influences the

output.To obtain in Matlab the empirical cumulative distri-butions F1(x) and F2(x) out of two sub-samples X1 andX2 the following code can be used (X1, X2 are the twoinput samples and F1x, F2x are their correspondingempirical CDFs):

% Set up an EDGES vector.

edges = [-inf;sort([X1;X2]);inf];

% Count number of values falling

% between elements in EDGES vector.

counts1 = histc(X1,edges)counts2 = histc(X2,edges)

20


31/57

7 VARIANCE-BASED METHODS

% Normalized cumulative sum of the values.

sumCounts1 = cumsum(counts1)./sum(counts1);sumCounts2 = cumsum(counts2)./sum(counts2);

% Remove last element.F1x = sumCounts1(1:end-1);F2x = sumCounts2(1:end-1);

The Smirnov test is defined as the greatest vertical dis-tance between the two empirical cumulative distribu-tions:

smirnov(Y, Xj) = M axXj |F1(Xi) F2(Xi)|, (6.9)where F1 and F2 are the cumulative distributions of Xjestimated on the two sub-samples and the difference isestimated at all the xij points, i = 1, . . . , N . In Matlabthe Smirnov statistics is computed as:

% Smirnov testsmirnov = max(abs(F1x - F2x));

A test similar to the Smirnov test, but with slightlymore calculations, is the Cramer-Von Mises test, whichis given by:

cramer(Y, Xj) =N1N2

(N1 + N2)2

Xj

(F1(Xi) F2(Xi))2,

(6.10)where the squared difference in the summation is com-puted at each xij , i.e. the statistic depends upon thetotal area between the two distributions. In Matlab

(n1, n2 are respectively the number of samples in thetwo sub-samples X1 and X2):

% Cramer-Von Mises test

cramer = n1*n2/(n1+n2) 2*sum((F1x - F2x).^2);

6.6 Discussion

According to Saltelli et al. [SM90],[SH91] the estima-tors PRCC and SRRC appears to be, in general, themost robust and reliable methods of the ones describedin this section, followed by Spearmans RCC and the

Smirnov test. Nevertheless, all above tests have beenincluded in Eikos since they could be of use in differentcontexts.

The rankings of SRCC and PRCC are usually identical,so they could be considered redundant. Differences oc-cur only when there are significant correlations amongstthe input factors.

Predictions using SRRC are strongly correlated to thoseof Spearmans RCC and the Smirnov test. Therefore,the value of the model coefficient of determination R2

plays a crucial role, as it indicates the degree of relia-bility of the regressed model in SRRCs as well as the

other techniques.

7 Variance-based methods

The main idea of the variance-based methods is to quan-

tify the amount of variance that each input factor Xicontributes with on the unconditional variance of theoutput V(Y).

We are considering the model function: Y = f(X),where Y is the output and X = (X1, X2, . . . , Xk) arek independent input factors, each one varying over itsown probability density function.

We want to rank the input factors according to theamount of variance that would disappear, if we knewthe true value xi of a given input factor Xi. V(Y|Xi =xi ) is the conditional variance of Y given Xi = x

i and

is obtained by taking the variance over all factors butXi.

In most cases we do not know the true value xi for eachXi. Therefore, the average of this conditional variancefor all possible values xi ofXi is used, i.e. E[V(Y|Xi)] isthe expectation value over the whole variation intervalof the input Xi. Having the unconditional variance ofthe output V(Y), the above average and by using thefollowing property of the variance:

V(Y) = V(E[Y|Xi]) + E[V(Y|Xi)], (7.1)

we obtain the variance of the conditional expectationVi = V(E[Y|Xi]). This measure is sometimes calledmain effect and is used as an indicator of the impor-tance of Xi on the variance of Y, i.e. the sensitivity ofY to Xi. Normalizing the main effect Vi by the uncon-ditional variance of the output we obtain:

Si =V(E[Y|Xi])

V(Y). (7.2)

The ratio Si was named first order sensitivity index bySobol [Sob93]. Various other names for this ratio canbe found in the literature: importance measure, cor-relation ratio and first order effect.

The first order sensitivity index measures only the maineffect contribution of each input parameter on the out-put variance. It doesnt take into account the interac-tions between input factors. Two factors are said tointeract if their total effect on the output isnt the sumof their first order effects. The effect of the interac-tion between two orthogonal factors Xi and Xj on theoutput Y would in terms of conditional variances be:

Vij = V(E[Y|Xi, Xj ]) V(E[Y|Xi]) V(E[Y|Xi]). (7.3)

V(E[Y

|Xi, Xj ]) describes the joint effect of the pair (Xi, Xj)

on Y. This effect is known as the second-order effect.

21


32/57

7 VARIANCE-BASED METHODS 7.1 Sobol indices

Higher-order effects can be computed in a similar fash-ion, i.e. the variance of the third-order effect betweenthe three orthogonal factors Xi, Xj and Xl would be:

Vijl = V(E[Y|Xi, Xj , Xl]) Vij Vil Vjl Vi Vj Vl. (7.4)

A model without interactions is said to be additive, forexample a linear one. The first order indices sums upto one in an additive model with orthogonal inputs.For additive models, the first order indices coincideswith what can be obtained with regression methods(described in section 6.2). For non-additive models wewould like to know information from all interactionsas well as the first order effect. For non-linear mod-els the sum of all first order indices can be very low.The sum of all the order effects that a factor accountsfor is called the total effect [HS96]. So for an inputXi, the total sensitivity index STi is defined as the sumof all indices relating to Xi (first and higher order).Having a model with three input factors (k = 3), thetotal sensitivity index for input factor X1 would thenbe: ST1 = S1 + S12 + S13 + S123. Computing all order-effects to obtain the total effect by brute force is notadvisable when the number of input factors k increases,since the number of terms needed to be evaluated areas many as 2k 1.In the following subsections, four methods to obtainthese sensitivity indices are described. These are theSobol indices, Jansens Winding Stairs technique, Fou-rier Amplitude Sensitivity Test (FAST) and the Ex-tended Fourier Amplitude Sensitivity Test (EFAST).All four methods except FAST can obtain both the firstand total order effect in an efficient way. The standardFAST method only computes the first order effect. TheSobol method can achieve all order effects, but the needof model evaluations increases too much to be practical.

The input factor space k is hereafter assumed to bethe k-dimensional unit hypercube:

k = (X

|0

Xi

1; i = 1, . . . , k). (7.5)

This doesnt give any loss of generality since the factorscan be deterministically transformed from the uniformdistributions to a general PDF as described in section3. The input factors are also assumed to be orthogonalagainst each other, thus no correlation structure can beinduced on the input factors.

The expected value of the output E(Y) can be evaluatedby the k-dimensional integral:

E(Y) =

k

f(X)p(X)dX =

k

f(X)dX, (7.6)

where p(X) is the joint probability density function as-

sumed to be uniform for each input factor.

7.1 Sobol indices

Sobol [Sob93] introduced the first order sensitivity in-

dex by decomposing the model function f into sum-mands of increasing dimensionality:

f(X1, . . . , Xk) = f0 +k

i=1

fi(Xi) +

k1=1

kj=i+1

fij(Xi, Xj) + . . . +

f1...k(X1, . . . , Xk). (7.7)

This representation of the model function f(X), whichis a decomposition into summands of increasing dimen-

sionality, holds if f0 is a constant (f0 is the expectationof the output, i.e. E[Y]) and the integrals of every sum-mand over any of its own variables are zero, i.e.:

10

fis (Xi1 , . . . , Xis )dXik = 0, if 1 k s (7.8)

As a consequence of this, all the summands are mutu-ally orthogonal.

The total variance V(Y) is defined as:

V(Y) = k f2(X)dX f20 (7.9)

and the partial variances are computed from each of theterms in Eq. (7.7)

Vi1...is =

10

10

f2i1...is (Xi1 , . . . , Xis )dXi1 . . . d Xis

(7.10)where 1 i1 < < is k and s = 1, . . . , k.The sensitivity indices are then obtained by:

Si1...is =Vi1...isV(Y)

, for 1 i1 < < is k. (7.11)

The integrals in Eq. (7.9) and Eq. (7.10) can be com-puted with Monte Carlo methods. For a given samplesize N the Monte Carlo estimate of f0 is:

f0 =1

N

Nm=1

f(Xm) (7.12)

where Xm is a sampled point in the input space k. InMatlab (sample matrix X is of size Nk, function2evaluateis the model function under consideration):

Y = function2evaluate(X);

f0 = mean(Y);

22


33/57

7.1 Sobol indices 7 VARIANCE-BASED METHODS

The Monte Carlo estimate of the output variance V(Y)is:

V(Y) =1

N

N

m=1

f2(Xm)

f20 (7.13)

In Matlab:

% second parameter makes var() normalize

% by N instead of N-1V = var(Y,1);

Main effect of input factor Xi is estimated as:

Vi =1

N

Nm=1

f(X(M1)im , X

(M1)im )f(X

(M2)im , X

(M1)im ) f20 .

(7.14)

Here we need to use two sampling matrices,X(M1)

andX(M2), both of size N k. X(M1)im means the full set ofsamples from X(M1) except the ith one. Matrix X(M1)

is usually called the data base matrix while X(M2) iscalled the resampling matrix [CST00].

In Matlab this is implemented as (XM1 and XM1 arethe two different sampling matrices both of size N k):

YXM1 = function2evaluate(XM1);

for i=1:k

XM2_i = XM2;XM2_i(:,i) = XM1(:,i);

YXM2_i = function2evaluate(XM2_i);

Vi(i) = 1/n*sum(YXM1.*YXM2_i)-f02;end

For formulas of computing partial variances of higherorder than in Eq. (7.14) I refer to [HS96], a separate MCintegral is required to compute any effect. Counting theMC integral needed for computing f0, a total of 2

k MCintegrals are therefore needed for a full characterizationof the system.

In 1996, Homma and Saltelli proposed an extensionfor direct evaluation of the total sensitivity index STi[HS96]. Now STi can be evaluated with just one Monte

Carlo integral instead of computing the 2k

integrals.They suggested dividing the set of input factors intotwo subsets, one containing the given variable Xi andthe other containing its complementary set Xci. Thedecomposition of f(X) would then become:

f(X) = f0 + fi(Xi) + fci(Xci) + fi,ci(Xi,Xci). (7.15)

The total output variance V(Y) would be:

V(Y) = Vi + Vci + Vi,ci (7.16)

and the total effect sensitivity index STi is:

STi = Si + Si,ci = 1 Sci. (7.17)

Thus, to obtain the total sensitivity index for variableXi we only need to obtain its complementary indexSci = Vci/V(Y). Homma and Saltelli [HS96] shows that

Vci can be estimated with just one Monte Carlo integral:

Vci =1

N

Nm=1

f(X(M1)im , X

(M1)im )f(X

(M1)im , X

(M2)im ) f20 .

(7.18)In Matlab:

for i=1:kXM1_i = XM1;

XM1_i(:,i) = XM2(:,i);

YXM1_i = function2evaluate(XM1_i);

Vci(i)= 1/n*sum(YXM1.*YXM1_i)-f02;end

So to compute the first and total order sensitivity in-dices we take Si = Vi/V(Y) and STi = 1 Vci/V(Y)respectively.

In Matlab:

Si = Vi/V;Sti = 1-Vci/V;

To obtain both first and total order sensitivity indicesfor k factors and N samples, with Sobol method, weneed to make N(2k + 1) model evaluations (this if we

use eitherX(M1) orX(M2) to compute the unconditionalvariance V(Y), described in Eq. (7.13)).

When the mean value (f0) is large, a loss of accuracyis induced in the computation of the variances by theMonte Carlo methods presented above [Sob01]. If wedefine c0 as an approximation to the mean value f0, thenew model function f(x) c0 could be used instead ofthe original model function (f(x)) in the analysis. Forthe new model function, the constant term will thenbe very small, and the loss of accuracy, due to a largemean

Eikos a Simulation Toolbox for Sensitivity Analysis

Documents

Transcript of Eikos a Simulation Toolbox for Sensitivity Analysis