OptiRisk Systems: White Paper Series Domain: Finance Reference Number: OPT 010

SCENARIO GENERATION FOR FINANCIAL MODELLING:

DESIRABLE PROPERTIES AND A CASE STUDY

Last Update 30 March 2009

Scenario generation for nancial modelling:Desirable properties and a case study

Leela Mitra ∗ (leelavati.mitra@brunel.ac.uk)Gautam Mitra ∗ (gautam.mitra@brunel.ac.uk)Diana Roman ∗ (diana.roman@brunel.ac.uk)

March 30, 2009

∗CARISMA: The Centre for the Analysis of Risk and Optimisation Modelling Applications, School of Information Systems, Computing andMathematics, Brunel University, UK and OptiRisk Systems, Uxbridge

1

Contents1 Introduction to the modelling framework 4

2 Desirable properties of scenario generators 7

3 Scenario generation methods for asset prices 8

4 Ex post evaluation 13

5 Assessing the quality of scenario generation methods 15

6 A case study 16

7 Conclusions 18

8 Acknowledgements 18

2

AbstractInvestment decisions are made ex ante, that is based on parameters that are not known at the time of decision making.Scenario generators are used not only in the models for (optimum) decision making under uncertainty, they are alsoused for evaluation of decisions through simulation modelling. In this paper, we review those properties of scenariogenerators which are regarded as desirable; these are not sufcient to guarantee the goodness of a scenario generator.We also review classical models for scenario generation of asset prices. In particular we consider some recentlyreported methods which have been proposed for distributions with `heavy tails'.

3

1 Introduction to the modelling frameworkThe acceptance and success of linear and integer programming has in turn focused attention on stochastic program-ming (SP) as the modelling paradigm for decision making under uncertainty. Although SP has been studied extensivelysince the 1970s most of the research has been directed towards modelling applications (Wallace and Ziemba 2005),solution algorithms (Van Slyke and Wets 1969) or in the realisation of software for SP (see series of SP trienial confer-ences and COSP). Yet SP is an extremely versatile and powerful modelling paradigm as it brings together the decisionmodelling capability of optimisation and the descriptive modelling concepts of simulation (see di Dominica, Lucas,Mitra and Valente 2007). Indeed the most far reaching applications of SP are to be found in nance, where SP isnot only used as a tool for making investment decisions, such decisions are then evaluated using backtesting and outof sample simulation studies. An essential component of these two modelling perspectives of SP is a scenario set ofuncertain parameter values obtained by scenario generation. Unfortunately, scenario generators for SP have been lessextensively studied in comparison with other aspects. This paper is concerned with the role of scenario generation(SG) within SP. In particular, we investigate and illustrate some desirable properties of SG.

In SP the concept of optimum decisions is revisited to consider parameter uncertainty. A decision problem whichinvolves random (stochastic) parameters is stated as

max g0(x; ξ )such that x ∈ X

gl(x; ξ )≤ 0, l = 1 . . .K(1)

where ξ which is a vector of random variables describing the stochastic parameters. For a particular decisionvector x ∈Rn, the functions gi(x, ·) : Ω→R for l = 0, . . . ,K are random variables. The distribution of ξ is assumed tobe known at the time the decision is made though the actual realisation is not known. In contrast to the deterministicperspective it can be seen that the problem is not clearly specied, since the future outcome of ξ is not known whenthe decision x is made. The related ex ante decision problem for choosing x is specied through alternative modelformulations. There are a number of formal SP decision models which have been postulated. These include singlestage SP models, two stage SP with recourse models, multi stage SP with recourse models, chance constrained SPmodels and integrated chance constrained SP models (Kall and Wallace 1994 and Birge and Louveaux 1997). Manypractical applications in nance, in particular asset allocation models are often formulated in a single stage setting.Three classical models applied in economics and nance for portfolio selection include mean-risk models, expectedutility maximisation and stochastic dominance.

As an example, let us consider the case of asset allocation: suppose that there are n assets into which one caninvest. The decision to take is how to divide the capital between these assets such that portfolio returns meet statedobjectives. Typically the objective is to maximise the portfolio return over a specied time horizon T . The return ofasset j after time T is denoted with R j (and it is a random variable), j = 1 . . .n. The decision variables (also calledportfolio weights) are denoted with x1, . . . ,xn, i.e. x j is the fraction of capital invested in asset j. Let X ⊂ Rn denotethe set of the feasible portfolios. Such a feasible set is often simply dened by the requirement that the weights mustsum to 1 and short selling is not allowed: X = (x1, . . . ,xn)|

n∑j=1

x j = 1,x j ≥ 0,∀ j ∈ 1, . . . ,n. In a more general

case, it is only assumed that X is a bounded convex polyhedron.

It is known that the return of the portfolio x = (x1, . . . ,xn), denoted by Rx, is given by x1R1 + . . .+ xnRn.Thus, the problem that we face is:

max R1x1 + . . .+Rnxnsuch that x ∈ X (2)

However, the parameters R1, . . . ,Rn are random variables, described by distributions. The portfolio return Rx =x1R1 + . . . + xnRn is a random variable; how can we maximise" a random variable? Unless for trivial cases, it is

4

difcult to decide whether a random return is "better" than another.

By specifying a model of choice (criteria for preferring one random return over another) the problem (2) can betransformed into an optimisation model. A popular approach is the so called mean-risk" model, which assumes thata random return is preferred over another if it has a larger expected value (mean) and a smaller "risk" value. Risk canbe dened in several ways (for a review, see Roman and Mitra 2009). Problem (2) becomes a two-objective problem,where the expected value is maximised while the risk is minimised. This is transformed into a single objective problemby using a trade-off coefcient τ ≥ 0 between mean and risk:

max E(R1x1 + . . .+Rnxn)− τρ(R1x1 + . . .+Rnxn)such that x ∈ X (3)

where E denotes the expected value and ρ the risk value.

Dynamic programming, stochastic control and robust optimisation models are alternative approaches to decisionmaking under risk (See Dupacova and Sladsky (2002) and di Domenica et al. (2007) and reference therein). Decisionmaking under risk requires

1. Models for decision making under risk These allow us to state a preference among random variables (repre-senting results of decisions); this should lead to a clearly dened optimisation model. These models are normallyformulated, given a multivariate distribution for the uncertain parameters. For example, for the asset allocationproblem how do we dene preferences between alternative portfolios' random returns.

2. Representation of parameter uncertainty: Scenario generation We need to determine distributional repre-sentations for the uncertain parameters. For example, for the asset allocation problem what is the nature ofthe candidate assets' price dynamics and uncertainty? In particular how should we describe the multivariatedistribution of the assets' random returns, R1, . . . ,Rn? The models used to characterise these distributions areknown as descriptive models.

The models for decision making under risk are heavily dependent on the representations of parameter uncertaintywhich are used. Hence 2. is an important consideration.

When the random parameters are described by continuous distributions there are only a few simple cases whichyield analytic solutions, for example, the classical Newsboy Problem. Most applied SP models are formulated withdiscrete distributions. In those cases where the stochastic parameters are described by continuous distributions, wenormally create a discrete approximation to the continuous distribution, such that the optimisation problem can besolved numerically. Scenario generation (SG) is the process of creating a nite collection of scenarios which describe(in many cases approximately) the distribution of (relevant) random parameters of the SP optimisation models, whichare used for ex ante (portfolio) decisions. Scenario sets are also applied in ex post (simulation) evaluation and riskquantication of portfolios; this aspect if further discussed in section 4.

A scenario is a single possible realisation of all uncertain parameters. Denote by ξ a vector of random variableswhich represent the stochastic model parameters. Scenarios describe the possible values of ξ at a future point intime. This can be generalised to multiple future time points by considering (discrete time) stochastic processes. Letξ t∈1,...,T be a stochastic process which species the evolution of the random parameters over time.

In scenario generation we construct scenarios that represent plausible outcomes, both pessimistic and optimistic.Each scenario is weighted by its probability of occurance. Forecasting is a related but a different problem. Scenariosare not forecasts; whereas a point forecast is a prediction of the (most likely) value for a random variable (or vector),a scenario set can be viewed as a density forecast.

5

Scenario generation involves

• identication of plausible events, that is possible future values of ξ ω for ξ , where ω ∈ 1, . . . ,S;or more generally identication of plausible evolution of the random vector over time (sample) paths ξ ω

t∈1,...,T =ξ ω

1 , . . . ,ξ ωT ∀ ω ∈ 1, . . . ,S ∀ t ∈ 1, . . . ,T.

• the assignment of probabilities to these events, P( ξ = ξ ω) = pω where ∑Sω=1 pω = 1

2 2k 2 2k

T

...... ...... ...... ...... ...... ...... ...... ............

...... ...... ..........................................

... ...

... ...... ... ... ...

2

3 3 3

4

Figure 1: Scenario tree for multiple time points

To summarise, there are two main issues of interest when making choices under uncertainty; the model of choiceand the scenario generation.

Remark 1 There is a third main issue in decision making under uncertainty: timing. Decisions are made at the presenttime, in order to obtain a good outcome at a specied future time point. These decisions are normally re-balanced,i.e. updated according to new information which arrives as time unfolds. There is also the case when only partialdecisions have to be taken at the present time (rst stage decisions) and the rest of the decisions (recourse actions)taken after (part of) the uncertainly had been revealed. These are multi-stage stochastic programming problems. Inthis paper, we only treat the case of single stage problems. For scenario generation in a multi-stage perspective, seedi Domenica et al. (2007).

The rest of the paper is structured as follows. In Section 2 we discuss some desirable properties of scenariogenerators. Testing the quality of a scenario generator is a difcult and subjective task. There are however conditionsthat can be tested and should be regarded as necessary (but not sufcient) for a scenario generator. In section 3

6

we review some of the popular scenario generation methods for asset prices, together with some newly proposedapproaches. In section 4 we consider the role of scenario generators in ex post evaluation of model decisions. Insection 5 we present a short case study with a numerical example. The case study draws upon a recently proposed SGmethod based on mixture distributions between GARCH and copula. This is used in a mean-CVaR optimisation model.The properties discussed in Section 2 are evaluated in this framework. In section 6 we discuss our investigations andpresent the conclusions.

2 Desirable properties of scenario generatorsFor nancial decision problems we require (forward looking) parameter distributions which account for uncertainty inthe economy and nancial markets. We introduce three desirable properties for scenario generation methods; Zenios(2007) introduces the properties of correctness and consistency. Kaut and Wallace (2007) outline that of stability.

1. Correctness: Naturally we wish to use scenario sets which are correct representations of assets' randomreturns. However, we do not know the correct distributions. Different descriptive models give alternativerepresentations of asset price dynamics. We should choose a model which derives from prevalent theory andwhich captures those aspects of asset price dynamics which we believe are important, that is, the model wepropose is correct. For example, if we believe return volatility varies over time, but is persistent, we mightuse a GARCH model.

2. Consistency: When we consider multiple related random variables, the values of these, under any particularscenario, should be consistent with each other. The generation of scenarios for multiple credit risky bondsis a particular example. All bonds' prices are driven by a few common sources of uncertainty (risk factors);short term yields, long term yields, credit rating migrations. Bonds of different maturities, cash ows (couponpayments) and credit ratings have differing prices but these are related by particular relationships which areexpected to hold under nance theory (no-arbitrage requirements for example).

3. Stability: Stability for a scenario generation method is considered in respect of a particular decision model.The combined use of a scenario generator and decision model should lead to good decisions. A gooddecision should at least be a stable decision, that is, the optimal solutions for different scenario sets do not varysignicantly.

Zenios (2007) also notes that scenarios should be acceptable to the problem owner (or decision maker) and ifnecessary they should be modied to gain acceptance. This can be extended to the model being defendable to a peergroup of specialists within the relevant domain. We can view this as an expansion of the concept of correctness. Themodel can be adapted to a more correct model in the decision maker's view.

It is important to note that the scenario generator should be chosen with consideration of the stochastic programfor which it is used. As an example, if the problem is to minimise the loss that could be incurred under extremelyunfavourable events, then the scenario generators should account for extreme events (e.g. copula-based). The condi-tions of correctness and consistency are some what subjective conditions. However the "stability" conditions can betested and are usually regarded as necessary conditions for a scenario generators.

7

3 Scenario generation methods for asset pricesOur perspective of scenario generation methods in nance is shown in Figure 2; a comparable view is given in Zenios(2007). The simplest way to determine scenarios is to use historic observations as scenarios or to bootstrap historicobservations. Other approaches, which are model-based methods, describe the asset price dynamics. These includestatistical models, continuous time models and discrete time models. These descriptive models often develop fromtheories of economics and nance. We rst calibrate these models using historical data. Given calibrated models wesimulate samples or sample paths to determine (large) discrete scenario sets which serve as approximations to thecorrect unknown parameters' distributions.

The classes of models shown in Figure 2 in some sense overlap with each other. Statistical analysis is used inmany discrete and continuous time models. For example, some factor models are composed of statistical factors (onesuch method uses principal component analysis). Many discrete time models in their limit lead to continuous timerepresentations. These are approximations of their continuous counterparts. For example, the random walk leads toGeometric Brownian motion as the time step becomes of innitesimal length (See for example Cox-Ross-Rubenstein1979).

HISTORIC DATA AND BOOTSTRAP SCENARIOS

The simplest approach to scenario generation is to use historic data (observations) of the relevant multivariate ran-dom variables as scenarios. Then a scenario is composed of observations of the relevant assets' returns at a historicdate. However, the scenario set produced this way is (relatively) small being limited by the amount of historic dataavailable. Another popular approach is bootstrapping this involves sampling with replacement from all relevanthistorical observations (Efron and Tibishirani 1993 and Vose 2002). This approach increases the number of scenar-ios. Both methods give scenarios which are correct and consistent with the economy and nancial markets sincethese scenarios actually occurred. It is assumed the structure and conditions of the market have not changed sincethese observations were made, that is, we have considered a suitable period of historic data. However, only observedevents are plausible under these methods; no further information is added to scenarios. These approaches do not giveany understanding of the underlying structure of the markets, for example, the relationships between macroeconomicvariables and asset returns. If the decision maker had a view on the economy or markets, for example the belief thatination rates would change in a given fashion, the asset returns could not be analysed with this information. Sincebootstrapping is a pure sampling approach and is not model based, expert intervention is limited.

MODELS DERIVED FROM ECONOMICS AND FINANCE LITERATURE

There are many existing and widely applied models for asset prices which have emerged from the well establishedand extensive literature of economics and nance. It is natural to turn to these in the context of scenario generationsince it is desirable to use a model which conforms to prevalent theory. However there are a few candidate modelsfor any particular asset class. Each descriptive model captures the asset price dynamics differently, hence each has itsown strengths and weaknesses. Scenarios should be derived using the model the decision maker believes to be cor-rect. In particular the chosen model should include those aspects of asset price uncertainty that the decision makeris concerned about. The selection of a particular model implies a subjective judgement by the decision maker on thenature of the distribution. In the model calibration historical data is used to determine plausible (realistic) values ofthe parameters. Scenarios which are subsequently produced reect historical events (data), furthermore they accountfor events which were not observed but are plausible.

Typical theories of nance and economics from which SG models develop include equilibrium approaches whichlead to models such as the Capital Asset Pricing (CAPM) model (Sharpe 1964 Lintner 1965 Luenberger 1998). Thisassumes that the economy and asset prices within it tend to an equilibrium level. The assumption of arbitrage-freemarkets leads to derivative pricing models based on risk neutral measures (Harrison and Kreps 1979 Harrison andPliska 1981). Within interest rate theory this assumption of no-arbitrage leads to the Heath-Jarrow-Morton condition

8

Historical Data Theory of economy

and

financial markets

Discrete

time models

Continuous

time models

Sampling

Scenario

Sets

“Bootstrapping”

Sampling with

replacement

Other models

Discrete

time models

Continuous

time models

Statistical models

Figure 2: Alternative scenario generation methodologies for asset return distributions

9

(Heath, Jarrow and Morton 1992) which restricts the co-movements of interest rates of differing maturities.

Models based on established economic theory result in the generation of consistent scenarios. The generationof scenarios for multiple bonds is a particular example discussed above. Factor models are the most commonly usedmodels for equity prices and are based on the Arbitrage Pricing Theory (APT) (Ross 1976) which assumes all equities'prices are driven by a few common factors. This results in consistent prices for multiple equities.

It is natural to take a view on particular aspects of an economic model. These approaches incorporate the domainexpert's (or problem owner's) view and so they lend themselves well to (expert) intervention.

Many continuous time models which are described by stochastic differential equations (SDEs) are applied inscenario generation. These models are sometimes approximated using discrete time lattices and the correspondingapproximation errors are unavoidable. It is desirable that the discretisation approximates the continuous time modelwell if we believe the continuous time model is the correct representation of returns.

STATISTICAL APPROACHES

These approaches include time series models for univariate random variables such as AR, MA, ARMA andARCH/GARCH models. Many of these have been extended to a multivariate setting and further explain the inter-action and co-movements of a group of timeseries variables. A few examples are vector autoregressive (VAR) models,VECM and multivariate GARCH models. Volosov, Mitra, Spagnolo and Lucas (2005) use a VECM model to generatescenarios for exchange rates which form the input to a stochastic decision model. These multivariate models allowjoint modelling of economic factors and asset returns. Such models when calibrated to historical data capture theobserved auto and cross correlations between variables. However, they may not be consistent with stronger economicrequirements, for example, absence of arbitrage. Any causal relationships between assets and other economic vari-ables are not captured in purely statistical models.

Econometric models combine economic theory with statistics. These models often have a cascade (hierachy) struc-ture and relate macro economic variables and asset returns through vector autoregressive moving average (VARMA)time series. The Wilkie (1995) model is such an approach, likewise the Towers Perrins model which was developed byMulvey (1996). These approaches are widely applied for asset liability studies. Analysis of the relationship betweenliabilities and macro economic variables (for example between ination and wages for pension fund ALM) allows thecreation of consistent scenarios.

The moment-matching method (Hoyland, Kaut and Wallace 2003) produces scenario sets consistent with user-specied values, which are the rst four marginal moments of the multiple random variables under consideration andtheir correlations. A heuristic algorithm is used.

EXAMPLE SCENARIO GENERATION METHODS

We describe below particular descriptive models which can be applied in scenario generation.

Geometric Brownian motion (GBM) is a well established and widely used descriptive model for an equity'sreturns. In particular, it was used in the Black-Scholes model for option pricing and underlies many of the subsequentoption pricing models. The model assumes that a stock's price evolution can be described using continuous timestochastic processes that are dependent on the "percentage drift" µ (expected rate of return), the volatility σ and a ran-dom element Wt , described by a Wiener Process (see for example Ross 2002). Obviously, stock prices are stochasticprocesses in discrete time; however, this approximation is accepted by practitioners for short to medium time periods.

10

With GBM, the price of a stock St , (t ≥ 0) follows the stochastic differential equation:

dSt = µStdt +σStdWt (4)

where µ and σ are constants (usually estimated from historical data).

Assuming an initial price S0, equation (4) has the analytical solution:

St = S0exp[(µ− σ 2

2 )t +σWt ] (5)

It is well known that a Wiener process Wt can be simulated as√

tZ, where Z is N(0,1) distributed.

Thus, generating scenarios for a future price St involves estimating µ and σ (e.g. from historical data), generatingsamples from N(0,1) and replacing the corresponding values in (5).

In case of multiple stocks the dependence between asset prices has to be taken into account. With GBM, thisdependence is characterised by the covariance matrix of stocks' returns (denoted by C), that has also to be estimated(e.g. from historical data).

A vector of M asset prices St = (S1t , . . .SM

t ) follows a multivariate GBM if

dSt = µStdt +BStdWt (6)

whereµ = (µ1, . . . ,µM) is the vector of expected rate of returns;B is a MxM matrix such that C = BBT (obtained by a Cholesky decomposition);Wt is a vector of M independent Wiener processes.

It is known (see for example Lemieux 2004) that the price of asset j = 1 . . .M is described by the equation:

S jt = S j

0exp[(µ j−σ2

j2 )t +σ jW j

t ] (7)

where W jt can be simulated as

M∑

i=1B jiZi with Zi i.i.d. N(0,1). Scenarios for the future asset prices can be generated

using (7).

GBM has several disadvantages. One of them is that it fails to capture extreme events; the stocks prices processis much smoothed as compared to its actual realisations - see for example Hardy (2001). Another disadvantage is theassumption of constant variance over time. Empirical studies show however time varying volatility of stock prices and"volatility clustering", meaning that periods of large volatility are followed by periods of small volatility and so forth.In order to address this aspect, Engle (1982) proposed the ARCH (Autoregressive Conditional Heteroscedasticity)model and Bollerslev (1986) proposed the GARCH (Generalised ARCH) model.

GARCH modelIf an asset's returns Rt is assumed to vary about a mean value c, we can write

yt = c+ εt (8)

where εt is an error term with mean 0. Vart−1(εt) = σ2t is the variance conditional on information at time t−1.

The GARCH(1,1) model,

σ2t = a0 +a1ε2

t−1 +b1σ2t−1 (9)

11

is widely used in nancial applications. It assumes that the variance of the current error term depends only on theprevious error term and of the variance of the previous error term. Parameters a0,a1,b1 are estimated using maximumlikelihood methods (see for example Fiorentini et al. 1996). Once this estimation is done, the variance of the currenterror term can be calculated, and then scenarios for the return Rt can be generated using (8).

A multivariate GARCH (M-GARCH) model captures time varying volatility of individual assets' returns and timevarying correlations between asset returns. A Principal Component Multivariate GARCH (PC-MGARCH) model canbe used. The multivariate assets returns' time series are transformed into uncorrelated time series (principal compo-nents). These are then modelled using univariate GARCH processes. For more details see Mitra et al. (2009).

In order to illustrate the MGARCH model for ve indices we t a PC-MGARCH model and simulate multiplesamples from this to determine a set of equi - probable scenarios. In Figure 3 we show the simulated paths for the S&P500.

Figure 3: Simulated sample paths for the S&P 500 from an PC-MGARCH model

GBM and GARCH are two well known models for describing asset prices that can be used as scenario generators,as shown above. More recent nancial scenario generators have been proposed in an attempt to address (some of) theshortcoming of the above methods. In particular they have concentrated on representing extreme price movements.The requirement for descriptive models which account for tail risk (risk associated with extreme events) has been notedin recent years. Scenario generators that capture extreme events are particularly important when used in conjunctionwith an optimisation model that minimises downside or tail risk.

Scenario generation based on Hidden Markov models (HMM) has been the subject of some recent papers, forexample Messina and Toscani (2007) for univariate data and by Roman, Mitra and Spagnolo (2009) for multivariatedata. The idea behind HMM is that, as well as the stochastic process of interest (stocks prices) that are observable (forwhich we want to generate future scenarios), there is another stochastic process, describing the "state of the system"or "state of the economy", that is not directly observable (hidden). At each time point, the "system" could be intoone of N states; an outcome for the stocks prices is generated according to the distribution corresponding to that state.Roman et al. (2009) generated using mixtures of normal distributions in each state (the parameters of the mixtures aredifferent for each state and estimated from historical data).

This "regime-switching" approach is motivated by empirical observations of sequentially changing behaviour ofnancial time series. For example, periods of low volatility may be followed by periods of high volatility or periodsof "crash" (extreme events). The numerical experiments of Roman et al. (2009) support the idea that one state is

12

"responsible" for generating extreme outcomes. The methods based on HMM are however difcult to implement andstill experimental.

Also with the view of appropriately generating extreme outcomes, Mitra et al. (2009) use a mixture model todescribe asset returns where downside events are captured by a copula model. The distribution for the upside events isdescribed by a multivariate generalized autoregressive conditional heteroskedasticity (MGARCH) model. The copulamodel uses the generalised Pareto distribution (GPD) for the marginal distributions and a t-copula to describe thedependence structure between the marginals. GPD is widely applied for extreme value theory problems. (See Mitra(2009)) The approach is motivated by the fact that M-GARCH, although suitable for modelling asset returns under"normal" conditions, may fail to do so under extreme market conditions. On the other hand, it is observed that assetreturns have a high level of dependency in the lower tails; this makes copula models more suitable for modellingworst-case scenarios. Mitra et al. (2009) used this scenario generator for mean-CVaR (Rockafellar and Uryasev 2000)and mean-variance-CVaR optimisation models (Roman et al 2007) with good results. We display and discuss some ofthese results in the next section.

4 Ex post evaluationA scenario generator for random parameter values and a decision model taken together dene a framework for decisionmaking under uncertainty. These are in sample use of scenario generators. The decisions reached by this or any othermeans are often evaluated through a collection of out of sample scenarios. The collection of such scenarios could besubsequent sequences of historical data (as applied in backtesting). The collection could be made up of bootstrappedscenarios or obtained through the application of another scenario generator. Backtesting involves testing the strategyusing historic data. The framework is applied to a historic period to determine the optimal decisions. The outcomewhich would have occurred, given the decision, is then found. The rationale of this approach is that if the strategyperformed well in the past it may prove an effective approach in the future. Backtesting considers a single particularrealisation for the uncertain parameters.

It is useful from a practioners' view point to be able to apply alternative scenario generators, to both the ex antedecision model and for ex post evaluation, in a exible fashion. In di Dominica et al. (2007) we put forward the designconsideration of connecting scenario generators to a SP modelling and solution system. In Figure 4 we outline an in-tegrated system with an open architecture which facilitates decision making and analysis of the resulting decisions. Inthis integrated modelling platform, the decision tool can be chosen from any of the well known approaches, expectedvalue linear program (EVLP), two stage SP, multi stage SP, chance constrained SP, integrated chance constrained SP,even a robust optimisation model. The results of these decision models can be subsequently evaluated using alternativescenario generators. The scenario generators can be chosen to reect the problem owner and domain experts view ofthe uncertainty.

Since the parameters are uncertain, outcomes of interest resulting from the decision taken (such as the objectivefunction value), are themselves random variables. The distributions of these items are dependent on the true distribu-tions of the parameters. Simulation involves using scenario generators to determine large scenario sets which are usedto investigate the (possible) distribution of random variables of interest. Such analysis may be undertaken in-sampleusing the same scenario generator that was applied in the ex ante decision making process, or out-of-sample using analternative scenario generator. Normally scenario sets for simulation analysis are larger than those used in the decisionmodels, since no optimisation is undertaken. For example, for the asset allocation problem we could investigate thedistribution of portfolio returns for a particular (optimal) choice of asset holdings. Statistical measures, such as themean, variance, skewness, kurtosis as well as ranges of this distribution can be reported. Similarly risk measures,which summarise the uncertainty of outcomes and possible adverse impact, can also be reported; for example, theConditional Value at Risk (CVaR). In the context of nance a number of performance measures such as the Sharperatio and Sortino ratio are often computed; in our modelling platform these indices are easily obtained.

13

Statistical measures: Mean, variance, skewness, kurtosis

Stochastic measures: EVPI, VSS

Risk measures: VaR (Value at Risk) CVaR (Conditional Value at Risk), Standard

deviation

Performance measures: Sharpe ratio, Sortino ratio

SIMULATIONAND

EVALUATION

Expected value LP

Two stage SP with recourse

Multi stage SP with recourse

Chance constrained SP

Integrated chance constrained SP

ScenarioGenerator 1

ScenarioGenerator 2

ScenarioGenerator 3

MODELS OF (PARAMETER) RANDOMNESS

EX

AN

TE

DE

CIS

IO

N M

OD

EL

S

Figure 4: Simulation ow

14

There are a group of stochastic measures associated with SPs. These include the Expected Value of Perfect In-formation (EVPI) and Value of Stochastic Solution (VSS). These reveal whether there is value in considering thedecision problem under a stochastic setting, that is, whether the stochasticity of the problem impacts the optimumdecision. (See Birge and Louveaux 1997). In di Dominica et al. (2007) we give extensive numerical results investi-gating these measures for the stochastic Newsboy Problem and the decisions are evaluated by using several alternativescenario generators as simulators.

5 Assessing the quality of scenario generation methodsIt is understood we do not know the correct(or true) distribution for asset returns. Though it is common to pos-tulate a theoretical distribution, this implies a belief. Sample based methods, if the sample is sufciently large andthe sampling method is accurate, converge arbitrarily close to this distribution. But given that we do not know thecorrect distribution it can be questioned how useful it is to have convergence to this postulated distribution. Eventhe use of historical data for bootstrapping, or for model calibration, is a question of belief, as we are implicitly as-suming the past is a reasonable description of the future. The period of history considered can signicantly inuencethe (parameter) estimates. It is well known the inclusion or exclusion of 1987 stock market crash inuences volatilityestimates signicantly (Black Monday see Alexander(2001)). We note that given this ambiguity about the correctdistribution, convergence properties are not the most important aspect of scenario generation. It is the performance ofthe scenarios as inputs to a decision model and the quality of the subsequent decisions that matter. Kaut and Wallace(2007) make this important point: We are not concerned about how well the distribution is approximated, as longas the scenario tree leads to `good' decisions.. That is the decisions should be good approximation of the optimaldecisions. However, it is not possible to tell exactly how good is the approximation we have obtained - if we knew the"true" optimal value and decisions we would not need scenario generation in the rst place. Testing the quality of ascenario is not a precise task and has subjective elements. The quality of the decisions can be assessed by backtesting,simulation, in-sample, and out-of-sample stability testing and other out-of-sample analysis, such as stress testing. Wediscussed some aspects of ex post analysis in section 4.

Kaut and Wallace (2007) note that a good decision should at least be a stable decision. They outline the require-ments of stability for a scenario generation method which is considered in respect of a particular decision model. Thecombined use of the scenario generator and decision model should lead to stable decisions.

A scenario generator is said to manifest in-sample stability if, when generating several scenario sets of the samesize and solving the optimisation problem on each of these scenario sets, the optimal objective values are similar (seefor example Kaut and Wallace 2007). (We note that it is not a requirement that the optimal decision variable valuesare similar as well; this is because the optimisation problems may have very different solutions leading to the same orsimilar values of the objective functions).

A scenario generator is said to manifest out-of-sample stability, if, when generating several scenario sets of thesame size and solving the optimisation model on each of these scenario sets, the optimal solutions obtained yield anobjective function value close to the "true" optimal objective function value.

While it is straightforward to test in-sample stability (we only solve the scenario-based optimisation problems),it is difcult to test the out-of-sample stability. The "true" distributions must be known and even then it may be notstraightforward to evaluate the objective function. In practice a "benchmark" scenario tree is applied : a very largescenario set obtained with an exogenous scenario generation method, that is known to be stable. This scenario set willstand for the "true" distribution. Of course, if the optimal solutions are similar, then there is out-of-sample stability,since these solutions are evaluated on the same scenario set. However this may be too strong a condition. We mayhave out-of-sample stability even if the optimal solutions are not similar.

15

To test the in-sample stability, the same scenario set is used for both making a decision and evaluating it. To testthe out-of-sample stability, a scenario set is used for making a decision and another scenario set (the "benchmark")is used for evaluating it. The out-of-sample stability is important since this implies that the real performance of thesolution is stable, that is, it does not depend on which scenario set with which we have solved the optimisation problem.In-sample stability does not imply the out-of sample one or vice versa. For more details, see Kaut and Wallace (2007).

6 A case studyWe provide a case study similar to that presented in Mitra et al (2009) based on a mixture model, used for a mean-CVaR optimisation model. We give an example of testing the stability properties (described above), in this framework.

Conditional Value at Risk (CVaR) of a random return R at a chosen condence level α ∈ (0,1) is a risk measurethat expresses the mean of the worst A% outcomes of R, where α = A% (α close to 0, it represents a small sampleof worst outcomes). It was proved by Rockafellar and Uryasev (2000, 2002) that for a given portfolio x, the CVaR ofthe portfolio return Rx can be calculated by solving a LP. Moreover, minimising the CVaR over the set of all feasibleportfolios is a LP problem as well. We consider the optimisation model:

min CVaR(Rx)such that x ∈ X

E(Rx)≥ d(10)

where d is a minimum value required for the expected portfolio return. For the algebraic form of the mean-CVaRmodel, see Rockafellar and Uryasev (2000, 2002), Mitra et al. (2009).

We have tested the in-sample stability using scenario sets of size 500,1000,2000,4000; we generate K = 30 sce-nario sets of each size. For each scenario set size we display the statistics of the 30 optimal objective values in Table 1.As expected, as the number of scenarios increases, the range (and standard deviation) of the objective values decreases.This is also illustrated in the box plot displayed in Figure 5.

An important point is that it is possible to have in-sample stable solutions. On a component by component basis ifthe values of the solution vector in the in-sample investigation, turn out to be narrowly bounded, this guarantees out-of-sample stability. This is because, if the solutions are almost identical then for any parameter distribution they willgive similar objective values. We analyse the 30 asset holding decisions obtained for the sets of 4000 scenarios. Thestatistics for these solutions are displayed in Table 2. These are relatively stable indicating we may have out-of-samplestability as well we in-sample stability.

In the decision model d (the minimum expected portfolio return required) is set to 0.005% per month (1.00512−1 = 0.0617% per annum). α is set to 0.05 = 5% (we minimise the expected loss in the worst 5% of cases). The indi-vidual asset holdings can be at a minimum 0.5% of the total portfolio and a maximum of 100% of the total portfolio,short holdings are excluded. Hence, X = (x1, . . . ,xn)|

n∑j=1

x j = 1,1≥ x j ≥ 0.005,∀ j ∈ 1, . . . ,n.

The mixture scenario sets describe the joint distribution of monthly returns for 5 Equity Index Exchange TradedFunds(EIETFs). These funds track the TOPIX, S&P European 350, FTSE 100, S&P 500 and the Hang Seng (SeeMitra et. al. (2009) for further details).

16

Table 1: In-sample stability of mean-CVaR objective value for varying sizes of the scenario set for K=30 sets

500 Scenarios 1000 Scenarios 2000 Scenarios 4000 ScenariosAverage 0.06189 0.06230 0.06375 0.06337Std dev 0.00392 0.00261 0.00210 0.00119Range 0.01792 0.00978 0.00844 0.00490Min 0.05269 0.05652 0.05895 0.06105Max 0.07061 0.06630 0.06739 0.06595

Figure 5: Box plot of distribution of optimum objective function values for varying scenario set sizes

Table 2: In-sample stability of mean-CVaR solution values for scenario sets of size 4000

Stocks Mean Standard deviation Range Minimum MaximumEURO350 0.17471 0.02999 0.10665 0.11332 0.21997FTSE100 0.14936 0.04435 0.16124 0.06506 0.22630SP500 0.40088 0.02589 0.09969 0.36086 0.46054TOPIX 0.18605 0.02151 0.08352 0.15009 0.23361HANGSENG 0.08901 0.02162 0.09392 0.03473 0.12865

17

7 ConclusionsWe have examined the role of scenario generation in the context of SP - both in ex ante decision making and ex postanalysis of decisions. Generally, using SP for decision making involves two aspects: a clearly dened optimisationmodel (the "decision" model) and a discrete representation of the uncertain parameters (the "scenarios"). Scenariogenerators are thus used for decision making through the numerical solution of the optimisation models; they are alsoused for evaluating the quality of a decision by simulating its possible performance. Testing the quality of a scenariogenerator is a difcult and subjective task. We emphasise the perspective that the quality of a scenario generator isintimately connected with the underlying computational (decision) model. A scenario generator should be "correct"(that is, it appropriately represents the randomness involved) and "consistent" (that is, it takes into account the depen-dency between multiple random variables). These are subjective assessments. Stability conditions are a formal wayof checking that an SG is t for purpose. The most far reaching applications of SP are for nance; thus, scenario gen-eration for nancial optimisation, particularly for asset prices, is an important and timely subject of research. Thereare several ways of creating scenario sets for future asset prices. Apart from general-purpose methods of scenariogeneration (e.g. bootstrapping), there are methods based on well-known models for asset price dynamics (GeometricBrownian Motion, GARCH). These models of randomness, although well established and with a wide range of appli-cations, may not be the most appropriate as a basis for scenario generation, particularly for optimisation problems thataddress hedging under worst case scenarios. We have considered some recently reported methods of scenario gener-ation which have been proposed for distributions with `heavy tails'. We have illustrated the stability property for oneof these recent methods (based on a mixture between a GARCH and a copula model) in a mean-CVaR optimisationsetting.

8 AcknowledgementsThe research reported in this chapter was variously supported in the following ways. Ms Leela Mitra is an Engineeringand Physical Sciences Research Council (EPSRC) UK , PhD scholar with industrial support provided by OptiRiskSystems. Dr Diana Roman worked as a Knowledge Transfer Partnership (KTP) research associate. This was fundedby the Department of Trade and Industry, UK and OptiRisk Systems. The support from these sources is gratefullyacknowledged. The authors also acknowledge the research discussions with Micheal Sun, PhD scholar and KemingYu, Reader in the Department of Mathematical Sciences at Brunel.

References[1] C. Alexander. Market Models: A Guide to Financial Data Analysis. John Wiley & Sons, Chichester, UK, 2001.

[2] J.R. Birge and F. Louveaux. Introduction to Stochastic Programming. Springer, 1997.

[3] F. Black and M. Scholes. The pricing of options and corporate liabilities. Journal of Political Economy,81(3):637659, 1973.

[4] T. Bollerslev. Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31(3):307327, 1986.

[5] N. Di Domenica, C. Lucas, P. Valente, and G. Mitra. Stochastic Programming and Scenario Generation within aSimulation Framework: A Modelling System Perspective. IMA Journal of Management Mathematics, 20(1):138, 2007.

[6] J. Dupacová, G. Consigli, and S.W. Wallace. Scenarios for multistage stochastic programs. Annals of OperationsResearch, 100(3):2553, 2000.

18

[7] J. Dupacova and K. Sladky. Comparison of multistage stochastic programs with recourse and stochastic dynamicprograms with discrete time. Zeitschrift für angewandte Mathematik und Mechanik, 82(11):753765, 2002.

[8] B. Efron and R.J. Tibshirani. An Introduction to the Bootstrap. Monographs on Statistics and Applied Probabil-ity, 57, 1993.

[9] R.F. Engle. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdomination. Econometrica, 50(4):9871007, 1982.

[10] W.K.K. Haneveld. Duality in stochastic linear and dynamic programming. Volume 274 of Lecture Notes inEconomics and Mathematical Systems, Springer-Verlag Berlin, 1986.

[11] J.M. Harrison and D.M. Kreps. Martingales and arbitrage in multiperiod securities markets. Journal of EconomicTheory, 20:381408, 1979.

[12] J.M. Harrison and S.R. Pliska. Martingales and stochastic integrals in the theory of continous trading. StochasticProcess Applications, 11:215260, 1981.

[13] D. Heath, R. Jarrow, and A. Morton. Bond pricing and the term structure of interest rates: A new methodologyfor contingent claims valuation. Econometrica, 60(1):77105, 1992.

[14] K. Høyland, M. Kaut, and S.W. Wallace. A heuristic for moment-matching scenario generation. ComputationalOptimization and Applications, 24(2):169185, 2003.

[15] P. Kall and S.W. Wallace. Stochastic programming. Wiley New York, 1994.

[16] M. Kaut and S.W. Wallace. Evaluation of scenario generation methods for stochastic programming. Pac. PacicJournal of Optimization, 3(2):257271, 2007.

[17] J. Lintner. The valuation of risk assets and the selection of risky investments in stock portfolios and capitalbudgets. Review of Economics and Statistics, 47(1):1337, 1965.

[18] D.G. Luenberger. Investment Science. Oxford University Press, 1998.

[19] L. Mitra, X. Sun, D. Roman, G. Mitra, and K. Yu. Mixture scenario sets for investment decisions with downsiderisk. Working Paper CARISMA, Brunel University, www.carisma.brunel.ac.uk; To appear Willmott Journal,2008.

[20] J.M. Mulvey. Generating scenarios for the Towers Perrin investment system. Interfaces, 26:115, 1996.

[21] R.T. Rockafellar and S. Uryasev. Optimization of Conditional Value-at-Risk. Journal of Risk, 2:2142, 2000.

[22] R.T. Rockafellar and S. Uryasev. Conditional Value-at-Risk for General Loss Distributions. Journal of Bankingand Finance, 26:14431471, 2002.

[23] D. Roman, K. Darby-Dowman, and G. Mitra. Mean-risk models using two risk measures: a multi-objectiveapproach. Quantitative Finance, 7(4):443458, 2007a.

[24] D. Roman and G. Mitra. Portfolio selection models: A review and new directions. To appear in Willmott Journal,2009.

[25] S.A. Ross. The Arbitrage Pricing Theory of Capital Asset Pricing. Journal of Economic Theory, 13(3):341360,1976.

[26] W.F. Sharpe. Capital Asset Prices: A Theory of Market Equilibrium Under Conditions of Risk. Journal ofFinance, 19(3):425442, 1964.

[27] RM Van Slyke and R. Wets. L-shaped linear programs with applications to optimal control and stochasticprogramming. SIAM Journal on Applied Mathematics, 17:638663, 1969.

19

[28] K. Volosov, G. Mitra, F. Spagnolo, and C. Lucas. Treasury management model with foreign exchange exposure.Computational Optimization and Applications, 32(1):179207, 2005.

[29] D. Vose. Risk Analysis: A Quantitative Guide. Wiley, 2002.

[30] S.W. Wallace and WT Ziemba. Applications of stochastic programming. SIAM mathematical series on optimi-sation, SIAM, Philadelphia, 2005.

[31] AD Wilkie. More on a stochastic asset model for actuarial use. British Actuarial Journal, 1(5):777964, 1995.

[32] S. Zenios. Practical Financial Optimization: Decision Making for Financial Engineers. Blackwell Publishers,Oxford, 2007.

20