Exploratory Modelling: Emerging Approaches for the ... · MulE-ObjecEve Robust OpEmisaon MORO...

ExploratoryModelling:EmergingApproachesfortheTreatmentofDeepUncertain;esinSystemsModelling

EnayatA.Moallemi28,September,2017

Outline•  Backgroundonexploratorymodelling•  Twoexploratorymodellingapproaches•  ApplicaEonsanddecisioninsights•  Concludingremarks

2

3

SystemsModelling&DeepUncertainty

Themodellingofreal-worldsystemischallengingbycomplexityanddeepuncertainty.TradiEonalSystemsmodellingapproachesfailtocopedeepuncertain2es(Lempertetal.2003).

Credit:duckfarmondeviantART

Lempert,R.J.,Popper,S.W.,&Bankes,S.C.(2003).Shapingthenextonehundredyears:newmethodsforquanEtaEve,long-termpolicyanalysis:RandCorporaEon.

4

ExploratorymodellingExploratorymodellingresultsinaporXolioofwhatcouldhappen,asopposedtowhatwillhappen(Bankes1993).

Itisgrowingrapidlyintoseveralapproaches(e.g.RDM,DAPP,MORDM,EEA).ItisadoptedinvariousapplicaEondomains(e.g.planninginwater,energy,defence,climate,infrastructure).

Bankes,S.(1993).Exploratorymodelingforpolicyanalysis.Opera2onsResearch,41(3),435-449.

5

TwoexploratorymodellingpracEcesinCSC

Approaches

•  RobustDecisionMaking•  MulE-ObjecEveRobustOpEmisaEon

Applica;onAcquisiEonandmaintenancemanagementofaircra]fleets

•  Decision:Numberofaircra]&sizeofmaintenance•  Systemperformance:availabilityofaircra]&totalcosts

∼RobustDecisionMaking(RDM)∼

6

7

RobustDecisionMakingParEcipatoryscoping(defineuncertainEes,

strategies,relaEonships,andobjecEves)

CasegeneraEon(esEmatetheperformanceof

strategiesinmanyfutures)

ScenarioexploraEonanddiscovery(characterise

vulnerabiliEesofstrategies)

Trade-offanalysis(displayandevaluatetrade-offsbetween

strategies)

PlanforsimulaEonmodelling

DatabaseofsimulaEonresults

InformaEonon

vulnerabiliEes

Insightsintomorerobuststrategies

Robuststrategy

InformaEonon

vulnerabiliEes

InformaEontohelpchoose

candidatestrategies

Scenariosthatilluminate

vulnerabiliEes

(Lempertetal.2003)

Lempert,R.J.,Popper,S.W.,&Bankes,S.C.(2003).Shapingthenextonehundredyears:newmethodsforquanEtaEve,long-termpolicyanalysis:RandCorporaEon.

RDMfocusesonextremecondi2onsandeventswithlesslikelihoodratherthanmost-likelyscenarios.EmphasisesonidenEficaEonfailurescenariosratherthansuccessscenarios.

8

TheRDMprocess ParEcipatoryscoping(defineuncertainEes,




ScenarioexploraEonanddiscovery

(characterisevulnerabiliEesofstrategies)


strategies)



InformaEononvulnerabiliEes


Robuststrategy


InformaEontohelpchoosecandidatestrategies


vulnerabiliEes

9

ParEcipatoryscoping(defineuncertainEes,







strategies)





Robuststrategy




vulnerabiliEes

TheRDMprocess

10








strategies)





Robuststrategy




vulnerabiliEes

TheRDMprocess

11

ScenarioDiscovery

𝐶𝑜𝑣𝑒𝑟𝑎𝑔𝑒= #𝑐𝑎𝑠𝑒𝑠_𝑜𝑓_𝑓𝑎𝑖𝑙𝑢𝑟𝑒 𝑖𝑛 𝐵𝑜𝑥 1 /#𝑐𝑎𝑠𝑒𝑠_𝑜𝑓_𝑓𝑎𝑖𝑙𝑢𝑟𝑒 𝑖𝑛 𝑡𝑜𝑡𝑎𝑙 

𝐷𝑒𝑛𝑠𝑖𝑡𝑦= #𝑐𝑎𝑠𝑒𝑠_𝑜𝑓_𝑓𝑎𝑖𝑙𝑢𝑟𝑒 𝑖𝑛 𝐵𝑜𝑥 1 /#𝑡𝑜𝑡𝑎𝑙_𝑐𝑎𝑠𝑒𝑠 𝑖𝑛 𝐵𝑜𝑥 1 

IdenEfyingandexplainingfailurescenariosusingtherelevantspaceofuncertaintyintheinputparameters(BryantandLempert2010).

Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Uncertaintya

Uncertaintyb

Box1

FailurecasesSuccesscases

Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Box16

Coverage

Density

Bryant,B.P.,&Lempert,R.J.(2010).Thinkinginsidethebox:AparEcipatory,computer-assistedapproachtoscenariodiscovery.TechnologicalForecas2ngandSocialChange,77(1),34-49.

12

ScenarioDiscovery



IdenEfyingandexplainingfailurecasesusingtherelevantspaceofuncertaintyintheinputparameters(BryantandLempert2010).

Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Uncertaintya

Uncertaintyb

Box1


Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Box16

Coverage

Density


13

ScenarioDiscovery




Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Uncertaintya

Uncertaintyb

Box1


Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Box16

Coverage

Density


14

ScenarioDiscovery




Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Uncertaintya

Uncertaintyb

Box1


Box1

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Box16

Coverage

Density


15








strategies)





Robuststrategy




vulnerabiliEes

TheRDMprocess

16

•  StrategyI:HighAcquisiEon

LowMaintenance

•  StrategyII:MediumAcquisiEonMediumMaintenance

•  StrategyIII:LowAcquisiEonHighMaintenance

Howvulnerableareourstrategiesinsecuringaverageflyinghours>5000hoursandtotalcosts<$1300billion?

Uncertain parameter Range The risk that that an aircraft is lost during operation 0.00026 – 0.00234 (-) Lifetime of aircraft 37440 – 336690 (hour) Total required flying hours with a uniform distribution 12 – 109 (hour/week) Expected time spent by an aircraft in CAP 20 – 28 (week) Time between CAP events 16 – 24 (week) Expected time spent by an aircraft in DM 8 – 10 (week) Time (flying hours) between DM events 200 – 1800 (hour) Expected time spent by an aircraft in OM 3 – 5 (week) Time between OM events 50 – 450 (hour) Cost of OM 0.1 – 2.0 ($ billion)

Strategies Uncertain;es

17

FlyinghoursobjecEve5000hours

CostobjecEve<$1300B

18


CostobjecEve<$1300B

19


CostobjecEve<$1300B

Strategy Measure of quality Uncertainty Range of failure P-value Medium Acquisition-Medium Maintenance

Coverage: 0.36 Time between operational maintenance

50 – 140 (hour) 2.7e-23

Density: 1 Low Acquisition- High Maintenance


51 – 220 (hour) 1.4e-6

Density: 0.48 Time between deep maintenance 290 – 1300 (hour) 1.4e-2

strategy Measure of quality Uncertainty Range of failure P-value Medium Acquisition-Medium Maintenance

Coverage: 0.54 Time between deep maintenance 330 – 930 (hour) 1.7e-10

Density: 0.88 Cost of operational maintenance 1 – 2 ($ billion) 1.9e-7

Time between operational maintenance 170 – 450 (hour) 4.1e-03

Low Acquisition- High Maintenance


Density: 0.98 Required rate of effort 39 – 110 (hour/week) 7.1e-6 Time between operational

maintenance 140 – 450 (hour) 1.6e-5 Risk of loss 0.00026 – 0.0018 (–) 4.6e-5 Lifetime 44000 – 320000

(hour) 2.5e-2

Time spent in operational maintenance 3 – 4.8 (week) 3.4e-2

Cost of operational maintenance 0.18 – 2 ($ billion) 5.0e-2

Scenariosleadingtolessthan5000(hour)averageflyinghours

Scenariosleadingtomorethan1300($billion)costs

20


CostobjecEve<$1300B

Strategy Measure of quality Uncertainty Range of failure P-value Medium Acquisition-Medium Maintenance


50 – 140 (hour) 2.7e-23

Density: 1 Low Acquisition- High Maintenance


51 – 220 (hour) 1.4e-6

Density: 0.48 Time between deep maintenance 290 – 1300 (hour) 1.4e-2

strategy Measure of quality Uncertainty Range of failure P-value Medium Acquisition-Medium Maintenance


Density: 0.88 Cost of operational maintenance 1 – 2 ($ billion) 1.9e-7

Time between operational maintenance 170 – 450 (hour) 4.1e-03

Low Acquisition- High Maintenance


Density: 0.98 Required rate of effort 39 – 110 (hour/week) 7.1e-6 Time between operational

maintenance 140 – 450 (hour) 1.6e-5 Risk of loss 0.00026 – 0.0018 (–) 4.6e-5 Lifetime 44000 – 320000

(hour) 2.5e-2

Time spent in operational maintenance 3 – 4.8 (week) 3.4e-2

Cost of operational maintenance 0.18 – 2 ($ billion) 5.0e-2

Scenariosleadingtolessthan5000(hour)averageflyinghours

Scenariosleadingtomorethan1300($billion)costs

∼MulE-ObjecEveRobustOpEmisaEon(MORO)∼

21

22

MulE-ObjecEveRobustOpEmisaEon

MOROconsidersmul2plecontradic2ngobjec2vesofstakeholdersandfuturepossibleadapta2ons.MOROselectadap2verobustsolu2onswhichfulfilstakeholderobjecEvesandcanbemodifiedunderchangingcircumstances.

1. Problem formulation

1.  Uncertainties 2.  Decision levers and

solutions 3.  Quantitative relationship 4.  Performance measures

2. Identification of scenario clusters

1.  Generation of future scenarios

2.  Clustering of scenarios 3.  Identification of the

conditions of scenario clusters

3. Enumeration of robust solutions for each scenario

cluster

1.  Identification of failure scenarios

2.  Enumeration of Pareto optimal sets

4. Adaptation and trade-off among robust solutions

1.  Identification of adaptation tipping point

2.  Development of adaptive pathways

23









cluster



4. Adaptation & trade-off among robust solutions



24









cluster






25









cluster






26









cluster






27

•  AllpossiblevariaEonofdecision

levers(343strategies)

Whatstrategiestochooseandwhentoadapttomaximiseaveragein-serviceaircra]andtominimisetotalcostsoverEme?

Strategies Uncertain;es

Context uncertainties Discrete range

The risk that an aircraft is lost during operations 0.00026, 0.00078, 0.0013, 0.00182, 0.00234 (-)

Total required flying hours 12, 48, 109 (-)

Expected time spent by an aircraft in CAP 20, 24, 28 (week)

Expected time spent by an aircraft in DM 8, 9, 10 (week)

Expected time spent by an aircraft in OM 3, 4, 5 (week)

Cost of OM 0.1, 0.575, 1.05, 1.525, 2 ($ billion)

Performance measure Objective Threshold Average of available aircraft for service Maximisation No less than 2 (–) Total acquisition and maintenance costs Minimisation No more than 400 (B$)

Performancemeasures

Futurescenarioclusters

28

Cluster0

Cluster1

Cluster3

Cluster2

Cluster4


29

Cluster0

Cluster1

Cluster3

Cluster2

Cluster4

Cluster Uncertainty Range P-value

Cluster 0 Number of purchased aircraft 3 – 5 (–) 8.4e-10 Required flying hours 45 – 109 (hour/week) 3.8e-09 Time between OM events 50 – 290 (hour) 7.0e-3

Cluster 1 Number of purchased aircraft 1 – 2 (–) 1.4e-34

Cluster 2 Required flying hours 12 – 37 (hour/week) 1.2e-13 Number of purchased aircraft 5 – 7 (–) 7.8e-8

Cluster 3 Number of purchased aircraft 3 – 4 (–) 3.2e-9 Required flying hours 12 – 55 (hour/week) 3.0e-8 Time between OM events 87 – 450 (hour) 1.0e-2 Time between DM events 470 – 1800 (hour) 1.4e-2

Cluster 4 Number of purchased aircraft 6 – 7 (–) 1.2e-12 Required flying hours 34 – 82 (hour/week) 9.6e-4 Time between DM events 350 – 1600 (hour) 4.2e-3

Condi;onsleadingtoeachscenariocluster


30

Cluster0

Cluster1

Cluster3

Cluster2

Cluster4

Cluster Uncertainty Range P-value

Cluster 0 Number of purchased aircraft 3 – 5 (–) 8.4e-10 Required flying hours 45 – 109 (hour/week) 3.8e-09 Time between OM events 50 – 290 (hour) 7.0e-3

Cluster 1 Number of purchased aircraft 1 – 2 (–) 1.4e-34

Cluster 2 Required flying hours 12 – 37 (hour/week) 1.2e-13 Number of purchased aircraft 5 – 7 (–) 7.8e-8

Cluster 3 Number of purchased aircraft 3 – 4 (–) 3.2e-9 Required flying hours 12 – 55 (hour/week) 3.0e-8 Time between OM events 87 – 450 (hour) 1.0e-2 Time between DM events 470 – 1800 (hour) 1.4e-2

Cluster 4 Number of purchased aircraft 6 – 7 (–) 1.2e-12 Required flying hours 34 – 82 (hour/week) 9.6e-4 Time between DM events 350 – 1600 (hour) 4.2e-3

Condi;onsleadingtoeachscenariocluster

Cluster0

Inputuncertaintyspace

ParetoopEmalstrategiesineachcluster

31

ParetoopEmalstrategiesundereachcluster

32Decisionvariables

ParetoopEmalstrategiesineachcluster

33Performance

ParetoopEmalstrategiesundereachcluster

34

35

Requiredflyinghours(40-60hour/week)



MonitoringforadaptaEon

36





37

Requiredflyinghours(40-60hour/week)•  Cluster0

Requiredflyinghours(25-40hour/week)•  Cluster4

Requiredflyinghours(10-25hour/week)•  Cluster1•  Cluster2•  Cluster3


38

Whatisthetrade-offamongParetoopEmaldecisionsunderlowrequiredflyinghours

whenin-serviceaircraS>6andtotalcosts<B$250?

39

Whatarethechosenstrategiesunderlowrequiredflyinghours


40



41



42

Conclusions:Whatarethebenefitsofexploratorymodellingtosystemsmodelling?

43


•  Exploratorymodellingincorporatesthediversityoftheirviewsandprovokesdelibera2on,experimenta2on,andlearning.

44


•  Exploratorymodellingcanenhancetheconfidenceofresultsbycapturingawiderangeofpossiblefuturesandconsideringunexpectedcircumstances.


45


•  Exploratorymodellingcanenhancetheconfidenceofresultsbycapturingawiderangeofpossiblefuturesandconsideringunexpectedcircumstances.

•  Exploratorymodellingproducesan2cipatoryandprotec2veac2onsinsteadofreacEveandmiEgaEngacEons.


46

FutureresearchdirecEons

•  Tointegratemoreofexploratorymodellingapproacheswithsystemengineeringtechniques

•  ToidenEfythenewusesofexploratorymodellinginthesystemsmodellingprocess

CapabilitySystemsCentre(CSE)SchoolofEngineeringandInformaEonTechnology

TheUniversityofNewSouthWales(UNSWCanberra)

PainEngintheEtlepage:RobertDelaunay,1938,Rythmen°1,DecoraEonfortheSalondesTuileries,oiloncanvas,Muséed'ArtModernedelavilledeParis. 47

@EnayatMoallemi

[email protected]

EnayatA.Moallemi

Exploratory Modelling: Emerging Approaches for the ... · MulE-ObjecEve Robust OpEmisaon MORO...

Documents

Transcript of Exploratory Modelling: Emerging Approaches for the ... · MulE-ObjecEve Robust OpEmisaon MORO...