Interactive Parallel & Distributed Simulation
description
Transcript of Interactive Parallel & Distributed Simulation
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 11
Interactive Parallel & Interactive Parallel & Distributed SimulationDistributed Simulation
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 22/39/39
OutlineOutline Introduction to Parallel and Introduction to Parallel and
distributed simulation (basic distributed simulation (basic concepts)concepts)
Methods in Interactive PADSMethods in Interactive PADS• OverviewOverview• ApproachesApproaches
Conclusion, current state, future Conclusion, current state, future workwork
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 33/39/39
Area DefinitionArea Definition
Time-steppedTime-stepped: : time advances by time advances by fixed time fixed time incrementsincrements
Event drivenEvent driven: : time advances time advances occur with irregular occur with irregular incrementsincrements
computersimulation
discretemodels
continuousmodels
eventdriven
time-stepped
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 44/39/39
Distributed SimulationDistributed Simulation
As fast as possibleAs fast as possible Analytical Analytical
simulationssimulations High Performance High Performance
Computing Computing CommunityCommunity
Conservative and Conservative and Optimistic (Time Optimistic (Time Warp) Warp) synchronizationsynchronization
RealtimeRealtime Virtual Virtual
environments, environments, hardware-in-the-hardware-in-the-looploop
Defense M&S Defense M&S communitycommunity
SIMNET -> DIS -> SIMNET -> DIS -> HLAHLA
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 55/39/39
Conservative SynchronizationConservative Synchronization
LP 4
LP 3
LP 2
LP 1
Simulation TimeT
events ready to be processed
T+L•L – L – LookaheadLookahead – Any message sent by a LP must have – Any message sent by a LP must have timestamp at least T + Ltimestamp at least T + L•Necessary to allow concurrent processing of events with different time stamps•Relies on model props, complicates the model developmentRelies on model props, complicates the model development
events not ready to be processed
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 66/39/39
Optimistic SynchronizationOptimistic Synchronization
LP 4
LP 3
LP 2
LP 1
Simulation TimeT
processed events
unprocessed events
rollback
•LP processes events without checking if an event in its past LP processes events without checking if an event in its past could later arrivecould later arrive•If an event arrives in the LPs past (If an event arrives in the LPs past (straggler messagestraggler message), the ), the computation is rolled backcomputation is rolled back
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 77/39/39
Optimistic SynchronizationOptimistic Synchronizationlocal control mechanismlocal control mechanism
Input Queue(event list)
straggler message arrives in the past, causing rollback
12 21 35
processed eventunprocessed eventsaved stateanti-message
State Queue
1212Output Queue(anti-messages)
•LP processes events in time order like sequential simulatorLP processes events in time order like sequential simulator•The state before processed events must be saved, the The state before processed events must be saved, the messages sent to other LPs must be also savedmessages sent to other LPs must be also saved•When a message arrives in the past (straggler message), the When a message arrives in the past (straggler message), the rollback is initiatedrollback is initiated
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 88/39/39
Optimistic SynchronizationOptimistic Synchronizationconcepts and notionsconcepts and notions
State savingState saving – various algorithms – various algorithms (Copy SS, Periodic SS, Incremental (Copy SS, Periodic SS, Incremental SS, …)SS, …)
Anti messagesAnti messages• In case of rollback are sent to In case of rollback are sent to
destination LPsdestination LPs• If LP gets anti message, the If LP gets anti message, the
corresponding waiting event is removed corresponding waiting event is removed from the queue (from the queue (annihilationannihilation))
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 99/39/39
Optimistic SynchronizationOptimistic Synchronizationconcepts and notionsconcepts and notions
Global virtual timeGlobal virtual time• Minimum timestamp of any unprocessed Minimum timestamp of any unprocessed
message or anti-message in the whole systemmessage or anti-message in the whole system• Used to reclaim memory used by SS and Used to reclaim memory used by SS and
output queues (output queues (fossil collectionfossil collection))• Also serves to commit blocking I/O operationsAlso serves to commit blocking I/O operations• Synchronous X Asynchronous GVT computationSynchronous X Asynchronous GVT computation• Wide spectrum of GVT algorithms exist today, Wide spectrum of GVT algorithms exist today,
the selection depends on application. The the selection depends on application. The algorithms are non-trivial in distributed algorithms are non-trivial in distributed environments.environments.
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1010/39/39
PADS ConclusionPADS Conclusion Conservative or Optimistic approachConservative or Optimistic approach Optimistic much more used than Optimistic much more used than
conservativeconservative
Next part: Interactive methods based Next part: Interactive methods based on PADSon PADS
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1111
Interactive simulationInteractive simulation
Methods in Interactive PADSMethods in Interactive PADS
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1212/39/39
Interactive simulation - outlineInteractive simulation - outline Introduction – applications & Introduction – applications &
requirementsrequirements Description of Interactive PADS Description of Interactive PADS
approachesapproaches• Multiple path explorationMultiple path exploration• Input / outputInput / output• Variable resolution modelingVariable resolution modeling
Summary, open issues, current stateSummary, open issues, current state
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1313/39/39
Applications of Interactive Applications of Interactive SimulationSimulation
Decision support - (Air) traffic control, Decision support - (Air) traffic control, planning, …planning, …
Interactive system analysis – Interactive system analysis – Networks, transportation systems, Networks, transportation systems, ecological systems, logistics, ecological systems, logistics, battlefield, …battlefield, …
Virtual environments – Training Virtual environments – Training simulators, medicine (virtual simulators, medicine (virtual surgery), … surgery), …
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1414/39/39
RequirementsRequirementswhat should an interactive simulator supportwhat should an interactive simulator support Exploration of different simulation Exploration of different simulation
futuresfutures• Rewind to some point and continue with Rewind to some point and continue with
different inputdifferent input• Execute multiple simulations Execute multiple simulations
concurrentlyconcurrently Fast input and outputFast input and output Variable resolution models Variable resolution models
• Interactive detail level changingInteractive detail level changing
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1515/39/39
Multiple Path ExplorationMultiple Path Exploration Uses – explore different simulation Uses – explore different simulation
paths based on different input paths based on different input parametersparameters
ApproachesApproaches• Multiple sequential simulations with Multiple sequential simulations with
different parametersdifferent parameters• Interactive simulation steering with Interactive simulation steering with
rollback & reexecutionrollback & reexecution• Cloning (Virtual processes)Cloning (Virtual processes)
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1616/39/39
Rollback & ReexecutionRollback & Reexecution Input events called ‘steering’ events – Input events called ‘steering’ events –
modify the simulation path according the modify the simulation path according the inputinput
Special ‘reexecution’ events allow to Special ‘reexecution’ events allow to rewind to the pastrewind to the past• Scheduled to the pastScheduled to the past• The simulation can take a new path by The simulation can take a new path by
inputting new data via steering eventsinputting new data via steering events• Suitable for Time Warp simulators that utilize Suitable for Time Warp simulators that utilize
the rollback as a part of the synchronizationthe rollback as a part of the synchronization
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1717/39/39
Rollback & ReexecutionRollback & Reexecution ProblemsProblems
• There must be a saved state prior the There must be a saved state prior the time of the reexecution eventtime of the reexecution event
• The available rollback length is limited - The available rollback length is limited - due to fossil collection in Time Warp due to fossil collection in Time Warp simulators the old saved states are simulators the old saved states are reclaimed as the GVT passesreclaimed as the GVT passes
• Some state saving schemes were Some state saving schemes were developed but the rollback is still limiteddeveloped but the rollback is still limited
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1818/39/39
Cloning (Virtual Processes)Cloning (Virtual Processes) Multiple simulation futures are computed Multiple simulation futures are computed
in parallelin parallel The futures share the computation until The futures share the computation until
they branch with different parametersthey branch with different parameters Uses the Virtual processes paradigmUses the Virtual processes paradigm
• All the computed simulations contain virtual All the computed simulations contain virtual processesprocesses
• All corresponding virtual processes are mapped All corresponding virtual processes are mapped to a single physical process, until the branchto a single physical process, until the branch
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 1919/39/39
Virtual ProcessesVirtual Processes
2 virtual simulations, cloned on process A2 virtual simulations, cloned on process A
A2 B2C22
VA1 VB1 VC1
VA2 VB2 VC2
PA2 PB1 PC1PA1
A1 B1C11
Virtual simulation planesVirtual simulation planes Mapping to phys. processes
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2020/39/39
Virtual ProcessesVirtual Processes CommunicationCommunication
• B -> C: message to PB -> C: message to PCC and thus to V and thus to VC1C1 and Vand VC2C2
• B -> A: message to PB -> A: message to PA1A1 (V (VA1A1) and copied ) and copied to Pto PA2A2 (V (VA2A2))
• A -> B (A1 to B1): PA -> B (A1 to B1): PB2B2 created and created and message sent to Pmessage sent to PB1B1 (V (VB1B1))
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2121/39/39
CloningCloning 3 phases: Both simulations identical; 3 phases: Both simulations identical;
Incremental cloning; Both Incremental cloning; Both simulations fully clonedsimulations fully cloned
Cloning rate depends on the amount Cloning rate depends on the amount of communication of communication
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2222/39/39
Multiple Path ExplorationMultiple Path Explorationsummarysummary
Reexecution more interactive, cloning Reexecution more interactive, cloning faster (offers more parallelism)faster (offers more parallelism)
Reexecution - limited rollback length; Reexecution - limited rollback length; cloning – all paths must be known before cloning – all paths must be known before the branchthe branch
Open issuesOpen issues• Greater rollback length, merging convergent Greater rollback length, merging convergent
sim. paths, create new branch point in the past sim. paths, create new branch point in the past and keep the current path, …and keep the current path, …
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2323/39/39
Input / OutputInput / Output Purpose: To provide accurate and Purpose: To provide accurate and
fast I/O to distributed simulatorsfast I/O to distributed simulators Several classes of I/O events:Several classes of I/O events:
• Blocking X non-blockingBlocking X non-blocking• Various types of access to the Various types of access to the
simulation statesimulation state Speed-up: optimistic outputSpeed-up: optimistic output
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2424/39/39
I/O EventsI/O Events 3 classes of output events:3 classes of output events:
• Sampling: provide a subset of state information Sampling: provide a subset of state information in regular intervalsin regular intervals
• Notification: occur in predefined transient Notification: occur in predefined transient conditioncondition
• Queries: require access to arbitrary state in Queries: require access to arbitrary state in previously unknown timepreviously unknown time
Input:Input:• Events injected by external entitiesEvents injected by external entities• Blocking requests for input scheduled by the Blocking requests for input scheduled by the
simulatorsimulator
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2525/39/39
Optimistic OutputOptimistic Output Output is often visually rich and Output is often visually rich and
computation intensivecomputation intensive Speedup:Speedup:
• Output is non-blocking eventOutput is non-blocking event• It’s optimistically precomputed and It’s optimistically precomputed and
thrown away in case of rollbackthrown away in case of rollback• Good for faster-than-realtime simulators Good for faster-than-realtime simulators
and computationally intensive outputand computationally intensive output
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2626/39/39
Input / OutputInput / Outputconclusionconclusion
Open issuesOpen issues• Queries in distributed simulation, over a Queries in distributed simulation, over a
wide time rangewide time range• Optimistic output:Optimistic output:
Limiting the optimism to save resourcesLimiting the optimism to save resources Relationship between optimistic output and Relationship between optimistic output and
the probability of inputthe probability of input
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2727/39/39
Variable Resolution ModelingVariable Resolution Modeling Why use VRM?Why use VRM?
• Speed-up – execute the focused portion Speed-up – execute the focused portion at full detail and the other at a lower d.at full detail and the other at a lower d.
• Zooming – overview the whole and focus Zooming – overview the whole and focus on details to better comprehend the on details to better comprehend the simulated phenomenonsimulated phenomenon
• Whole-greater-than-sum-of-parts Whole-greater-than-sum-of-parts relationshiprelationship
• Combine different legacy models with Combine different legacy models with diff. detail -> heterogeneous sim. diff. detail -> heterogeneous sim.
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2828/39/39
Variable Resolution ModelingVariable Resolution Modelingapproachesapproaches
Selective viewingSelective viewing• Simulate only the most detailed model Simulate only the most detailed model • Calculate the other views from the most Calculate the other views from the most
detailed modeldetailed model Aggregation/DisaggregationAggregation/Disaggregation
• One level is executed at one timeOne level is executed at one time• When needed, low resolution entity is When needed, low resolution entity is
replaced with a set of high res. entities replaced with a set of high res. entities (disaggregation) and vice versa(disaggregation) and vice versa
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 2929/39/39
Variable Resolution ModelingVariable Resolution Modelingapproachesapproaches
Concurrent representationConcurrent representation• All the entities are in the model All the entities are in the model
concurrentlyconcurrently• Only the selected detail level is Only the selected detail level is
executedexecuted• Other detail levels maintain consistency Other detail levels maintain consistency
by transforming the results from the by transforming the results from the running entitiesrunning entities
Existing methods use mix of these Existing methods use mix of these approachesapproaches
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3030/39/39
Multi Representation EntitiesMulti Representation Entities Single MRE has Single MRE has
multiple detail levelsmultiple detail levels Communication with Communication with
other entities at other entities at appropriate levelappropriate level
Consistency Consistency maintained by C.E. maintained by C.E. which has a set of which has a set of mapping functions to mapping functions to convert the state convert the state valuesvalues
MRE E1
A
B1 B1 B1
Consistencyenforcer
E2
E3
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3131/39/39
Hooking TransactionsHooking Transactions Model designed at multiple detail Model designed at multiple detail
levelslevels At the run time, the channel leading At the run time, the channel leading
to the LRE can be redirected to a to the LRE can be redirected to a HRE to increase the detail (zoom in)HRE to increase the detail (zoom in)
The disaggregated LRE can continue The disaggregated LRE can continue or is stripped completelyor is stripped completely
Simulation language primitives exist Simulation language primitives exist to be used by modelersto be used by modelers
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3232/39/39
Hooking TransactionsHooking Transactions
LRE 1 LRE 2 LRE 3
HRE 1
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3333/39/39
Hooking TransactionsHooking Transactions Consistency: Attribute ghostingConsistency: Attribute ghosting
• Attributes of the striped entity are Attributes of the striped entity are calculated from the newly hooked entitycalculated from the newly hooked entity
Mapping consistencyMapping consistency• The state of the HRE on disaggregation The state of the HRE on disaggregation
must be properly initialized from the must be properly initialized from the LRE’s state (and vice versa) LRE’s state (and vice versa)
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3434/39/39
Aggregation / DisaggregationAggregation / Disaggregation Used often in the HLA areaUsed often in the HLA area Low resolution entities (aircraft patrols) Low resolution entities (aircraft patrols)
automatically disaggregate to a set of high automatically disaggregate to a set of high resolution entities (single aircrafts) when resolution entities (single aircrafts) when approaching other HREs (ground radars)approaching other HREs (ground radars)
HLA is enriched with a set of functions that HLA is enriched with a set of functions that enable the registration of HREs and LREs, enable the registration of HREs and LREs, together with transformation functions and together with transformation functions and disaggregation conditionsdisaggregation conditions
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3535/39/39
Aggregation / DisaggregationAggregation / Disaggregation
LRE 1 LRE 3LRE 2HRE2
HRE1
HRE3
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3636/39/39
Other ApproachesOther Approaches Optimization based MRMOptimization based MRM
• Models at different resolution level Models at different resolution level provide a set of inputsprovide a set of inputs
• Mathematical function describing Mathematical function describing differences between the outputs is differences between the outputs is defineddefined
• Using mathematical analysis the inputs Using mathematical analysis the inputs are calculated so the differences are calculated so the differences between the outputs are minimizedbetween the outputs are minimized
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3737/39/39
Variable Resolution ModelingVariable Resolution Modelingsummarysummary
VRM – rich area with variety of possible VRM – rich area with variety of possible applicationsapplications
Hooking Hooking • Highly interactiveHighly interactive• Currently only for sequential simulationCurrently only for sequential simulation
MREMRE• Solid designSolid design• Not widely used (high complexity and demands on Not widely used (high complexity and demands on
modelers)modelers) Aggregation/DisaggregationAggregation/Disaggregation
• Oldest approach, simpleOldest approach, simple• Good for real-time simulation area (HLA) Good for real-time simulation area (HLA)
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3838/39/39
Current StateCurrent Stateand future workand future work
Prof. Prof. ŠafaříkŠafařík - 2 diploma theses to - 2 diploma theses to incorporate optimistic and incorporate optimistic and conservative synchronization to J-Sim conservative synchronization to J-Sim ((C-Sim) C-Sim)
Base on this and incorporate the Base on this and incorporate the interactivity as shown in this interactivity as shown in this presentation, trying to overcome the presentation, trying to overcome the shortcomings of the presented shortcomings of the presented approachesapproaches
24.11.200324.11.2003 Interactive Parallel and Distributed SimulatInteractive Parallel and Distributed Simulation - DSS - Jan Klírion - DSS - Jan Klír 3939
The EndThe EndQuestions ???Questions ???
Some pictures were reproduced from the Parallel and Distributed Simulation Some pictures were reproduced from the Parallel and Distributed Simulation course by M. Hybinette, PADS presentations by R. Fujimoto, and from various course by M. Hybinette, PADS presentations by R. Fujimoto, and from various papers cited in the papers cited in the Methods in Interactive Parallel and Distributed Simulation Methods in Interactive Parallel and Distributed Simulation
paper paper