Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web...
-
Upload
johnathan-cain -
Category
Documents
-
view
213 -
download
0
Transcript of Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web...
![Page 1: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/1.jpg)
Simulation Tutorial
By Bing Wang Assistant professor,
CSE Department, University of Connecticut Web site
![Page 2: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/2.jpg)
Network Simulation
Motivation: learn fundamentals
of evaluating network performance via simulation
Overview: fundamentals of
discrete event simulation
analyzing simulation outputs
ns-2 simulation
![Page 3: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/3.jpg)
The evaluation spectrum
numericalmodels
simulation
emulation
prototype
operational system
![Page 4: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/4.jpg)
What is simulation?
system under study(has deterministic rules governing its behavior)
exogenous inputsto system
(the environment)
system boundary
observer
“real” life
computer programsimulates deterministic rules governing behavior
pseudo random inputsto system
(models environment)
program boundary
observer
“simulated” life
![Page 5: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/5.jpg)
Why Simulation?
goal: study system performance, operation real-system not available, is complex/costly or
dangerous (eg: space simulations, flight simulations)
quickly evaluate design alternatives (eg: different system configurations)
evaluate complex functions for which closed form formulas or numerical techniques not available
![Page 6: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/6.jpg)
Simulation: advantages/drawbacks
advantages:
drawbacks/dangers:
![Page 7: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/7.jpg)
Programming a simulationWhat ‘s in a simulation program? simulated time: internal (to simulation program)
variable that keeps track of simulated time system “state”: variables maintained by
simulation program define system “state” e.g., may track number (possibly order) of packets in
queue, current value of retransmission timer events: points in time when system changes state
each event has associated event time• e.g., arrival of packet to queue, departure from
queue• precisely at these points in time that simulation must
take action (change state and may cause new future events)
model for time between events (probabilistic) caused by external environment
![Page 8: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/8.jpg)
Discrete Event Simulation
simulation program maintains and updates list of future events: event list
simulator structure: initialize event list
get next (nearest future)event from event list
time = event time
process event(change state values, add/delete
future events from event list
update statistics
done?n
Need: well defined set of
events for each event:
simulated system action, updating of event list
y
![Page 9: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/9.jpg)
Simulation: example
packets arrive (avg. interrarrival time: 1/ ) to router (avg. execution time 1/) with two outgoing links
arriving packet joins link i with probability i
state of system: size of each queue system events:
job arrivals service time completions
define performance measure to be gathered
![Page 10: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/10.jpg)
Simulation: example
Simulator actions on arrival event choose a link
if link idle {place pkt in service, determine service time (random number drawn from service time distribution) add future event onto event list for pkt transfer completion, set number of pkts in queue to 1}
if buffer full {increment # dropped packets, ignore arrival}
else increment number in queue where queued
create event for next arrival (generate interarrival time) stick event on event list
![Page 11: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/11.jpg)
Simulation: example
Simulator actions on departure event remove event, update simulation time,
update performance statistics decrement counter of number of pkts in
queue If (number of jobs in queue > 0) put next pkt
into service – schedule completion event (generate service time for put)
![Page 12: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/12.jpg)
Gathering Performance Statistics
avg delay at queue i: record Dij : delay of customer j at queue i. Let Ni be # customers passing through queue i
iii TN average queue length at i:
iNtotal simulated time
throughput at queue i, i =
i
N
jij
i N
D
T
i
1
Little’s Law
![Page 13: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/13.jpg)
Analyzing Output Results
Each time we run a simulation, (using different random number streams), we will get different output results!
distribution of random numbersto be used during simulation(interarrival, service times)
random number sequence 1 simulation output results 1input output
random number sequence 2 simulation output results 2input output
random number sequence M simulation output results Minput output
… … … … … …
![Page 14: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/14.jpg)
Analyzing Output Results
W2,n: delay of nth departing customer from queue 2
![Page 15: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/15.jpg)
Analyzing Output Results
each run shows variation in customer delay
one run different from next
statistical characterization of delay must be made expected delay of n-
th customer behavior as n
approaches infinity average of n
customers
![Page 16: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/16.jpg)
Transient Behavior
simulation outputs that depend on initial condition (i.e., output value changes when initial conditions change) are called transient characteristics “early” part of simulation later part of simulation less dependent on initial
conditions
![Page 17: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/17.jpg)
Effect of initial conditions histogram of delay of 20th
customer, given initially empty (1000 runs)
histogram of delay of 20th customer, given non-empty conditions (1000 runs)
![Page 18: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/18.jpg)
Simulation: example
packets arrive (avg. interrarrival time: 1/ ) to router (avg. execution time 1/) with two outgoing links
arriving packet joins link i with probability i
![Page 19: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/19.jpg)
Steady state behavior output results may converge to limiting “steady state”
value if simulation run “long enough”
discard statistics gathered during transient phase, e.g., ignore first n0 measurements of delay at queue 2
avg delayof packets
[n, n+10]
0
0
nN
D
Ti
N
njij
i
i
pick n0 so statistic is “approximately the same” for different random number streams and remains same as n increases
avg of 5 simulations
![Page 20: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/20.jpg)
![Page 21: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/21.jpg)
![Page 22: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/22.jpg)
Example: Random Waypoint ModelSimplest random waypoint model: mobile picks next waypoint Mn uniformly in area,
independent of past and present mobile picks next speed Vn uniformly in [vmin;
vmax] independent of past and present
mobile moves towards Mn at constant speed Vn
Mn
Mn+1
![Page 23: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/23.jpg)
Issue with RWP Model: Decay
Distributions of node speed, position, distances, etc change with time
100 users average
1 user
Time (s)
Spe
ed (
m/s
)
![Page 24: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/24.jpg)
Confidence Intervals
run simulation: get estimate X1 as estimate of performance metrics of interest
repeat simulation M times (each with new set of random numbers), get X2, … XM – all different!
which of X1, … XM is “right”?
intuitively, average of M samples should be “better” than choosing any one of M samples
M
X
X
M
jj
1
How “confident”are we in X?
![Page 25: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/25.jpg)
Confidence Intervals
cannot get perfect estimate of true mean, , with finite # samples
look for bounds: find c1 and c2 such that:Probability(c1 < < c2) = 1 –
c1,c2confidence interval100(1-): confidence level
![Page 26: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/26.jpg)
Confidence Intervals: Central Limit Thm
Central Limit Theorem: If samples X1, … XM independent and from same population with population mean and standard deviation then
M
X
X
M
jj
1
is approximately normally distributed with mean u and standard deviation
sample mean:
M
![Page 27: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/27.jpg)
Confidence Intervals .. more
don’t know population standard deviation; estimate it using sample (observed) standard deviation:
M
mmX XX
M 1
22 )(1
1
given we find upper and lower tails of normal distributions containing 100% of mass
2, XX
![Page 28: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/28.jpg)
Confidence Intervals .. the recipe
M
mmX XX
M 1
22 )(1
1
Given samples X1, …, XM, (e.g., having repeated simulation M times), compute
M
X
X
M
jj
1
95% confidence interval:M
X X96.1
![Page 29: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/29.jpg)
Interpretation of Confidence Interval
If we calculate confidence intervals as in recipe, 95% of confidence intervals thus computed will contain true (unknown) population mean.
![Page 30: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/30.jpg)
Generating confidence intervals forsteady state measures independent replications with deletions method of batch means autoregressive method regenerative method
![Page 31: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/31.jpg)
Independent replications
1. generate n independent replications with m samples, remove first l0 samples from each to obtain
2. calculate sample mean and variance from 3. use t-distribution to compute confidence
intervals4. can combine with sequential stopping rule to
obtain confidence interval of specified width.
),(,),,( 001 lmXlmX n ),( 0lmX i
![Page 32: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/32.jpg)
0ther methods
batch means: take single run, delete first l0 observations, divide remainder into n groups and obtain Xi for i-th, i = 1,…,n follow procedure for independent replications complication due to nonindependence of Xis
potential efficiency due to deletion of only l0 observation
autoregressive method: spectrum analysis: based on study of correlation of observations
![Page 33: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/33.jpg)
regenerative method: applicable to systems with regeneration points regeneration point future independent of
past can construct observations for intervals
between regeneration points that will be iid use of CLT provides confidence intervals
![Page 34: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/34.jpg)
Comparing two different systems
Example: want to compare mean response times of two queues where arrival process remains unchanged but speed of servers are different.
run each system n times (n sufficiently large) to get {X1,j} and {X2,j} and take Zj = (X1,j ,X2,j) as the observations to determine confidence intervals for
method of common random number using the same streams to generate rvs for j-th runs
of both systems usually results in smaller sample variance of {Zj}
![Page 35: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/35.jpg)
ns-2, the network simulator
discrete event simulator modeling network
protocols wired, wireless, satellite TCP, UDP, multicast,
unicast web, telnet, ftp ad hoc, sensor nets infrastructure: stats,
tracing, error models, etc.
prepackaged protocols and modules, or create your own
Our goal: flavor of ns: simple
example, modification, execution and trace analysis
![Page 36: Simulation Tutorial By Bing Wang Assistant professor, CSE Department, University of Connecticut Web site.](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f155503460f94c2b672/html5/thumbnails/36.jpg)
“ns” components
ns, the simulator itself (this is all we’ll have time for)
nam, the Network AniMator visualize ns (or other) output GUI input simple ns scenarios
pre-processing: traffic and topology generators
post-processing: simple trace analysis, often in Awk, Perl, or Tcl
tutorial: http://www.isi.edu/nsnam/ns/tutorial/index.html ns by example: http://nile.wpi.edu/NS/