Digital Media Technology Week 6. XML Source XSLT Stylesheet XML Result XSLT.
Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML...
Transcript of Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML...
![Page 1: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/1.jpg)
G.Casale – G.Serazzi 1
Quantitative System Evaluation with Java Modelling Tools
Giuliano Casale Giuseppe Serazzi
Politecnico di Milano Dip. Elettronica e Informazione Milan, Italy
Imperial College London [email protected]
Politecnico di Milano [email protected]
.NET Working Group - BCAM
![Page 2: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/2.jpg)
G.Casale – G.Serazzi 2
tutorial outline
overview of Java Modelling Tools (http://jmt.sf.net)
case study 1 (CS1): bottlenecks identification, performance evaluation, optimal load
case study 2 (CS2): model with multiple exit paths
case study 3 (CS3): resource contention
case study 4 (CS4): multi-tier applications, web services
![Page 3: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/3.jpg)
G.Casale – G.Serazzi 3
Java Modelling Tools (http://jmt.sf.net)
CS4
CS4
CS1
CS1
CS2
CS3
![Page 4: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/4.jpg)
G.Casale – G.Serazzi 4
architecture
XML
jSIMengine
JAVA/JWAT/JMVA JSIMwiz JSIMgraph
XML XSLT
XSLT
Status
Update
“Views”
“Model”
“Controller”
JMT framework
![Page 5: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/5.jpg)
G.Casale – G.Serazzi 5
software development
JMT is open source, Java code and ANT build scripts at http://jmt.sourceforge.net/Download.html
size: ~4,000 classes; 21MB code; 174,805 lines
subversion svn co https://jmt.svn.sourceforge.net/svnroot/jmt jmt
source tree trunk (root also for help, examples, license information, ...)
src jmt
analytical (jMVA algorithms)
commandline (command line wrappers)
common (shared utilities)
engine (main algorithms & data structures)
framework (misc utilities)
gui (graphical user interfaces) jmarkov (JMCH)
test (application testing)
![Page 6: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/6.jpg)
G.Casale – G.Serazzi 6
core algorithms - jMVA
Mean Value Analysis (MVA) algorithm (e.g., [Lazowska et al., 1984])
fast solution of product-form queueing networks
open models: efficient solution in all cases
closed models: efficient for models with up to 4-5 classes
Product-form queueing networks solvable by MVA
PS/FCFS/LCFS/IS scheduling
Identical mean service times for multiclass FCFS
Mixed models (open + closed), load-dependent
Service at a queue does not depend on state of other queues
No blocking, finite buffers, priorities
Some theoretical extensions exist, not implemented in jMVA
![Page 7: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/7.jpg)
G.Casale – G.Serazzi 7
core algorithms – jSIMengine: simulation
components in the simulation are defined by 3 sections
discrete-event simulation engine
external arrivals
(open class)
queueing station component sections
admit
serve
complete
route
![Page 8: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/8.jpg)
transient filtering flowchart
G.Casale – G.Serazzi 8
core algorithms – jSIMengine: statistical analysis
[Heidelberger&Welch, CACM, 1981] [Pawlikowski, CSUR, 1990]
[Spratt, M.S. Thesis, 1998]
Transient
(Steady State)
![Page 9: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/9.jpg)
G.Casale – G.Serazzi
9
core algorithms – jSIMengine: simulation stop
simulation stops automatically
confidence level
maximum
relative error
traditional control
parameters
9
![Page 10: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/10.jpg)
CASE STUDY 1: Bottlenecks identification Performance evaluation
Optimal load
closed model multiclass workload
JABA + JMVA
Politecnico di Milano Dip. Elettronica e Informazione Milan, Italy
G.Casale – G.Serazzi 10
![Page 11: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/11.jpg)
11
Outline
objectives
system topology
bottlenecks detection and common saturation sectors
performance evaluation
optimal loading
G.Casale – G.Serazzi
![Page 12: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/12.jpg)
12
characteristics of the system
e-business services: a variety of activities, among them
information retrieval and display, data processing and updating
(mainly data intensive) are the most important ones
two classes of requests with different resource loads and
performance requirements
presentation tier: light load (less demanding than that of the
other two tiers)
application tier: business logic computations
data tier: store and fetch DB data (search, upload, download)
to reduce the number of parameters (and to simplify obtaining
their values) we have choosen to parameterize the model in
term of global loads Li, i.e., service demands Di
G.Casale – G.Serazzi
![Page 13: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/13.jpg)
13
topology of a 3-tier enterprise system
Application
Servers
Storage
Servers
Web
Server
workload 2
workload 1
Internet
...
clients 3-tier e-business system
Application Servers Storage ServersWeb Server
presentation tier business tier data tier
workload 2
workload 1 closed model
N customers
2 classes
G.Casale – G.Serazzi
![Page 14: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/14.jpg)
14
workload parameters
resource Loadings matrix: Service Demands, i resources, r classes Dir = Vir * Sir
global number of customers: N=100
system population: N={N1,N2} {1,99}→{99,1}
population mix: β={β1,β2}, fraction of jobs per class,
β variable: study of the optimal load (optimal mix)
asymptotic behavior: β constant, N increasing
G.Casale – G.Serazzi
![Page 15: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/15.jpg)
15
Service Demands (resource Loadings)
natural bottleneck
of class 1
(Storage 2) natural bottleneck
of class 2
(Storage 1)
Storage 3:
potential system bottleneck
name of the model
G.Casale – G.Serazzi
![Page 16: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/16.jpg)
16
What-if analysis (JMVA with multiple executions)
fraction of
class 1 requests
number of models requested
(may be not all not executed)
parameter that changes
among different executions
G.Casale – G.Serazzi
![Page 17: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/17.jpg)
17
Bottlenecks switching (JABA asymptotic analysis)
global loadings of class 1
global loadings of class 2
bottlenecks
fraction of class 2 jobs that
saturate two resources concurrently
(Common Saturation Sector)
bottlenecks
G.Casale – G.Serazzi
![Page 18: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/18.jpg)
18
throughput and Response time {N=1,99}-{99,1}, JMVA
class 1
class 2
system
Common
Saturation
Sector class 1
class 2
system
Common
Saturation
Sector
throughput X Response times
equiload
0.0181 r/ms
0.48
5.5 ms
G.Casale – G.Serazzi
![Page 19: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/19.jpg)
19
Utilizations and Power {N=1,99}–{99,1}
Common
Saturation
Sector
Storage 3
Storage 1
Storage 2
Utilizations Power (X/R)
class 1
class 2
system
best QoS
to class 1
best QoS
to class 2
G.Casale – G.Serazzi
![Page 20: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/20.jpg)
20
optimized load: service demands and bottlenecks
multiple bottlenecks
equi-utilization line
2
Class 1
94.5
94.5
95
G.Casale – G.Serazzi
![Page 21: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/21.jpg)
21
optimized load: U and X
equi-utilization
mix
Storage 1
Storage 2
Storage 3
Utilizations throughput X
class 2
class 1
system 0.0209 r/ms
0.48
G.Casale – G.Serazzi
![Page 22: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/22.jpg)
22
optimized load: Response times and Residence times
Response times
system
system
class 1
class 2
Common
Saturation
Sector
Storage 3
Storage 1
Storage 2
Residence times
4.78 ms
0.48
4.78 ms
0.48
G.Casale – G.Serazzi
![Page 23: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/23.jpg)
CASE STUDY 2: model with multiple exit paths
open model
single class workload different routing policies
JSIMgraph
Politecnico di Milano Dip. Elettronica e Informazione Milan, Italy
G.Casale – G.Serazzi 23
![Page 24: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/24.jpg)
24
Outline
objectives
system topology
what-if analysis
performance with “probabilistic” routing
performance with “least utilization” routing
performance with “Joint the Shortest Queue” routing
G.Casale – G.Serazzi
![Page 25: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/25.jpg)
25
objectives
fallacies in using the index system response time also in single class models
open model with multiple exit paths (sinks), e.g., drops,
alternative processing, multi-core, load balancing, clouds, ...
differencies between response time per sink and system res
ponse time
impact on performance of different routing policies
G.Casale – G.Serazzi
![Page 26: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/26.jpg)
26 Casale - Serazzi
system topology
source of requests
selection of the
routing policy
λ = 1 req/s
S = 0.3 sec
S = 1 sec
S = 0.2 sec
exponential distributions
0.5
0.5
utilizations
path 2
path 1
![Page 27: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/27.jpg)
27
What-if analysis settings
number of models
requested
final arrival rate
initial arrival rate
control parameter
enable the
what-if analysis
G.Casale – G.Serazzi
![Page 28: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/28.jpg)
28
n. of customers N in the two paths (prob. routing)
mean N = 9.13 j mean N = 0.37 j
path 1 path 2
G.Casale – G.Serazzi
![Page 29: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/29.jpg)
29
Utilizations (per path) with prob. routing
path 1 path 2
U = 0.89
U = 0.27
G.Casale – G.Serazzi
![Page 30: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/30.jpg)
30
system Response time (prob. routing)
mean R = 5.51 s
perf. indices collected
no requested precision
number of models
executed
in this run (What-if)
![Page 31: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/31.jpg)
31
Response time per path (prob. routing)
mean R = 0.72 s
path 1 path 2
mean R = 10.38 s
system response time R = 5.5 sec
G.Casale – G.Serazzi
![Page 32: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/32.jpg)
32
Utilizations with “least utilization” routing
path 1 path 2
U = 0.41
U = 0.41
utilizations well balanced
G.Casale – G.Serazzi
![Page 33: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/33.jpg)
33
Response times with “least utilization” routing
path 1 path 2
R = 3.55 sec
R = 0.88 sec
system response time R = 1.5 sec
G.Casale – G.Serazzi
![Page 34: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/34.jpg)
34
Utilizations with “Joint the Shortest Queue” routing
path 1 path 2
U = 0.61
U = 0.35
G.Casale – G.Serazzi
![Page 35: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/35.jpg)
35
N of customers with JSQ routing
path 1 path 2
N = 0.88
N = 0.47
G.Casale – G.Serazzi
![Page 36: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/36.jpg)
36
Response times with JSQ routing
path 1 path 2
R = 1.72 sec
R = 0.70 sec
system response time R = 1.05 sec
G.Casale – G.Serazzi
![Page 37: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/37.jpg)
G.Casale – G.Serazzi 37
CASE STUDY 3 Resource Contention
(use of Finite Capacity Regions - FCR)
contention of components
hardware: I/O devices, memory, servers, ... software: threads, locks, semaphores, ...
bandwidth
open model single class workload
JSIMgraph
Politecnico di Milano Dip. Elettronica e Informazione Milan, Italy
![Page 38: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/38.jpg)
G.Casale – G.Serazzi 38
modeling contention
fixed number of hw/sw components (threads, db locks, semaphores, ...)
clients compete for the available component free
request execution time: wait time for the next free component + wait time for the hardware resources (CPU, I/O, ...) + execution time
request interarrival times exponentially distributed
payload of different sizes (exponentially distributed)
evaluate the execution time of requests when the number of clients ranges from 1 to 20 and the number of components ranges from 1 to 10 (∞), evaluate the drop rate and the wait time in queue for the next available component
implement several models with different level of completeness
![Page 39: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/39.jpg)
G.Casale – G.Serazzi 39
threads (resource hw/sw) contention (simple model)
server
...
sink
threads = 1÷∞
clients
thread requests queue
(inside the server)
...
λ=1÷20 r/s
CPU I/O
DCPU=0.010s
DI/O=0.047s
![Page 40: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/40.jpg)
G.Casale – G.Serazzi 40
model definition (unlimited threads and queue size)
λ = 1 ÷ 20 req/sec
source of requests queue resource
sink
name of the model
fraction of
capacity used
selection of perf.indices
simulation results
fraction of
n.o of requests
![Page 41: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/41.jpg)
G.Casale – G.Serazzi 41
input parameters (service demands)
mean service time = 0.010 s
mean service time = 0.047 s
![Page 42: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/42.jpg)
G.Casale – G.Serazzi 42
system Response time (λ=20 req/sec)
confidence interval
transient duration
the number of
samples analyzed is
greater than the
max defined here
perf.indexes selected
default values
of parameters
actual sim. parameters
![Page 43: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/43.jpg)
43
λ=1÷20 req/s, unlimited threads & queue size (JSIMgraph)
UI/O
= λDI/O
= 20*0.047
= 0.94 (exact)
Utilization of I/O
throughput
system Response time
same as λ
no limitations
R = 0.784 s (sim) 0.931 (sim)
X = 19.86 r/s
system Power
R = 0.795 s (exact)
G.Casale – G.Serazzi
![Page 44: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/44.jpg)
G.Casale – G.Serazzi 44
Number of requests (unlimited threads & queue size)
0.25 req. 15.39 req
N = 15.64 req (sim)
N = XR = 15.91 req (exact)
![Page 45: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/45.jpg)
G.Casale – G.Serazzi 45
set of a Finite Capacity Region – FCR
step 1 – select the components
of the FCR
step 2 – set the FCR
region with constrained
number of customers
drop
queue
![Page 46: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/46.jpg)
G.Casale – G.Serazzi 46
FCR parameters
global capacity of the FCR
max number of requests
per class in the FCR
drop the requests when the region
capacity is reached
(for both the constraints)
![Page 47: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/47.jpg)
G.Casale – G.Serazzi 47
system Number of requests (limited n. threads and drop)
5 threads
unlimited
10 threads
15 threads
![Page 48: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/48.jpg)
G.Casale – G.Serazzi 48
Utilization of I/O server (limited n. threads and drop)
10 threads
unlimited 15 threads
5 threads
![Page 49: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/49.jpg)
G.Casale – G.Serazzi 49
system Response time (limited n. threads and drop)
5 threads 10 threads
unlimited 15 threads
![Page 50: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/50.jpg)
G.Casale – G.Serazzi 50
external finite queue for limited threads
server ..
.
sink
threads = 5
clients
queue for threads with finite capacity
(outside the server)
λ=20 r/s
server
Dserver=0.047s
Blocking After
Service policy
queue
drop policy
the queue for threads is limited (e.g., to limit the number of connections in case of denial of service attack, to guarantee a negotiated response time for the accepted requests, ...)
the requests arriving when the queue is full are rejected (drop policy)
the number of threads is limited and the requests are queued in a resource different from the server (load balancer, firewall, ...)
evaluate the combination of different admission policies
![Page 51: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/51.jpg)
G.Casale – G.Serazzi 51
set Block After Service (BAS) blocking policy
max number of requests
in the station
station with finite capacity
selection of the
BAS policy
BAS policy:
requests are blocked in the
sender station when the max
capacity of the receiver
is reached
![Page 52: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/52.jpg)
G.Casale – G.Serazzi 52
λ=20 req/s N R U X Drop Queue and Server
stations
Qsize= ∞ Q
Ser=5, queue S
0
16.11
0
0.77
0
0.95 20.06 0
Qsize= ∞ Q
Ser=5, BAS S
11.03
4.77
0.53
0.24
0
0.923 19.82 0
Qsize=5 drop Q
Ser=5, BAS S
0.94
3.82
0.05
0.20
0
0.88 18.76 1.14
Qsize= ∞ Q
Ser=5, drop S
0
2.34
0
0.136
0
0.812 17.16 2.866
Server Queue
∞ 5 ∞
Server Queue
∞ 5
BAS
Server Queue
5 5
BAS
drop
Server Queue
∞ 5 drop
different admission policies for Queue and Server
![Page 53: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/53.jpg)
G.Casale – G.Serazzi 53
CASE STUDY 4
Multi-Tier Applications and Web Services
(Worker Threads, Workflows, Logging, Distributions)
closed models single class and multiclass workloads
fork-join
JSIMgraph+JWAT
Politecnico di Milano Dip. Elettronica e Informazione Milan, Italy
![Page 54: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/54.jpg)
G.Casale – G.Serazzi 54
performance evaluation of a multi-tier application
multi-tier application serves a transactional workload which requires processing by an application server (AS) and by a database (DB)
the AS serves requests using a fixed set of worker threads
requests waiting for a worker thread are queued by the admission control system
utilization measurements available for the AS and for the DB
– know both for AS and DB the average service time S
– e.g., linear regression estimate U=SX+Y, U = utilization, X = throughput, Y =noise
evaluate response time for increasing worker threads
![Page 55: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/55.jpg)
G.Casale – G.Serazzi 55
transaction lifecycle
Worker thread admission time
Service time (1)
Queueing time
DB query time (1)
Service time (2)
Service time (3)
DB query time (2)
Server Response time
Network latency (1)
Network latency (2)
Client-Side Application Server
Request Response time
Request arrives
Response arrives
Admission control
Load context in memory
Data access
Data access
CPU
CPU
CPU
DB Server
Worker Thread
Simultaneous
Resource Possession
![Page 56: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/56.jpg)
G.Casale – G.Serazzi 56
modelling abstraction (easier to define and study)
Server admission time
Service time (1)
Queueing time
DB query time (1)
Service time (2)
Service time (...)
DB query time (2)
Server Response time
Network latency (1)
Network latency (2)
Client-Side Server-Side
Request Response time
Request arrives
Response arrives
Admission control
Load context in memory
Data access
Data access
CPU
CPU+I/O
CPU+I/O
Application
Server
Steps
DB Server
Steps
Worker Thread
![Page 57: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/57.jpg)
G.Casale – G.Serazzi 57
modelling multi-tier applications
Exponential
Distributions
Scpu = 0.072s Sdb = 0.032s
Zload = 0.015s
FCR Admission
Policy
FCR Capacity
FCR
4 Servers (Cores)
FCR Admission
Queue is Hidden !
PS scheduling
N=300
app users
send to jMVA
simulate
![Page 58: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/58.jpg)
G.Casale – G.Serazzi 58
simulation vs jMVA model
FCR not included in
product-form model
![Page 59: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/59.jpg)
G.Casale – G.Serazzi 59
SAP Business Suite [Li, Casale, Ellahi; ICPE 2010]
M MVA M M S
S
SIM
REAL
R
R
R
S
Quad-Core Server
N=300 users
Response Time
![Page 60: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/60.jpg)
G.Casale – G.Serazzi 60
what-if analysis – adding a web service class
some requests now access the service composition engine of the multi-tier application to create a business travel plan
services are composed on the fly from external providers (travel agencies, flight booking service) according to a workflow
worker thread remains busy for the entire duration of the web service workflow
evaluate end-to-end response time for each class
![Page 61: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/61.jpg)
G.Casale – G.Serazzi 61
business trip planning (BTP) web service
FCR Class-Based
Admission
N=300 app users
Nbtp=50 BTP users
pBTP=1.0
Sbtp =?, Exp?
![Page 62: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/62.jpg)
G.Casale – G.Serazzi 62
BTP web service sub-model
Logger
S0=?, Exp?
Zsce=0.025s, Exp
N=1 WS instance S1=?, Exp?
S2=?, Exp?
![Page 63: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/63.jpg)
G.Casale – G.Serazzi 63
jWAT – Workload Analysis Tool
Specify Format
Column-Oriented
Log File
Load Data
Data Format
Templates
![Page 64: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/64.jpg)
G.Casale – G.Serazzi 64
Ignore Negative
Samples
jWAT – data filtering
![Page 65: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/65.jpg)
G.Casale – G.Serazzi 65
jWAT – descriptive statistics
Scatter plots
Histogram
c=std. dev. /mean
Hyper-Exp
(c >1)
![Page 66: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/66.jpg)
G.Casale – G.Serazzi 66
Outliers?
Scatter plot
jWAT – scatter plot
![Page 67: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/67.jpg)
G.Casale – G.Serazzi 67
BTP web service sub-model
log inter-arrival
times
Zsce=0.025s, Exp
N=1 WS instance
S2=0.911
HyperExp c=2.9081
S1=2.151,
HyperExp c=1.689
S0=0.967
HyperExp c=3.1434
![Page 68: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/68.jpg)
G.Casale – G.Serazzi 68
BTP response times
logarithmic
transformation
e.g., Weibull,
Lognormal.
Gamma
![Page 69: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/69.jpg)
G.Casale – G.Serazzi 69
response time distribution – logger components
Sbtp = 3.611s
Gamma c=1.44
timestamp, class id,
job id
timestamp, class id,
job id
global.csv
job id (same throughout
simulation)
job class
logger id
![Page 70: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/70.jpg)
G.Casale – G.Serazzi 70
response time distribution analysis
cumulative distribution
95th
percentile
[seconds]
cdf
(matlab)
![Page 71: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/71.jpg)
CONCLUSION
Politecnico di Milano Dip. Elettronica e Informazione Milan, Italy
71
![Page 72: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/72.jpg)
G.Casale – G.Serazzi 72
Final remarks
Analysis with Java Modelling Tools (http://jmt.sf.net)
– Queueing network simulation
– Bottlenecks identification
– Workload analysis
– Mean value analysis
– ...
JMT-Based examples and exercises (http://perflib.net)
Topics not covered by this tutorial
– jMCH
– Burstiness analysis
– Trace-driven simulation
– ...
JMT discussion forum: http://sourceforge.net/forum/?group_id=163838
![Page 73: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/73.jpg)
G.Casale – G.Serazzi 73
References
G.Casale, G.Serazzi. Quantitative System Evaluation with Java Modelling Tools (Tutorial). in Proc. of ACM/SPEC ICPE 2011 (companion paper).
M.Bertoli, G.Casale, G.Serazzi. User-Friendly Approach to Capacity Planning Studies with Java Modelling Tools, in Proc. of SIMUTOOLS 2009.
M.Bertoli, G.Casale, G.Serazzi. JMT - Performance Engineering Tools for System Modeling. ACM Perf. Eval. Rev., 36(4), 2009
M.Bertoli, G.Casale, G.Serazzi. The JMT Simulator for Performance Evaluation of Non Product-Form Queueing Networks, in Proc. of SCS Annual Simulation Symposium 2007, 3-10, Norfolk, VA, Mar 2007.
M.Bertoli, G.Casale, G.Serazzi. Java Modelling Tools: an Open Source Suite for Queueing Network Modelling and Workload Analysis, in Proc. of QEST 2006, 119-120, Sep 2006.
E.Lazowska, J.Zahorjan, G.S.Graham, K.C.Sevcik, Quantitative System Performance: Computer System Analysis Using Queueing Network Models, Prentice-Hall, 1994.
K.Pawlikowski: Steady-State Simulation of Queuing Processes: A Survey of Problems and Solutions. ACM Comput. Surv. 22(2): 123-170, 1990.
P.Heidelberger and P.D.Welch. A spectral method for confidence interval generation and run length control in simulations. Comm. ACM. 24, 233-245, 1981.
S.C.Spratt. Heuristics for the startup problem. M.S. Thesis, Department of Systems Engineering, University of Virginia, 1998.
![Page 74: Quantitative System Evaluation with Java Modelling ToolsG.Casale –G.Serazzi 4 architecture XML jSIMengine JAVA/JWAT/JMVA JSIMwiz JSIMgraph XSLT XML XSLT Status Update “Views”](https://reader036.fdocuments.us/reader036/viewer/2022081411/60b34625ce660a206e3d49e5/html5/thumbnails/74.jpg)
Contact us!
[email protected] [email protected]
Politecnico di Milano Dip. Elettronica e Informazione Milan, Italy
74