Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot...
Transcript of Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot...
![Page 1: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/1.jpg)
1 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Performance Prediction
Using RETIT & Dynatrace App Mon
Beyond Monitoring
Delete &
Insert
Headshot
here
Delete &
Insert
Headshot
here
![Page 2: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/2.jpg)
2 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE2
Agenda
Performance Prediction
Use cases, Benefits and examples
Demo
![Page 3: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/3.jpg)
3 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE3
Performance prediction
Take your Continuous delivery to the next level.
![Page 4: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/4.jpg)
4 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Performance Modeling - Why?
A new version of your application is on the way:
• … review realtime application performance.
• … evaluate realtime performance metrics.
• … deep dive into single components of your environment.
Enterprise ApplicationHardwareSoftware
Measurements Measurement Data
![Page 5: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/5.jpg)
6 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Performance Modeling - Why?
Capacity
Planning
Performance
Analysis
Enterprise ApplicationHardwareSoftware
Measurement Measurement Data
Capacity
Management
What happens if…
• … you change yur deployment topology?
• … you migrate to a different hardware environment?
• … the workload changes?
• … you reduce the number of CPU cores?
![Page 6: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/6.jpg)
7 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Performance Modeling - What?
Performance
Model
Resource
Demand
Software
Architecture
Hardware
Environment
Workload
Simulation
Response Time
Throughput
Resource
Utilization
![Page 7: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/7.jpg)
8 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
A Brief History of Performance Modeling
Queuing Theory(Erlang, ~1906)
Queuing Networks (Jackson, ~1963)
LayeredQueuing Networks (Neilson / Woodside, ~1995)
Architecture-Level Performance Models (Reussner / Becker ~2009)
w
t
S
λ
Arriving
Jobs
Departing
Jobs
Queue Service
Station
1
4
CPU
HDD
Application Server
Arriving
Transactions
Completed
Transactions
HTTPRequest
[Z=2 s]
User
{m=1200}
WebService
[s=15 ms]
HTTP Server
Thread
{m=50}
Purchase
[s=80 ms]
AppServer
Thread
{m=25}
Browse
[s=20 ms]
(y=1)
(y=0.6) (y=0.4)
User Device
{m=1200}
User Device
{m=1200}
HTTP Server
Machine
HTTP Server
Machine
AppServer
Machine
{m=2}
AppServer
Machine
{m=2}
Repository Model
Resource Environment
System Model
Allocation Model
Usage Model
Pa
llad
io C
om
po
nen
tM
od
el (P
CM
)
Foundation for tools such
as CA Capacity Manager
(was: Hyperformix) or
Sumarian Capacity Planner
Foundation for tools such
as Performance
Assurance‘s ePASA
Foundation for
RETIT solutions
![Page 8: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/8.jpg)
9 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Performance Modeling – How?
Knowledge
Generation
Capacity
Planning
Performance
Analysis
Enterprise ApplicationHardwareSoftware
Performance
Model Repository
Measurement
Modeling
Simulation
Measurement Data
Model Parameter
Simulation Results
Capacity
Management
![Page 9: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/9.jpg)
10 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Performance Modeling – When?
OpsDev
Implementation
Operations
1 4.5 10 27.5 50
500
0
100
200
300
400
500
600
Rel
aive
cost
fact
or
(mea
n)*
Fix performance-related bugs earlier
and for less costs by managing
performance knowledge using models.
Source: http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20100036670.pdf
*The cost factor for fixing a performance-related problem is normalized in
the different phases relative to the cost of fixing a defect in the requirements phase.
Extrapolate your (load/performance) test
results and reduce labor, machinery and
license costs by reducing the amount tests.
Detect performance change in every version
created in a continuous delivery pipeline without
the need for expensive performance tests.
Provide models along with your
application binaries to simplify
capacity planning activities.
Right-size your environment for seasonal peaks or to
reduce license / operating cost (e.g., reduce the
amount of cloud instances).
![Page 10: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/10.jpg)
11 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Dynatrace-Compatible RETIT Solutions
• RETIT Capacity Manager (RCM)
• Performance Modeling Environment
• Connect to any of your Dynatrace deployments to
generate performance models
• “What-If” scenario simulations:
• Hardware changes
• Workload changes
• Software architectural changes
![Page 11: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/11.jpg)
12 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Dynatrace-Compatible RETIT Solutions
• RETIT Continuous Delivery (RCD)
• Plugin for the Jenkins CI server
• Detects performance changes automatically
• Uses Dynatrace data collected in previous test phases
(e.g., acceptance tests) to generate models
• Can evaluate performance for multiple hardware
environments and workloads
![Page 12: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/12.jpg)
13 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE13
Agenda
Performance Modeling
Use cases, Benefits and examples
Demo
![Page 13: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/13.jpg)
14 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Use Cases – Design Time Performance Evaluations
Implementation
Operations
Existing
Component 1 Existing
Component …
Existing
Component N
New
Application
Design
Performance-
relevant aspects
known and depicted
in performance models
Structure and
behaviour
defined in design
models
e.g., minimum
response time –
even you assume
that a new
application
consumes no
time
![Page 14: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/14.jpg)
15 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Benefits – Design Time Performance Evaluations
1 4.5 1027.5
50
500
0
100
200
300
400
500
600
Requirements Design Implementation Testing Deployment Operations
Rel
ativ
e co
stfa
cto
r(m
ean
)*
Fix performance problems in your software architecture earlier with a lot less cost than during test, deployment or operations!
Source: http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20100036670.pdf
*The cost factor for fixing a performance-related problem is normalized in the
different phases relative to the cost of fixing a defect in the requirements phase.
Implementation
Operations
![Page 15: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/15.jpg)
16 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Real Example - Design Time Perf. Evaluations
• Evaluating Performance in a Service-oriented Architecture (SOA)
Implementation
Operations
• Service-Consumers:
• Process-oriented user interfaces
• Orchestrated by a BPM Engine
• Service-Providers:
• Common data sources and application services
Enterprise Service Bus (ESB)
Business
Dev
Ops
![Page 16: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/16.jpg)
17 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Real Example - Design Time Perf. Evaluations
• Evaluating Performance in a Service-oriented Architecture (SOA)
Implementation
Operations
• Service-Consumers:
• Process-oriented user interfaces
• Orchestrated by a BPM Engine
• Service-Providers:
• Common data sources and application services
• What happens if we introduce/automate new business processes?• Can you achieve the required response time and business process lead
time goals?• How much does it cost to increase the IT system performance to improve
the business process lead times?
Business
Dev
Ops
![Page 17: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/17.jpg)
18 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Real Example - Design Time Perf. Evaluations
• Evaluating Performance in a Service-oriented Architecture (SOA)
Implementation
Operations
• Service-Consumers:
• Process-oriented user interfaces
• Orchestrated by a BPM Engine
• Service-Providers:
• Common data sources and application services
Business
Dev
Ops
• Can we achieve the desired performance and process lead
time goals using our existing service-level agreements?
• Which service needs to improve most in order to achieve the
business goals?
• Should we negotiate new SLAs with multiple service proviers
and ask them to improve their performance slidely or with a
few providers but as for more radical changes?
![Page 18: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/18.jpg)
19 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Real Example - Design Time Perf. Evaluations
• Evaluating Performance in a Service-oriented Architecture (SOA)
Implementation
Operations
• Service-Consumers:
• Process-oriented user interfaces
• Orchestrated by a BPM Engine
• Service-Providers:
• Common data sources and application services
• How much additional load will new business processes generate? • Will the existing systems be able to handle the load?• Do we need to increase our capacity?• Who pays for the additional capacity?
Business
Dev
Ops
![Page 19: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/19.jpg)
20 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Real Example - Design Time Perf. Evaluations
• Evaluating Performance in a Service-oriented Architecture (SOA)
Implementation
Operations
• Service-Consumers:
• Process-oriented user interfaces
• Orchestrated by a BPM Engine
• Service-Providers:
• Common data sources and application services
Business
Dev
Ops
APM Data
APM Data
Business
process
descriptions
Software
designs for
new processes
![Page 20: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/20.jpg)
21 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Real Example - Design Time Perf. Evaluations
• Evaluating Performance in a Service-oriented Architecture (SOA)
Implementation
Operations
• Service-Consumers:
• Process-oriented user interfaces
• Orchestrated by a BPM Engine
• Service-Providers:
• Common data sources and application services
Business
Dev
Ops
• You should talk toservice provider XYZ!
• Your workload will increase by X!• You should buy … new servers!
• You can achieve the desiredbusiness process lead times but itwill cost you …!
![Page 21: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/21.jpg)
22 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Real Example - Design Time Perf. Evaluations
Modeling improves the collaboration of all parties involved in thesoftware lifecycle (Business, Dev and Ops)!
Business has a level of granularity (business processes) which easesthe communication with the IT department
Service consumers (Dev) can better estimate the expected responsetimes for new business processes
Service providers (Ops) have early access to workloadinformation when new business processes are released
Implementation
Operations
![Page 22: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/22.jpg)
23 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Use Cases – Model-based Evaluations in CD
Implementation
Operations
Evaluate the performance impact of feature additions and bugfixes
For multiple hardware environments and workloads
Without the need to own corresponding test systems!
Deployment Pipeline in a
Continuous Delivery Process
Model-based Performance
Change Detection
Commit
Stage
Automated
Acceptance
Test
Manual
TestingRelease
Developer
checks in
Notify Developer about Performance Change
![Page 23: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/23.jpg)
24 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Benefits – Model-based Evaluations in CD
Implementation
Operations
Ensure that no version gets released with performance regressions
Leverage cost-benefits of fixing performance problems early in thedevelopment process
Increase the performance awareness of developers by immediate feedback on check-ins
Avoid the need to setup and prepare load/performance testenvironments for each and every project
Leverage your existing acceptance/regression testinginvestments for performance evaluations
![Page 24: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/24.jpg)
25 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• A customer (software vendor) has 5 major enterprise applications (EA):
• 1 CI system per EA (Jenkins)
• 5 customer scenarios (workload/hardware environment combinations) should be tested for each new build
• A small performance test environment costs 10 k € / year
• 250.000 € / year
• RETIT Continuous Delivery (RCD) can realise this scenario withone license per EA!
• Saves the huge investment in test environments
Real Example – Model-based Evaluations in CD
Implementation
Operations
![Page 25: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/25.jpg)
26 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Use Cases – Integrating Load Tests w/ Predictions
Implementation
Operations
Setup testenvironment
Define testscenarios
Execute testsAnalyzeresults
Enterprise Applications HardwareSoftware
Measurement Measurement Data
0,00 ms
20,00 ms
40,00 ms
60,00 ms
80,00 ms
100,00 ms
120,00 ms
140,00 ms
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
600 Users 800 Users 1000 Users
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9
Diagrammtitel
Throughput
Response
Time
Resource
Utilization
0,00 ms
20,00 ms
40,00 ms
60,00 ms
80,00 ms
100,00 ms
120,00 ms
140,00 ms
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
600 Users 800 Users 1000 Users
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9
Diagrammtitel
Throughput
Response
Time
Resource
Utilization
0,00 ms
20,00 ms
40,00 ms
60,00 ms
80,00 ms
100,00 ms
120,00 ms
140,00 ms
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
600 Users 800 Users 1000 Users
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9
Diagrammtitel
Throughput
Response
Time
Resource
Utilization
Generate model and
predict performance for
multiple scenarios
![Page 26: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/26.jpg)
27 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Benefits – Integrating Load Tests w/ Predictions
Save costs by reducing the amount of load/performance tests
Increase the coverage of your tests
Evaluate scenarios without buying the corresponding hardware
Easily grow the coverage as the deployment count of yourapplication increases
Implementation
Operations
![Page 27: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/27.jpg)
28 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Save costs by reducing the amount of load/performancetests• Effort for load tests (real customer example, incl. script development, test setup,
execution and result analysis):
• Small scale: 13 person days (PD), medium scale 23 PD, large scale: 41 PD
• Replacing one medium or large scale test by a small scale test with predictionssaves between 8 and 26 PD (assuming 2 PD for predictions)
Real Example – Integrating LT with Predictions
Implementation
Operations
0
100
200
300
400
500
600
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Savi
ngs
in
per
son
day
s(P
D)
Number of performance tests
PD savings when replacing large scale tests withsmall scale tests and predictions
PD savings when replacing medium scale testswith small scale tests and predictions
![Page 28: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/28.jpg)
29 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Use Cases – Model-based Capacity Management
Implementation
Operations
Enterprise ApplicationHardwareSoftware
Measurement Measurement Data
0,00 ms
20,00 ms
40,00 ms
60,00 ms
80,00 ms
100,00 ms
120,00 ms
140,00 ms
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
600 Users 800 Users 1000 Users
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9
Diagrammtitel
Throughput
Response
Time
Resource
Utilization
0,00 ms
20,00 ms
40,00 ms
60,00 ms
80,00 ms
100,00 ms
120,00 ms
140,00 ms
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
600 Users 800 Users 1000 Users
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9
Diagrammtitel
Throughput
Response
Time
Resource
Utilization
0,00 ms
20,00 ms
40,00 ms
60,00 ms
80,00 ms
100,00 ms
120,00 ms
140,00 ms
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
B MRT
B SRT
M MRT
M SRT
P MRT
P SRT
600 Users 800 Users 1000 Users
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9
Diagrammtitel
Throughput
Response
Time
Resource
Utilization
Generate model based on
production data and predict
performance for future scenarios.
What happens if we
migrate to AWS?
How much capacity is required if
we have ten-times more users
during Christmas season?
What happens if we change
our deployment topology?
![Page 29: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/29.jpg)
30 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Right-size your environments to pay only for what you really need
Avoid the need to setup expensive test environments to evaluatechanges
Reduce risk for hardware environment (e.g., cloud) migrations
Reduce the time for capacity management activities
Increased accuracy as the simulations avoid the need forlinear assumptions
Benefits – Model-based Capacity Management
Implementation
Operations
![Page 30: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/30.jpg)
31 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
12:00 12:15 12:30 12:45 13:00 13:15 13:30 13:45 14:00 14:15 14:30 14:45
€/MWh
time
EPEX SPOT INTRADAY AUCTION - 2014-12-11
pi
Smart Grid Capacity Planing for several millionhouseholds
Real Example – Model-based CM
maximum 15 min
Implementation
Operations
![Page 31: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/31.jpg)
32 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE32
Agenda
Performance Modeling
Use cases, Benefits and examples
Demo
![Page 32: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/32.jpg)
33 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
RETIT Solution Demo
• Demo
![Page 33: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/33.jpg)
34 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• What did we learn today?
• Performance modeling…
Summary
Improves cross-
team collaboration
…by integrating multiple
data sources
Increases performance
awareness
… through immediate feedback
during development.
Extends test
coverage
… by allowing you to test more
workloads and hardware
environments.
Saves cost
… by taking the guess-work
out of capacity planning
activities.
![Page 35: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/35.jpg)
36 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Backup Evaluation Results
![Page 36: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/36.jpg)
37 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Beispiel – Erhebung der Daten
IBM System X3755M3
4 AMD Opteron 6172
12 cores and 2,1 GHz
256 GB RAM
VMWare ESXi 5.1.0
Benchmark Driver
Faban Harness
16 Virtual CPUs
8 GB RAM
openSuse 12.3 64bit
UI Server
GlassFish AS 4.0
2 CPUs Enabled
10 GB Heap
openSuse 12.2 64bit
Dynatrace / RETIT APM
WS Server
JBoss 7.1.1
6 CPUs Enabled
6 GB Heap
openSuse 12.2 64bit
Dynatrace / RETIT APM
DB Server
PostgreSQL 9.2.7
4 CPUs Enabled
96 GB RAM
openSuse 12.3 64bit
SAR
IBM System X3755M3
4 AMD Opteron 6172
12 cores and 2,1 GHz
256 GB RAM
IBM System X3755M3
4 AMD Opteron 6172
12 cores and 2,1 GHz
256 GB RAM
IBM System X3550
2 Intel Xeon
6 cores and 2,4 GHz
96 GB RAM
![Page 37: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/37.jpg)
38 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Beispiel - Ergebnisse
• Measurement
• Simulations using data collected by dynatrace + SAR
• Simulations using data collected by RETIT APM + SAR
IBM System X3755M3
4 AMD Opteron 6172
12 cores and 2,1 GHz
256 GB RAM
VMWare ESXi 5.1.0
Benchmark Driver
Faban Harness
16 Virtual CPUs
8 GB RAM
openSuse 12.3 64bit
![Page 38: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/38.jpg)
39 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Beispiel - Ergebnisse
UI Server
GlassFish AS 4.0
2 CPU Enabled
10 GB Heap
openSuse 12.2 64bit
WS Server
JBoss 7.1.1
6 CPU Enabled
6 GB Heap
openSuse 12.2 64bit
DB Server
PostgreSQL 9.2.7
4 CPU Enabled
96 GB RAM
openSuse 12.3 64bit
SAR
IBM System X3755M3
4 AMD Opteron 6172
12 cores and 2,1 GHz
256 GB RAM
IBM System X3755M3
4 AMD Opteron 6172
12 cores and 2,1 GHz
256 GB RAM
IBM System X3550
2 Intel Xeon
6 cores and 2,4 GHz
96 GB RAM
RETIT APM RETIT APM
![Page 39: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/39.jpg)
40 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Ergebnisse -> VM zu realer Hardware
Modellgenerierung mit RCM & RETIT APM
![Page 40: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/40.jpg)
41 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Ergebnisse -> VM zu realer Hardware
Modellgenerierung mit RCM & RETIT APM
![Page 41: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/41.jpg)
42 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Änderungen des Nutzerverhaltens:
• Interarrival Anpassen
• Variation der Transaktionsverteilung
• Neue Usage Szenarien anlegen
• z.B. Closed Workloads
Modellgenerierung mit RCM & RETIT APM
![Page 42: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/42.jpg)
43 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Variation der Nutzer, ihrer Verteilung und der Think Time / Arrival Rate
Modellgenerierung mit RCM & RETIT APM
![Page 43: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/43.jpg)
44 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Variation der Nutzer, ihrer Verteilung und der Think Time / Arrival Rate
Modellgenerierung mit RCM & RETIT APM
![Page 44: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/44.jpg)
45 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Beispiele – AMD Opteron 16 Cores -> Meas. Vs. Simulation (SPECjEnterprise2010 – Java EE 5.0 Benchmark)
Höhere Skalierungen Testen
![Page 45: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/45.jpg)
46 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
• Beispiele – Intel Xeon 8 Cores (adapted from AMD) -> Meas. Vs. Simulation (SPECjEnterprise2010 –Java EE 5.0 Benchmark)
Höhere Skalierungen Testen
![Page 46: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/46.jpg)
47 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
![Page 47: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/47.jpg)
48 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
![Page 48: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/48.jpg)
49 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE49
![Page 49: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/49.jpg)
50 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE50
![Page 50: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/50.jpg)
51 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE51
![Page 51: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/51.jpg)
52 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015
Persona Icons – For your use
Recolor these icons as
needed to match
Dynatrace brand
colors. PictureTools/Format/Color
Customer Conversion
Fanatic
for
Digital Business
Owners
Deployment-obsessed
Releasearaptor
for
Development
War Room
Peacemaker
for
Operations
![Page 52: Beyond Monitoring - d1o3t0kvuxq0hm.cloudfront.net · Beyond Monitoring Delete & Insert Headshot here Delete & Insert Headshot here. 2 COMPANY CONFIDENTIAL –DO NOT DISTRIBUTE #Perform2015](https://reader034.fdocuments.us/reader034/viewer/2022042307/5ed3b848ac5ce417bc527567/html5/thumbnails/52.jpg)
53 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE