Cloud Computing Cloud Computing Overview of Distributed Computing.
With Cloud Computing, Who Needs Performance Testing?
-
Upload
eurostar-conference -
Category
Technology
-
view
298 -
download
1
description
Transcript of With Cloud Computing, Who Needs Performance Testing?
![Page 1: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/1.jpg)
With cloud computing:who needs performance testing
Webinar June 24th 2014Albert Witteveen
![Page 2: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/2.jpg)
You just woke up after a 10 years nap:
Team member:“We can add extra processing power and memory on the fly.
An extra database has a lead time of two weeks”
Imagine
![Page 3: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/3.jpg)
Does this sound familiar: Performance test: everything OK Day 1 on production: we end up adding more than four
times the hardware
Question
![Page 4: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/4.jpg)
1. the tools simulate but are not quite equal2. load profiles are based on too many assumptions3. we report more accurately than we can measure4. long setup time → limited amount of tests5. we hide it all in complex reports
Load testing weak points
![Page 5: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/5.jpg)
We send and accept the same requests and responses but can't anticipate slight changes
In production, a lot more is going on than just our test Did we really get a good response Similar hardware is expensive
Our tools simulate reality but are not equal
![Page 6: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/6.jpg)
Cloud computing: adding extra hardware can be done on the fly and on a moments notice
With the high costs of performance testing and how easy we can 'speed things up' if needed:
Why bother testing? The money is better spent on that extra hardware
Cloud computing
![Page 7: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/7.jpg)
Just start with an overkill of hardware and scale down to what is actually used!
Dutch auction?
![Page 8: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/8.jpg)
Introducing: The Queuing theory
![Page 9: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/9.jpg)
The naïve tester (me)
![Page 10: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/10.jpg)
Computers are running or idling. The queuing theory is an established model for
performance engineers It can describe the behavior of systems on every layer
Queuing theory
![Page 11: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/11.jpg)
Simple queue
![Page 12: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/12.jpg)
Multiple registers
![Page 13: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/13.jpg)
Take a number
![Page 14: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/14.jpg)
Queuing center: a location in our system where waiting (queuing) occurs a Bottleneck if you will
◦ They can exist anywhere: CPU, Memory, Network, IO, other systems◦ There is always one or more queuing centers◦ A queuing center really determines the performance◦ The queuing center provide key information on scalability◦ Service and wait time are the real components of performance
Queuing center
![Page 15: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/15.jpg)
Queuing model describe anything: large connected systems, small, embedded ...
You can 'zoom in' and the model can describe the behavior or the server
You can keep zooming in to CPU, network etc.
Multiple layers
![Page 16: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/16.jpg)
Multiple zoom levels Residence time = wait + service time There is always a queuing center No queuing center found: look harder
Queuing models
![Page 17: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/17.jpg)
Cloud computing not infinite: Financial limit Technical: IO/Network/CPU speed per process
We don't build supercomputers to calculate a mortgage offer
Back to the cloud
![Page 18: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/18.jpg)
Always find the queuing centers Based on the result: judge 'yes we are likely to meet
requirement X Y and Z' Show where the risks are 'requirement x cannot be feasibly
met for function y' Explore the risks
How to apply the queuing theory
![Page 19: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/19.jpg)
Explore identified resource heavy components with stakeholders, developers and oracles◦ Other use of this component?◦ Real frequency of usage?◦ Validity of the (generic) requirement for this function?
Place the results in context: ◦ You may have a bigger issue than you thought◦ Or it is actually OK for this usage
Exploring the risks
![Page 20: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/20.jpg)
Define a set of key functions/use cases with stakeholders and experts (i.e. functional testers)
Per test identify at least one queuing center Compare with generic requirements
◦ Can meet ?◦ Risk exist → explore → place in context →define further test
The model allows you to place real behavior in context and a realistic assessment of risk
Approach
![Page 21: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/21.jpg)
If no queuing center was found → monitoring was not sufficient
Queuing centers:◦ Tell you about the risks to core functionality: performance and
financial◦ Tell you on the ability to scale◦ Improve response time in scaling up
For stakeholders
![Page 22: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/22.jpg)
Stakeholders don't (necessarily) understand queuing models
Explain in what matters to them: i.e. when making the offer it takes 15 seconds to generate
Think of the systems as queuing systems and explain behavior
Make a model?
![Page 23: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/23.jpg)
Knowing what the behavior is can tell you:◦ if you can handle requirements ◦ how to scale if needed◦ estimate if performance can be met within budget◦ if you need to adapt your cloud (i.e. improve IO/network, CPU)
So yes: it still makes sense to do performance testing
Summary for the cloud
![Page 24: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/24.jpg)
Batch process tested to be run from multiple servers Process needed to be faster Risk: 'on-line' processes on server should not be impacted
Finding: 3 servers, three times as fast. But no queuing center found???
Deep diving in CPU monitoring showed the queuing center: Process was pausing/waiting after each cycle
Conclusion: → on-line processes not impacted as there was sufficient CPU time for other processes
Example: batch process
![Page 25: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/25.jpg)
Stress point found Unclear where queuing center was
Cause: JAVA memory management can be deceiving on OS level.
Rule that the queuing center needed to be found made us find out. The absence of a queuing center makes you look further
Example JAVA
![Page 26: With Cloud Computing, Who Needs Performance Testing?](https://reader035.fdocuments.us/reader035/viewer/2022070315/554f3851b4c905cd048b4ebe/html5/thumbnails/26.jpg)
Next