A Simplified Universal Relation Assumption and Its Properties
Managing Application Performance: A Simplified Universal Approach
22
Presented by: Scott Brought to you by: 340 Corporate Way, Suite Orange Park, FL 32073 888‐2 MH PM Tutorial 4/7/2014 1:00 PM “Managing Application Performance: A Simplified Universal Approach” Barber SmartBear 300, 68‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
-
Upload
techwellpresentations -
Category
Technology
-
view
547 -
download
2
description
In response to increasing market demand for well-performing applications, many organizations implement performance testing programs, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. Scott Barber shares the fundamentals of implementing T4APM™ including specific examples from recent client implementations. T4APM™ is a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The approach hinges on applying a simple and unobtrusive "Target, Test, Trend, Tune” cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.
Transcript of Managing Application Performance: A Simplified Universal Approach
Presented by:
Scott
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073 888‐2
MH PM Tutorial4/7/2014 1:00 PM
“Managing Application Performance: A Simplified Universal Approach”
Barber
SmartBear
300,68‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
hief performance evangelist for SmartBear Scott Barber is a respected leader in the
r
as director of
e
Scott Barber SmartBear
Cadvancement of software testing practices, an industry activist, and load testing celebrity of sorts. Scott authored several books―Performance Testing Guidance foWeb Applications, Beautiful Testing, How to Reduce the Cost of Testing, and Web Load Testing for Dummies―and more than 100 articles and blog posts. Founder/president of PerfTestPlus, Scott co-founded the WOPR, servedthe AST and CMG, and is a founding member of ISST. His industry writing, speaking, and activism focus on improving the effectiveness and business alignment of softwardevelopment practices. Learn more about Scott Barber.
2/19/2014
1
Managing Application Performance:A Simplified Universal Approach
Revised for:
Managing Application Performance:A Simplified Universal Approach
By:
Scott BarberProduct Owner – Load Testing Tools
SmartBear Software, Inc.
2/19/2014
2
Product Owner – Load Testing ToolsSmartBear Software, Inc. scott barber@smartbear [email protected]@sbarber
Co‐Founder: Workshop On Performance and Reliabilitywww.performance‐workshop.org
Co‐Author:Author: Contributing Author:
Books: www.perftestplus.com/pubsAbout me: about.me/scott.barber
System or application characteristics related to:
Speed: responsiveness
What is Performance?
‐ responsiveness‐ user experience
Scalability: ‐ capacity‐ load‐ volume
Stability: ‐ consistency‐ reliability‐ stress
2/19/2014
3
What is a Performant System?A system optimized to balance:‐ Cost‐ Cost‐ Time to Market‐ Capacity
…while remaining focused on: Quality of Service to System Users
What is Performance Testing?Testing designed to:
Inform software system optimization byInform software system optimization by balancing:
‐ Cost‐ Time to MarketCapacity‐ Capacity
while remaining focused on the quality of service to system users.
2/19/2014
4
Performance vs. Load Testing?
Conception to Headstone
The Performance LifeCycle is…
Not
Cradle to Grave
2/19/2014
5
Prevent Poor Performance…
© PerfTestPlus, Inc.
… don’t just react when it happens.
Who is Responsible?
2/19/2014
6
© PerfTestPlus, Inc.
Where does it fit in the SDLC?
2/19/2014
7
Delivering Performant Systems;Commonly, Y2K ‐> Today
SPE (Software Performance Engineering)Math‐centric Architect/Design method
+ ProfilingContext‐free code optimization
+ Load TestingDemonstrate (too late) poor performance characteristics
+ Capacity Planning+ Capacity PlanningMath‐centric guessing to determine hardware needs
+ APM (Application Performance Management)Proactively react to production issues
=> Inefficient & Reactionary Delivery & Maintenance
Delivering Performant Systems;Moving Forward
Minimize Need for Complex MathematicsFew teams have mathematicians w/ advanced degreesFew teams have mathematicians w/ advanced degrees
Disconnect Accuracy from Actual Production DataBecause we can’t depend on what we don’t have yet
Remove Socio‐Political BarriersCollaboration improves information sharing & efficiency
Keep/Adapt most Valuable PracticesKeep/Adapt most Valuable PracticesMany good concepts have been poorly implemented
Distribute Valuable Practices Throughout LifeCycleMany valuable practices have suffered from stove‐piping
We need one model for the whole ‘Performance Journey’
2/19/2014
8
© PerfTestPlus, Inc.
Key Points• Entire Lifecycle • Entire Team Focused• Individual, Role, & Team Accountability & Responsibility• Continuous & Collaborative
“DevOps” Perf Team Individual
• Code Profiling• Requirements Revision
• Load/Stress Testing• Environment Config
• Snapshots• Comparisonsq
• Design Validation• Budgets• Units & Components• Monitoring• Tuning
g• Capacity Planning• Concurrency Checks• Situation Analysis
p• Niche Concerns• Mini‐Field Studies
(UAT)• “What‐if…”
© PerfTestPlus, Inc.
2/19/2014
9
Preventing Poor Performance with
© PerfTestPlus, Inc.
Preventing Poor Performance witha little work…every day…
from every one.
Units StoriesTiersResourcesGoals
ProactiveMicro & Macro Establish GoalsUpdate Targets
Dev & Prod
© PerfTestPlus, Inc.
AssessCompareInvestigateAcceptAnswer
Dev & Prod TimesResourcesSizesFrequenciesDashboard!
2/19/2014
10
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
2/19/2014
11
© PerfTestPlus, Inc.
Unit‐Level Testing Tools(‘cause folks always ask)
FireBenchmarks; Performance testing addin for NUnitJUnitPerf; a collection of JUnit test decorators for performance
Firefox Performance Tester's Pack
HTTPerf
2/19/2014
12
© PerfTestPlus, Inc.
An Ounce of Prevention…
2/19/2014
13
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
2/19/2014
14
Load Tests
Stress Tests
d
What is it?
Endurance Tests
Infrastructure/Configuration Verification
Load Variability Analysis
Background Load for Acceptance Tests…
Or any other deliberate approximation of
Production Conditions
© PerfTestPlus, Inc.
2/19/2014
15
RPT is:Inspired by Rapid Software TestingInspired by Rapid Software TestingConsistent with Rapid Software Testing themes
Sanctioned by James Bach, Michael Bolton & the RST instructors to as a specific implementation of the Rapid Testing MethodologyRapid Testing Methodology
For more information about RST, visit:http://www.satisfice.com/info_rst.shtml
“What have we got?What do we want?
Evolved from:
What do we want?How do we get there…?”
‐‐Bob Barber (Scott’s dad)
… as quickly, simply, and … as qu c y, s p y, a dcheaply as possible?
2/19/2014
16
…a common man’s way of expressing the problem solving
h h l i l iapproach that classical engineers employ.
• Given: “What have we got?”• Find: “What do we want?”• Solve: “How do we get there?”
An approach to respond to a specific performance‐related question after 4 or fewer
What is it?
hours of team effort with 1 or more of:
A) The answerB) A partial answer
• To determine the value of additional effort Th l l f ff t t id th• The level of effort to provide the answer
C) Better questions to address the underlying concern
2/19/2014
17
1. Receive Question• Clarify the question• Understand the driver(s) behind the question
2. Generate Test Coverage Outline (TCO) (~20 min)• Simplest path to (partial) answer(s)
Conceptual Approach
• Simplest path to (partial) answer(s)• Comprehensive path to (partial) answer(s)
3. Transform TCO into Rapid Strategy (~20 min)• Only tasks that fit in time box• Stick to tasks requiring available resources
4. Execute Strategy (~2.5 hrs)• Snapshots are your friends• Anecdotal is sufficient
~4 hr
5. Consolidate/Analyze Data (~30 min)• Identify patterns• Confirm patterns (time permitting)
6. Report Results (~20 min)• Answer(s) / Time or Effort to get answer(s)• Follow‐on questions of interest
© PerfTestPlus, Inc.
© PerfTestPlus, Inc.
2/19/2014
18
© PerfTestPlus, Inc.
The Bottom Line
© PerfTestPlus, Inc.
2/19/2014
19
Addendum
Questions?
2/19/2014
20
Contact Info
about me/scott barberabout.me/scott.barber
Product Owner – Load Testing ToolsSmartBear Softwear, Inc.
E-mail:Scott barber@smartbear com
Web Site:[email protected]
Blog:scott-barber.blogspot.com
SmartBear.com
Twitter:@sbarber
