8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 1/25
Implementing Stress Testing Guidelines
and Processes for E-Business
Peg Sonday - Progressive Insurance
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 2/25
Introduction
Taking the steps to implement a stress
testing process across a company takes
a strong commitment but the results arewell worth the effort!
Manual or Functional testing do not
identify performance issues
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 3/25
Agenda
First Steps
Environment
Project Life Cycle
Guidelines
Monitoring
Staffing
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 4/25
First Things First...
Management Buy-in
Partnership with Application Team
Budget
± Hardware
± Software
± Staff
Commitment to Quality
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 5/25
Disney World Analogy (with
apologies to Brian Lawrence*) Commitment to Quality
Washed down the entire park & grounds
Dried the Benches
Whole Customer Experience Valued
± *Brian Lawrence is Technical Editor of
Software Testing & Quality EngineeringMagazine - from his May/June 2000
Editorial
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 6/25
Setting Test Objectives
Specific
Measurable
Consider number of concurrent users
Consider number of transactions
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 7/25
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 8/25
If We Build It, They Will
Come«. Test Environment Must Reflect
Production Environment
± Hardware ± Network
± Configuration
±A
pplications ± Software
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 9/25
Project Life Cycle
Stress Testing can be integrated into
existing PLC
Frequently an iterative process
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 10/25
Project Life Cycle
Tasks are added to Project Plan and
scheduled
± Define Testing Approach (Goals & Objectives) ± Plan Test
± Develop and Test Scripts
± Configure Test Scenario
± Execute Test ± Evaluate Results
± Retest
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 11/25
Project Plan Tasks
Test Planning
Test Environment Setup
± Servers
± Network
± Database
± Application Under Test
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 12/25
Project Plan Tasks
Develop Test Scripts
Develop Test Data
Define Test Scenario
Execution
Analysis/Summary
Retest
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 13/25
Guidelines
Develop Guidelines for testing...
± Components
New ComponentMajor Changes
Language
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 14/25
Guidelines
± Applications
Scheduled release
Architecture change
Unscheduled maintenance
Scheduled Benchmark Tests
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 15/25
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 16/25
Guidelines
± Other Hardware
SSL Encryption cards
Routers
Firewalls
Circuits
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 17/25
Guidelines
± Other Software
OS Upgrades
DB Upgrades
Driver upgrades
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 18/25
Monitoring
Guidelines vary by test
Dependent upon goals
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 19/25
Monitoring
Develop guidelines for monitoring
tests...
± CPU ± ASP Queues (for MS shop)
± Other counters for non-MS shop
± Disk
Access
± Memory Utilization
± Handle Count
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 20/25
Monitoring
Database
± DB A
Network ± Sniffer/Network Delay
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 21/25
Staffing
Skill set is mix of both programming and
testing
Architecture knowledge needed Invest in Relevant Training
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 22/25
Teamwork
Developers can run initial tests
± Jumpstart with Generic Test Plans
± Train on Recording Scripts ± Train on Using Text Checks
Test Engineers (or dedicated personal)
± Larger tests ± Complex tests
± Coordinating tests
± Work closely with Development team
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 23/25
Avoid Pitfalls
Test Environment Does Not Reflect
Production
± How would we know if a test truly fails or might pass in production?
Stress Testing Not Included in Project
Plan
± No time to properly test an application
8/6/2019 Stress Testing Guidelines Processes EBusiness
http://slidepdf.com/reader/full/stress-testing-guidelines-processes-ebusiness 24/25
Tying it all together
Commit to Stress Testing
Gain Support from Development Team
Grow Expertise ± Use Test Engineers to mentor project
teams
Buy Expertise ± When expertise not available in house
Single point of contact
± Using one individual or team to coordinate
efforts