PERFORMANCE TESTING

DAVID TZEMACHWWW.DTVISIONTECH.COM

JAN 1 2016

OVERVIEW • PERFORMANCE TESTING IS ONE OF THE MOST IMPORTANT TESTING

METHODS YOU NEED TO GET FAMILIAR WITH.• THE MAIN REASON IS THAT THIS TYPE OF TEST IS ONE OF THE BEST

WAYS TO EVALUATE YOUR SYSTEMS' STABILITY, RELIABILITY, AND EFFECTIVENESS ETC., UNDER A GIVEN WORKLOAD.

• WHILE EXECUTING PERFORMANCE TESTS, WE MUST VALIDATE THE SYSTEM'S ABILITY TO DELIVER AN EXPECTED PERFORMANCE BASED ON THE PREDEFINED REQUIREMENTS AND SPECIFICATIONS.

PERFORMANCE TESTSMAIN GOALS

TEST THE APPLICATION STABILITY • IS THE APPLICATION STABLE TO USE WHEN RUNNING FOR LONG TIME? • ARE THERE ANY ARCHITECTURES THAT LEADS TO

STABILIZATION ISSUES?• IS THE APPLICATION STABLE TO USE UNDER

DIFFERENT LOADS? • IS THE APPLICATION STABLE ENOUGH FOR THE CLIENT

USE?

TEST THE APPLICATION SCALABILITY• WHAT ARE THE ENVIRONMENTS THAT WE CAN USE TO EXECUTE THE SOFTWARE?

• HOW MANY USERS CAN INTERACT WITH THE SOFTWARE AT THE SAME TIME?

• HOW MANY REQUESTS THE SOFTWARE CAN HANDLE ON SPECIFIC TIME?

• IS THE SOFTWARE CAN HANDLE A LARGE DATA GROWTH OVER TIME?

TEST THE APPLICATION RESPONSIVENESS• HOW FAST THE SYSTEM INTERACTS UNDER DIFFERENT ARCHITECTURES?

• HOW FAST THE SYSTEM CAN PROCESS A DIFFERENT TYPE OF DATA?

• HOW FAST THE SYSTEM CAN RETURN THE EXPECTED OUTPUT? • HOW FAST THE SYSTEM CAN HANDLE USER REQUEST? • HOW FAST THE SYSTEM WILL START?

TEST THE APPLICATION RELIABILITY • IS THE APPLICATION RELIABLE FOR USE BY THE END USERS? • ARE THE APPLICATION CRITICAL BUGS REMOVED

BEFORE WE DEPLOY THE SYSTEM ON USER'S ARCHITECTURES?• IS THE APPLICATION CAN HANDLE SYSTEM FAILURES

WITHOUT LOSING THE CUSTOMER DATA ?

PERFORMANCE TESTS THE ADVANTAGES

THE ADVANTAGES (1)• PERFORMANCE TESTS MAY INVOLVE SOME ADDITIONAL

CORRESPONDING TESTING TYPES THAT LEAD TO GREATER COVERAGE (ENDURANCE, SPICE, AND LOAD ETC.).

• PERFORMANCE TESTS ARE ALSO USED TO IMPROVE THE END USER EXPERIENCE.

• PERFORMANCE TESTS CAN BE USED TO EVALUATE AND COMPARE TWO SYSTEMS AGAINST EACH OTHER.

• POOR PERFORMANCE RESULTS MAY INDICATE FOR A CONFIGURATION/FUNCTIONAL DEFECT.

THE ADVANTAGES (2)• PERFORMANCE TESTS CAN EVALUATE MULTIPLE ASPECTS OF

THE SOFTWARE (STABILITY, OVERALL PERFORMANCE SPEED, BOTTLENECKS, EFFECTIVENESS ETC.).

• PERFORMANCE TESTS WILL ALLOW THE TESTING TEAM TO RAISE THE PROBLEMATIC AREAS THAT ARE AFFECTING THE APPLICATION'S PERFORMANCE.

• PERFORMANCE TESTS ALLOWS THE COMPANY TO UNDERSTAND THE OVERALL LIMITS OF THE SYSTEM.

THE ADVANTAGES (3)• PERFORMANCE TESTS ARE ONE OF THE BETTER WAYS TO EVALUATE THE SOFTWARE UNDER DIFFERENT ARCHITECTURES AND OVERLOADS. • PERFORMANCE TEST ARE USED TO EVALUATE THE

PRODUCT AGAINST THE PREDEFINED REQUIREMENTS AND SPECIFICATIONS. • PERFORMANCE TESTS, WILL PROVIDE CONFIDENCE IN

THE PRODUCT (BOTH FROM STOCKHOLDERS AND MANAGEMENT).

PERFORMANCE TESTSTHE DISADVANTAGES

THE DISADVANTAGES (1)• PERFORMANCE TESTS IS A PROCESS THAT TAKE TIME, VERY

LONG TIME.• IN MOST CASES, PERFORMANCE TESTS ARE EXECUTED WITH

3RD PARTY APPLICATIONS THAT NEED FURTHER LEARNING AND INVESTIGATION.

• SOMETIMES YOU CANNOT ANTICIPATE THE EXPECTED RESULTS.

• PERFORMANCE TESTS ARE A COSTLY PROCESS.

THE DISADVANTAGES (2)• SOMETIMES IT’S VERY HARD TO UNDERSTAND THE ROOT CAUSE OF A PROBLEM.

• PERFORMANCE TESTS NEED A DEDICATED AND LARGE TESTING ENVIRONMENT.

• PERFORMANCE TESTS NEED TO BE CAREFULLY DESIGNED AND EVALUATED.

• PERFORMANCE TESTS MAY NOT FIND FUNCTIONAL BUGS THAT YOU CAN FIND WITH OTHER TESTING TYPES.

PERFORMANCE AND CORRESPONDING

TESTING METHODS

SPIKE TESTS TESTING THE SOFTWARE WITH SUDDEN OVERLOADS, THE MAIN GOAL IS VALIDATE THAT THE SOFTWARE CAN HANDLE SITUATIONS WHERE THERE IS A SUDDEN JUMP IN THE USER ACTIVITY OR DRAMATIC CHANGES IN LOAD.

RELIABILITY / ENDURANCE / SOAK TESTS TESTING THE SYSTEM WITH DIFFERENT LOAD FOR A LONG PERIOD OF TIME, THE MAIN GOAL IS TO SEE THAT THE SYSTEM CAN HANDLE SUSTAINED USE FOR A LONG PERIOD OF TIME.

LOAD TESTS TESTING THE SOFTWARE WITH INCREASING SET OF DATA (NEAR TO THE SOFTWARE LIMITATION), THE MAIN GOAL IS TO SEE THAT THE SOFTWARE CAN HANDLE SUCH LOADS WITHOUT HAVING ANY SIDE EFFECTS (UNEXPECTED CRUSHES, MEMORY LEAKS, BOTTLENECKS ETC.).

SCALABILITY TESTS TESTING THE SOFTWARE PERFORMANCE WHILE INCREASING THE VOLUME OF USERS/ENVIRONMENT AND DETERMINE HOW THE SOFTWARE WILL ADJUST TO THE GROWING SCALE.

STRESS TESTS TESTING THE SOFTWARE UNDER LOAD THAT EXCEED THE SYSTEM LIMITATION, THE MAIN GOAL IS TO CAUSE SYSTEM FAILURES, ANALYZE THE CRASH, AND DETERMINE THE WAY THAT THE SOFTWARE RECOVER IN CASE OF SUCH FAILURE.

FAIL-OVER/FAULT-TOLERANCE TESTS TESTING THAT THE SOFTWARE CORRESPONDING TO FAILURES WHILE INVOLVING TWO OR MORE PHYSICAL NODES, IN THIS TYPE OF TESTS, WE NEED TO VALIDATE THAT IN CASE OF FAILURE THE USERS CAN CONTINUE WITH THEIR WORK AND THE APPLICATION DATA/SERVICES ARE MOVED TO THE AVAILABLE “ACTIVE” NODE.

REDUNDANCY TESTSTESTING THE APPLICATION REDUNDANCY MECHANISM IN CASE OF LOAD, THE MAIN GOAL IS TO DETERMINE THE EFFECTIVENESS OF THE BALANCING SYSTEM.

VOLUME TESTS TESTING THE APPLICATION WITH DIFFERENT AMOUNT OF DATA, THE MAIN GOAL HERE IS TO UNDERSTAND THE LIMITS OF OUR APPLICATION AND DEMONSTRATE HOW A SPECIFIC AMOUNT OF DATA CAN AFFECT THE PREDEFINED REQUIREMENTS.

DEFINING A SUCCESSFUL

PERFORMANCE TESTING PROJECT

YOU SUCCEEDED IN IMPROVING THE CURRENT PERFORMANCE (1)

SO YOU CONDUCT YOUR TESTS, SPEND THOUSANDS OF DOLLARS, BUT THE APPLICATION BEHAVES THE SAME WAY AS IT WAS PRIOR TO STARTING YOUR TESTS. IF THAT’S THE RESULT THEN WE FAILED, BUT IF WE SUCCEEDED TO IMPROVE THE PERFORMANCE OF THE APPLICATION THEN WE CAN SAY THAT WE ACHIEVE ONE OF THE MAIN GOALS THAT WE START THE TEST IN THE FIRST PLACE

YOU SUCCEEDED IN FINDING THE APPLICATION BUGS EARLY (2)WELL, YOU ALL KNOW THAT BUGS ARE CHEAPER WHEN YOU FIND THEM ON THE EARLY STAGES OF THE TESTING PROCESS, THIS FACT IS RELEVANT TO PERFORMANCE TESTS AS IT’S RELEVANT TO ANY OTHER TESTING PROCESS.

YOU CAN DELIVER DOCUMENTATION THAT REFLECTS THE PROCESS (3)LIKE ANY OTHER TESTING PROJECT, WE ALWAYS NEED TO ASK OURSELVES A SIMPLE QUESTION “DID WE SPEND THE TIME WE HAD IN AN EFFICIENT WAY..?” WELL, TO ANSWER THIS QUESTION WE NEED TO EXAMINE HOW IT’S ACTUALLY SPENT DURING THE PROJECT, THE BEST WAY TO ACCOMPLISH THIS TASK IS TO EXAMINE THE DOCUMENTATION THAT WE CREATE AND USED DURING THE PROCESS.• EXAMPLES: • THE TEST STRATEGY THAT WE USED. • STP.• STD.• STR.• ETC.

YOU KNOW THE NUMBERS INSTEAD OF ASSUMPTIONS (4)A TRUE SUCCESS MUST INCLUDE “NUMBERS”, EVERY ASSUMPTION THAT YOU HAVE PRIOR TO THE TESTS SHOULD BE TRANSLATED INTO NUMBERS, THAT YOU CAN ANALYZE AND EXAMINE IN DIFFERENT PHASES OF THE PROJECT.

YOU CREATE A BASELINE FOR FUTURE PROJECT (5)PERFORMANCE TESTS WILL CONSUME A HUGE AMOUNT OF RESOURCES, TIME AND MONEY, A GREAT SUCCESS FACTOR IS ACHIEVED IF YOU CAN USE THE CURRENT TESTING ON FUTURE PROJECTS, IT’S LIKE “RECYCLING”, THINK ABOUT THIS FOR A SEC, EACH PERFORMANCE TESTS WILL LEAD TO FURTHER COSTS ON HARDWARE, SOFTWARE AND TESTING TOOLS. IF YOU SUCCEED TO MAINTAIN THIS OBJECTS, YOU CAN USE THEM ON FUTURE PROJECTS AND REDUCE THE COSTS.

FOR ADDITIONAL KB’S PLEASE VISIT MY BLOG

WWW.DTVISIONTECH.COM