SAP CRM WEB UI - Performance Testing and executing

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BOC - boc.sap.com | UAC - uac.sap.com

© 2011 SAP AG 1

SAP CRM WEB UI 7.0 -

Performance Testing

Applies to:

CRM WEB UI 7.0 version Performance Testing process. For more information, visit the Customer Relationship Management homepage.

Summary

This document describes about the performance testing of SAP CRM WEB UI using Vugen, Load runner and HP Performance Center (Controller and Load runner) tools. This document consists of Baseline, Load and Stress test procedure for SAP CRM WEB UI.

Author: Jataprolu Krishna Prathyusha

Company: Accenture Services Private Limited

Created on: 12 June 2011

Author Bio

Author is having above 5 years of experience in SAP CRM. Author has experience in Sales, Service,

Marketing, Interaction Center modules in CRM and has precise expertise in understanding the CRM system.

https://www.sdn.sap.com/irj/sdn/bpx-crm


SAP CRM WEB UI 7.0 - Performance Testing


© 2011 SAP AG 2

Table of Contents

Document Objective: .......................................................................................................................................... 3

Introduction: ........................................................................................................................................................ 3

Types of Performance Testing Related to SAP-CRM – Web - UI: ................................................................. 3

Performance Testing Usage: .............................................................................................................................. 4

Pre-requisites of Performance Testing: .............................................................................................................. 4

Installation details for Performance Testing: ...................................................................................................... 4

Test Script Preparation: ...................................................................................................................................... 6

Intall Vugen: .................................................................................................................................................... 6

Parameterization: .......................................................................................................................................... 17

Correlation: .................................................................................................................................................... 21

Uploading the script to Performance Center: ................................................................................................ 24

Test Script Execution: ....................................................................................................................................... 26

Test Analysis and Final Report: ........................................................................................................................ 36

Issues Faced while Executing PT: .................................................................................................................... 39

Related Content ................................................................................................................................................ 41

Disclaimer and Liability Notice .......................................................................................................................... 42



© 2011 SAP AG 3

Document Objective:

This documents describes about the performance testing of CRM WEB UI 7.0 using Loadrunner and HP Performance Center tools. With the rigorous load-testing functionalities of the SAP LoadRunner application, organizations can better deliver high-performing business processes and can execute, upgrade,or modify existing processes on time and within budget.

Introduction:

Performance Testing is testing that is performed, to determine how fast some aspect or scenario of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage. Performance testing can refer to the assessment of the performance of the system. This is used to determine the speed or effectiveness of a system, network, software program or device. This process can involve quantitative tests like measuring the response time of the server or the Transactions Per Second (TPS) etc.,

Types of Performance Testing Related to SAP-CRM – Web - UI:

Types of Performance Testing for SAP CRM WEB UI:

There are three types of performance testing can be done for SAP CRM UI.

1) Baseline Testing

2) Load Testing

3) Stress Testing

1) Baseline Testing: Single user executes each of the identified online activity and reports one at a time. The objective of Baseline testing is to determine baseline performance of the application and identify individual actions requires for fine tuning of the application. The Response Time of the server is key measure of this Testing.

For Ex: Search accounts on search screen with default option with one user login

2) Load Testing: Load testing is the simplest form of performance testing. A load test is usually conducted to understand the behavior of the application under a specific expected load. This load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration. This test will give out the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software.

For Ex: Search accounts on account search screen with default option with „n‟ number of users login at the same point of time.

3) Stress Testing: Stress testing is normally used to understand the upper limits of capacity within the application landscape. This kind of test is done to determine the application's robustness in terms of extreme load and helps application administrators to determine if the application will perform sufficiently if the current load goes well above the expected maximum. It will be useful to test, in order to see what happens when an acceptable load is exceeded –does the system crash? How long does it take to recover if a large load is reduced? Does it fail in a way that causes collateral damage? Etc.,

For Ex: Search account on account screen with „n‟ number of user login initially and increase the user login till the system crashes



© 2011 SAP AG 4

Performance Testing Usage:

Performance testing can serve different purposes.

It can demonstrate that the system meets performance criteria.

It can compare two systems to find which performs better.

It can measure what parts of the system or workload causes the system to perform badly

Pre-requisites of Performance Testing:

The stable application which must resemble the Production environment is possible to do the performance testing. This is dangerous as if an UAT or Integration test or other tests are going on in the same environment, then the results obtained from the performance testing may not be reliable. As a best practice it is always advisable to have a separate performance testing environment resembling the production environment as much as possible.

Test conditions: In Performance Testing, it is often crucial or difficult to create the test conditions similar to the expected actual use. It is not entirely possible in actual practice to exactly replicate production system workload variability in the performance testing application.

Timing: The timing of the test conditions also crucial in the performance testing. It is critical to the cost performance of a new system which performance test efforts begin at the inception of the development project and extend through to deployment. The later a performance defect is detected, the higher the cost of remediation to fix the defect.

Installation details for Performance Testing:

Tools for Performance Testing:

HP load runner software is an automated performance and load testing product. It is used to examining the system behavior and performance, while generating actual load. HP Load Runner can emulate hundreds or thousands of concurrent users to put the application through the rigors of real-life user loads, while collecting information from key infrastructure components. The results can then be analyzed in detail, to explore the reasons for particular behavior. HP Load Runner consists of several different tools: Virtual User Generator (VuGen), Controller, load generators, Analysis and Launches. In this SAP CRM WEB UI Performance Testing Virtual User Generator and HP Performance Center have been used to prepare test scripts and execute the scripts accordingly. HP Performance Center login details should be created by the admin team of HP PC.

1. Virtual User Generator: This tool is used to prepare the test scripts for the performance testing. The

Virtual User Generator (VuGen) is used to emulate the steps of real human users. VuGen can also run

scripts for debugging. VuGen lets the user record and/or script a test to be performed against an

application under test, and play back and make modifications to the script as needed, such as defining

Parameterization (selecting data for keyword-driven testing). Vugen can be installed on the system from

HPQC Performance Center.

Vugen is used to record the scenario in WEB UI. After the recording, it will create a code in C language

automatically. We can test the scenario from the Vugen itself for one user and can correct it or modify it

if any errors occurred at the time of creation. We can even update the code at Vugen according to our

scenario. Once the script is running fine in Vugen, we can execute the script from performance center to

get the summary of the scenario execution.

http://en.wikipedia.org/wiki/Integration_test



© 2011 SAP AG 5

For Ex: Create a vugen script by executing the scenario like search accounts from account search

screen with one user login to WEB UI.

HP Load Runner supports more than 51 protocols including Web HTTP/HTTPS, Remote Terminal

Emulator, Oracle and Web Services. A protocol acts as a communication medium between a client and

a server. For example an AS400 or mainframe-based application can use a terminal emulator to talk to a

server, and an on-line banking application can use HTTP/HTTPS with some Java and Web services.

Load Runner can record scripts in both single and multi-protocol modes.

The protocol used for CRM WEB UI is: SAP-WEB.

During recording, VuGen records a tester's actions by routing data through a proxy. The type of proxy

depends upon the protocol being used and affects the resulting script. For some protocols, various

recording modes are available to further refine the resulting script. When recording web activity, HP

Load Runner allows a user to choose one of three types of recording modes:

GUI based, URL based and HTML based.

The recording mode for CRM WEB UI is HTML based.

Parameterization: Load Runner allows users editing a script with VuGen to replace recorded values in

a script with parameters. This is called parameterization. Parameterization is often used:

1. When the application needs unique data (such as user name)

2. Data dependency (such as passwords)

3. Data cache

4. Date constraints

Correlation: HP Load Runner uses Correlation to handle dynamic content. Dynamic content refers to

page components that are dynamically created during the execution of a business process, and the

value may differ from the value generated in a previous run. For Examples of dynamic content include

the multiple users‟ login to WEB UI and most importantly the unique session ID that is created each time

a user logs in. Such dynamic values must be correlated in order to run the recorded script. The script

while running will return an error if such a dynamic value has not been correlated, since the value

changes each time the user logs in.

If the dynamic content is a part of a previous server response, Load Runner can correlate that value with

future instances in which it must be sent back to the server. HP Load Runner saves the changing values

into parameters, which are used during emulation.

For Ex: Create a test script from Vugen for account search scenario for different levels (For Ex: 8 levels) of users in Organization hierarchy. The script will be created to check the timelines for each and every level user to search the accounts created by them at CRM WEB UI. For this scenario one script will be created with account search scenario by passing 8 level users as parameters.

2. Performance Center: HP Performance Center is the de-facto standard for enterprise performance

testing. Built on HP Load Runner, it includes the management framework for a web-based, globally

accessible platform that facilitates enterprise-wide testing. The login details for HP Performance

Center. Using Performance Center, you can standardize and centralize your testing resources and

build a testing CoE that supports distributed teams and enables them to collaborate across the

application lifecycle. And you can give everyone from executives to LOB managers to testers‟

visibility into project status and application quality.

http://en.wikipedia.org/wiki/Proxy_server



© 2011 SAP AG 6

HP Performance Center features and benefits:

1) Tests a range of applications and technology, from rich internet applications (RIA) with Web 2.0

technologies, to legacy applications and technology

2) Pinpoints root-cause application performance problems

3) Helps improve application performance while reducing hardware and software costs

4) Reduces the risk of deploying systems that fail performance requirements

5) Supports single and multi-project-based testing

The Test scripts prepared in Vugen can be used to test the Baseline, Load and Stress tests from

performance Center to get the timelines, CPU usage and Transactions per Second for that scenario.

For Ex: The script created for Account search on account screen for 8 levels of users can be uploaded to

Performance Center. The scenario can be executed for specific timelines to check the time taken for

each and every level user to search the accounts created by them on CRM UI.

Test Script Preparation:

Intall Vugen:



© 2011 SAP AG 7

Click on „Download‟ to download Vugen software



© 2011 SAP AG 8

After the installation of Vugen software, system will show the mandatory softwares to be installed to use the

vugen from the system. Need to install the softwares which do not exist in the system. Then only Vugen will

open in the system.



© 2011 SAP AG 9

Vugen has been installed successfully on the system. Go to „Start‟ program and click on „Virtual User

Generator‟ to access Vugen.

Run the Vugen and create a test script for searching accounts on account search screen with created by „Me‟ option (login user).



© 2011 SAP AG 10

Go to File -> Click on new to create a new script. Select SAP – Web protocol to create the test script of CRM

WEB UI. If WEB protocol is not available on the screen, click on „New Multiple Protocol Script‟ -> Select SAP

– Web and click on right arrow to get it on the list.

Check the recording settings and General settings under Tool menu option before recording the script



© 2011 SAP AG 11

Click on SAP – Web to create the script

Click on „OK‟ to record the script.



© 2011 SAP AG 12

Vuser_init: This is used to record the login function of the system

Action: Action part is used to record the particular action for the scenario. For Ex: here it is used to record

the search account screen search functionality action.

User_end: This is used to record the logoff function of the system

According to the client requirement, the script can be recorded by splitting the login, action and logoff details

separately or within the action part itself. In this scenario everything is recorded under Action part.

Comments session is used to give the description of each and every action performed while recording. This

is useful to understand the process sequence of the application on the coding page.

Start Transaction: This option is used to indicate the starting of every transaction

End Transaction: This option is used to indicate the ending of the every transaction.

Login page is already opening. Click on „End Transaction‟ and enter „Account Search_login‟ which indicates

that the login page action has been completed.



© 2011 SAP AG 13

Click on Start Transaction to store the next happening transaction after the login process. Click on „Start

Transaction‟ – Enter „Account Search_Homepage‟ as Homepage will display after the login.

Enter the username and password details and click on „Logon‟ button. The events will be recorded on the

screen.



© 2011 SAP AG 14

Homepage will display on the screen. Once the recording events count is stopped, the transaction „Account

Search_Homepage‟ can be ended by clicking on „End Transaction‟.

Next action will be account search screen display. Click on „Start Transaction‟ and enter „Account Search_Search Screen‟.

Click on „Account Search‟ option on the screen which will open the account search screen.



© 2011 SAP AG 15

Account Search screen has opened. Click on „End Transaction‟ and select „Account Search_Search Screen‟

as the action has been completed.

Next action is to display the Result screen. Click on „Start Transaction‟ and enter „Account Search_Result

Screen‟. Click on search button to see the results.

Click on „End transaction‟ to end the „Account Search_Result Screen‟ action.



© 2011 SAP AG 16

Next action is to logoff the system. Click on „Start Transaction‟ and enter „Account Search_Logoff‟ before

proceeding with the logoff action.

Click on „Logoff‟ to logoff from the application.

Click on „End Transaction‟ and select „Account Search_Logoff‟ to end the transaction.

The script recording is completed. Click on „Stop‟ to stop the script recording. Coding for the above script

will be created on Vugen automatically. Add „lr_start_transaction (“Account Search_Login”) to complete the

loop for login page display.



© 2011 SAP AG 17

Parameterization:

Click on the parameter value -> right click -> select replace with parameter to pass the values automatically.

In this example: same scenario can be executed with different logins (8 different levels of users from

organization model). The login details user Id and password can be parameterized to pass 8 level of user

logon to the system automatically at runtime.

Enter the parameter ID and click on „Properties‟. Select the options mentioned in the following screen shot.

Dat file will be created with the user ID login details.



© 2011 SAP AG 18

Click on „Password‟ to pass the password details also as parameters along with user IDs.

Select „Filepath‟ dropdown as User_ID.dat to create password along with the user ID details. Select the

option „Same line as user_id‟ to pass the correct passwords with the correct user ids. Click on „Edit with

Notepad‟ to add more user ID and passwords to the input parameters file for allowing multiple logons to the

system.

Unique: Read only once

Sequential: parameters access sequentially at run time

Random: parameters selected randomly at run time.

To check the parameter selections for each run click on „simulate parameter‟ and pass the required values

and simulate it. It will display the parameter selection for iteration of the test script.

The script can be executed from vugen and check whether it is picking the correct parameters values or not

before proceeding with the performance center. Before executing the script in Vugen, settings need to be

updated in the Vugen.



© 2011 SAP AG 19

Number of iteration: The script will be executed for 1 time. The script will execute one time with one

parameter value entered in the parameter file.

While executing the script, parameters values will be displayed on the screen to check whether the correct

parameters are passing or not.



© 2011 SAP AG 20

Think time: The idle time taken by the user while executing the scenario. This value has to be taken from the

client. For Ex: If the think time is 10 sec - While recording the scenario, if the user spends 25 seconds idle

time it will be corrected to 10 while executing the scenario. In this scenario the think time has been

considered „As Recorded‟ – whatever may be the think time while recording the script, same time only used

while executing the test script.



© 2011 SAP AG 21

Correlation:

Correlation has to be done for this scenario. A unique system ID will be generated whenever user login to

the system. This is to be handled dynamically using correlation function module to capture the SIDs for user

login.

For Ex: using the following correlation FM the SID has been handled dynamically in this scenario. The

correlation FM should be entered before the server response code. It will pass the dynamic value to the

server to get the correct response.

/*This Correlation function is to handle the Session ID of the application*/

web_reg_save_param( "SID", "LB=(", "RB=)", "Ord=1","Search=Body",LAST );

Correlation Process:

Check correlation options before correlation process.



© 2011 SAP AG 22

Manual Correlation:

Correlation for single user – Create two test scripts for same scenario with single user. Go to Tools ->

Compare with script. Compare one script with the other in Vugen. It will show the differences between the

scripts with yellow lines. The different values between the scripts should be correlated using correlation

function module.

Correlation for multiple users - Create two test scripts for same scenario with two different users. Go to

Tools -> Compare with script. Compare one script with the other in Vugen. It will show the differences

between the scripts with yellow lines. The different values between the scripts should be correlated using

correlation function module.

In this scenario correlation should be done to handle multiple user logins to the system.

Session ID value is different in both the scripts. Session ID value should be handled using correlation.

This will show in „uif_callback 2‟ block in the code.



© 2011 SAP AG 23

Click on „View Tree‟ to check the server coding. Here it needs to handle the Session ID before the server

response. In the above screen shot the SID difference is showing yellow in uif_callback2. This needs to be

checked in before block „uif_callback‟ itself to handle the Session ID.

Click on the Server Response tab and click on „uif_callback‟ on the right navigation bar. Find the Session ID

in the coding and capture the left boundary and right boundary of the SID. These values need to be passed

to the correlation function module to handle this SID dynamically.



© 2011 SAP AG 24

Test Run of the script from Vugen for one user ID: Check the settings mentioned above before running the

script from Vugen. Click on „Run‟ option on the screen as shown below.

The script will be executed from Vugen and results will be displayed as shown below

Script passed successfully. Click on the required steps on the right side navigation pane to check the

screens display in each step.

Uploading the script to Performance Center:

Click on „Tools‟ and select „Performance Center Connection‟ to connect to PC and upload the script to

Performance Center.



© 2011 SAP AG 25

Enter the Performance Center user ID and password to connect to PC from Vugen.

Select the script from the list and click on „OK‟ the script will be uploaded to Performance Center.



© 2011 SAP AG 26

Script has been uploaded to performance center successfully.

Test Script Execution:

Login to Performance Center. Click on Vuser Scripts to check the script uploaded in PC.

Click on „Manage‟ to create the baseline testing for account search scenario.

Click on „new Load Test‟ to create a new baseline testing.



© 2011 SAP AG 27

Enter the basic details on the General tab

Click on Workload tab to assign a load generator to the script. Load generator has to be attached to the

script to execute the scenario. Select the workload type



© 2011 SAP AG 28

Select LG Distribution Option:

Adding LG Manually:



© 2011 SAP AG 29

Click on „Select Virtual Load Generators‟ link and select and LG from the list by checking the check box.

Click on OK.

Enter „LG‟ field value as „1‟ to assign the default load generator



© 2011 SAP AG 30

Select the script as „Account Search_Me‟ from the dropdown under script field.

Assigned the number of Vusers „8‟ (i.e., the number of users assigned for Baseline Test scenario), Load Generator (Which generates Load) , Running VuGen scenarios (i.e., Simultaneously or Gradually) and Duration of Run (15 minutes, For: Time Specific running scenarios, Indefinitely: No Time Bound) for Load

Test. Then the scenario Duration of Vusers to run

Select Duration „for‟ 15 minutes to run the baseline testing for all 8 users simultaneously. End vusers also

simultaneously at a time once the duration of the time is over.



© 2011 SAP AG 31

Set the total duration of the scenario for 30 minutes (15 minutes for script execution and 15 minutes for

report generation).

Note: If the load generator connection failed at the time of executing the scenario. Go to „Project‟ in the left side options and click on „Hosts‟ to check the load generator server is active or not. Check with the PC admin team if the load generator server is showing red status at hosts screen.

Hosts Screen shot:



© 2011 SAP AG 32

Click on „Proceed‟ button.

Note: To cancel the run of the scenario click on „Cancel‟. Timeslot booked for this scenario have to be deleted from „Project‟ -> Timeslots. Select the timeslot booked on the screen and click on „Delete‟. Then only the timeslot

assigned for this scenario will be deleted and can be used for next run.

Timeslot Screenshot:



© 2011 SAP AG 33

Script Execution from PC:

After the Test Run, click on Test Runs – Before creating Analysis Data to view the Results

Baseline testing has been done the above scenario successfully. Load testing also can be done in the same

way by executing the combination of scenario test scripts from the Performance Center according to the

client requirement. Stress also can be executed in the same like load test. While execution, users need to

be passed for the stress test scenarios till the system crashes.

Click on „+‟ to add multiple script at the time of load testing.



© 2011 SAP AG 35

View Script on PC using the following option:

Run time settings can be edited at PC using the following option:



© 2011 SAP AG 36

Test Analysis and Final Report:

Download the Analysis software from the Performance Center to display the analysis report results.

Run it and install analysis software to analyze the results of the scenario.



© 2011 SAP AG 37

Click on „Test run‟ on the HPQC and click on result URL which will display the following Rawresult page

Click on Raw Results and save the results zip file on the desktop. Open the LRA file to check the summary report. Select the required graphs and add it to the report to show the client according to their requirement.



© 2011 SAP AG 38

The following summary will explain the minimum, maximum, average times of each and every transaction for the account search scenario.

Standard deviation value will be used if there is any specific time given by the client to execute the

transaction. SD value is used to find the deviation from the standard value specified by the client.

90 Percentile: This time is calculated after the 90% execution of the script

Result Analysis Screen of Base Line Test:

In Result screen it will display the result for all 8 levels of users. The Minimum, Average and Maximum

timelines taken by the server will display in the summary report for base line testing. In the following graph

the „Average Response time‟ for the 8 levels of users to execute the scenario („Account Search Party involve

„Me‟) has been shown clearly to present it to the client.

Graph:

(Average Response Time Graph for Account Default Search)



© 2011 SAP AG 39

Issues Faced while Executing PT:

The ICM (Internet Communication Manager) parameters should be noted initially before the test is

conducted. Supposing the threads at ICM is maintained as 50 and request sent by the user is more than 50,

server will start giving error as “Connection got shutdown prematurely. Hence maintain the no of requests

sent less than the thread parameter maintain at ICM”

(T-code: RZ10 and SMICM)

1. Maintain „CPIC_MAX_CONV‟ parameter in ITS Internet Transaction Server) at OS level, as after CRM 5.0 version onwards the ITS is a part of SAP CRM package.

2. For executing BI report, maintained co-relation parameters for Drill down scripts. 3. Co-relation parameters should be maintain for „F4 help‟ and „New‟ button in CRM web UI

4. Need to work with the Basis team closely while executing the scripts at on peak and off peak levels.

Network settings should be changed according the number of users used in baseline\load testing.



© 2011 SAP AG 40

5. Most of the issues we faced while running the test script. We need to check the problem areas by keeping debugging points in the test script. We faced the issues with Correlation function modules syntax and coding errors.

6. According to the client requirement 1 – 2% errors in PT can be acceptable. 7. If the test run is giving more than the above mentioned errors, then runtime settings should be

adjusted like Network Simulation timings etc., We changed it from 129 to 999 to stop the erros. 8. According to the scenario test scripts can be executed by keeping all the coding in „Action‟ part of by

dividing the test scripts. 9. As we created one test scripts with different users we faced most of the issues at the time or

correlating the scripts only. 10. Need to work closely with the performance testing developers while creating the test scripts for the

scenarios.



© 2011 SAP AG 42

Disclaimer and Liability Notice

This document may discuss sample coding or other information that does not include SAP official interfaces and therefore is not supported by SAP. Changes made based on this information are not supported and can be overwritten during an upgrade.

SAP will not be held liable for any damages caused by using or misusing the information, code or methods suggested in this document, and anyone using these methods does so at his/her own risk.

SAP offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this technical article or code sample, including any liability resulting from incompatibility between the content within this document and the materials and services offered by SAP. You agree that you will not hold, or seek to hold, SAP responsible or liable with respect to the content of this document.

SAP CRM WEB UI - Performance Testing and executing

Documents

Transcript of SAP CRM WEB UI - Performance Testing and executing