AppLoader User Guide

Protocol Independent

Load Testing

AppLoader7

User Guide

User Guide: AppLoader

© 2012 NRG Global. All rights reserved.

Page 1

CONTENTS

INTRODUCTION TO APPLOADER .................................................................................................................. 4

APPLOADER ARCHITECTURE ............................................................................................................................. 4

INSTALL APPLOADER SUITE ......................................................................................................................... 5

PREREQUISITES ............................................................................................................................................... 5

1. INSTALL APPLOADER .................................................................................................................................... 6

2. INSTALL INJECTOR ...................................................................................................................................... 11

3. INSTALL SCENARIOBUILDER ........................................................................................................................ 17

END USER LICENSE AGREEMENT .............................................................................................................. 22

POST-INSTALLATION ................................................................................................................................... 24

APPLOADER OVERVIEW .............................................................................................................................. 25

TERMINOLOGY ............................................................................................................................................... 26

APPLOADER CONTROLLER ............................................................................................................................. 27

LEFT PANE .................................................................................................................................................... 28

MANAGE INJECTORS ................................................................................................................................... 29

CREATE RUSERS ........................................................................................................................................... 29

Format CSV File........................................................................................................................................ 31

START RUSERS .............................................................................................................................................. 31

rUserManager ........................................................................................................................................... 32

INJECTOR ACTIONS ........................................................................................................................................ 35

Make injector “active” or “inactive” ............................................................................................................. 35

Delete Injector ........................................................................................................................................... 35

Edit Injector ............................................................................................................................................... 36

Manage Injector rUsers ............................................................................................................................. 37

STOP RUSERS ............................................................................................................................................... 38

DISCONNECT RUSER ...................................................................................................................................... 38

ADD NEW INJECTOR ....................................................................................................................................... 39

SETUP ........................................................................................................................................................... 39

Send Scenarios ......................................................................................................................................... 40

INJECTOR UTILITIES ....................................................................................................................................... 40

Run Remote Command............................................................................................................................. 40

Kill Remote Process .................................................................................................................................. 40

Log Off rUsers ........................................................................................................................................... 40

Restart rUsers Process ............................................................................................................................. 41

Advanced Settings .................................................................................................................................... 41

MANAGE BACKEND MONITORS .................................................................................................................. 43

ADD NEW SYSTEM ......................................................................................................................................... 43

TEMPLATES ................................................................................................................................................... 47

BACKEND MONITORS ACTIONS ....................................................................................................................... 49



Page 2

PLAN AND RUN TESTS ................................................................................................................................. 50

CREATE NEW PLAN ........................................................................................................................................ 51

Details ....................................................................................................................................................... 52

Test Plan ................................................................................................................................................... 52

Scenarios .................................................................................................................................................. 53

CSV files ................................................................................................................................................... 53

Rendezvous Points ................................................................................................................................... 54

SLA ........................................................................................................................................................... 55

Save / Review / Run Test Plan .................................................................................................................. 55

Test Plans List .......................................................................................................................................... 57

Delete Test Plan ....................................................................................................................................... 57

Edit Test Plan ............................................................................................................................................ 57

Clone Test Plan ........................................................................................................................................ 58

VIEW REAL TIME RESULTS .......................................................................................................................... 59

VIEW LOAD TEST STATUS - SUMMARY ............................................................................................................. 59

VIEW LOAD TEST STATUS - DETAIL ................................................................................................................. 61

VIEW THE EXECUTION PLAN ............................................................................................................................ 61

VIEW RUNTIME INFORMATION ......................................................................................................................... 63

Test Failures / Screenshots ....................................................................................................................... 64

Alerts ......................................................................................................................................................... 64

VIEW ALERTS................................................................................................................................................. 64

RESTART RUSERS .......................................................................................................................................... 64

END LOAD TEST ............................................................................................................................................. 65

RESET ........................................................................................................................................................... 65

VIEW RUN LOG .............................................................................................................................................. 65

EXPORT SUMMARY......................................................................................................................................... 65

TEST ANALYSIS ............................................................................................................................................ 66

TEST ANALYSIS PAGE LAYOUT ........................................................................................................................ 66

Load Test .................................................................................................................................................. 66

Pie Chart, Stats and Mixed Data Graph .................................................................................................... 67

SEARCH PREVIOUS TEST ................................................................................................................................ 70

DELETE SELECTED TESTS .............................................................................................................................. 73

ARCHIVE SELECTED TESTS ............................................................................................................................. 73

ANALYZE SELECTED TEST .............................................................................................................................. 74

EXPORT SELECTED LOAD TEST DATA ............................................................................................................. 74

REPORTS ...................................................................................................................................................... 74

Load Test Summary Report ...................................................................................................................... 74

Load Test Failure Analysis Report ............................................................................................................ 75

Load Test Errors Report ............................................................................................................................ 76

GO TO CURRENT TEST ................................................................................................................................... 77

ANALYZE CURRENT TEST ............................................................................................................................... 77

EXPORT LOAD TEST DATA .............................................................................................................................. 77



Page 3

CREATE SCENARIOS IN SCENARIOBUILDER TO SEND TO APPLOADER .............................................. 78

LAUNCH SCENARIOBUILDER ........................................................................................................................... 79

Create a Simple Client/Server Scenario .................................................................................................... 80

Create a Simple Web Scenario ................................................................................................................. 87

APPENDIX A-1: SCENARIOBUILDER ACTION GROUPS ....................................................................................... 92



Page 4

INTRODUCTION TO APPLOADER

AppLoader is NRG Global’s tool for load testing business applications. With AppLoader, you can create test

Scenarios to verify the performance of your application under varying load levels. AppLoader simulates real

user load activity for applications, so you can be confident that a new or upgraded application meets expected

performance requirements before bringing it into a live environment. With AppLoader, you can stress test the

performance of your IT systems by simulating peak traffic from multiple locations simultaneously.

AppLoader gives you the answers to the following questions:

Does your application perform properly under load?

Do application response times meet your target requirements?

How many simultaneous users can your application handle?

APPLOADER ARCHITECTURE

Start with ScenarioBuilder, the easy-to-use tool that facilitates the creation of a wide range of test Scenarios for

load testing. Send your Scenario to the AppLoader Controller and incorporate it into a test plan. The

Controller manages, drives, and monitors load test in real time, as Scenarios play on Injectors. AppLoader

collects application response times and back-end metrics, and provides test analyses through summarized and

detailed reports and graphs.



Page 5

INSTALL APPLOADER SUITE

PREREQUISITES

The AppLoader Suite consists of three components – AppLoader, Injector and ScenarioBuilder. Together,

they comprise NRG Global’s easy-to-use, script-free load testing solution.

AppLoader works with Terminal Services from Microsoft to generate multiple users simultaneously interacting

with an application from one Windows server.

Install the AppLoader components in the following manner:

1. AppLoader – Install on a Windows PC or Windows Server. Installation sets up the Controller which

manages, drives, and monitors load tests in real-time, and provides test analyses through summarized

and detailed reports and graphs.

2. Injector – Install on a Windows server*. Installation sets up the Injector which hosts multiple users

performing real life Scenarios on the application under test.

3. ScenarioBuilder – Install on a Windows Server (same machine or clone of the “Injector” Server). This

easy-to-use tool facilitates the creation of a wide range of test Scenarios for load testing.

Start by downloading the AppLoader installation package from the Product Downloads page on the NRG

Global website. Save the apploader_suite_setup.zip file to your Windows PC and extract the contents of the

file to a new folder. You will find a separate folder for each of the components; AppLoader, Injector and

ScenarioBuilder. Within each folder is a setup file for its respective component.

*The Injector can be installed on a PC, however only a single user can be generated from a PC.



Page 6

1. INSTALL APPLOADER

Run the setup.exe file in the AppLoader folder

Click “Next” to advance to the “End User License Agreement” (EULA). (Note: If upgrading from an older

version of AppLoader, a window will open indicating that the “AppLoader Installer has detected an older

version (x.x.x.x) of AppLoader installed on your system.” Choose “Upgrade” to proceed.)



Page 7

Review the terms of the EULA and click “I Agree” to continue.



Page 8

Select the destination folder for the AppLoader program installation. Click “Next” to accept the default location

or “Browse” to an alternate location, then click “Next.”



Page 9

The AppLoader Controller can be opened in a web browser. The “Web Server Port” setting determines the

port through which AppLoader’s web services are accessible.

The AppLoader Controller and the Injector communicate through web services. The “Controller Port Range”

setting determines the ports through which they will communicate.

Choose the defaults unless they are used by other applications. You can check if a port is open in your

network by issuing the telnet* command. If the port is open, you will see a blank screen after issuing the

following command:

telnet [ipaddress][port]

where:

[ipaddress] is the IP address of the server to which you are trying to connect

[port] is the port number you are checking

If the port is open, you will see a blank screen, indicating that the port is open and available for AppLoader.

Enter port settings and click “Next” to initiate the AppLoader installation process.

* In Windows Vista and Windows 7 you may need to enable telnet first.



Page 10

Click “Finish” to complete the AppLoader installation.



Page 11

2. INSTALL INJECTOR

Run the setup.exe file in the Injector folder


version of Injector, a window will open indicating that the “Injector has detected that a previous version of

Injector is already installed on this computer. This version must be uninstalled to continue with the Injector

installation.” Click “OK” to uninstall.)



Page 12




Page 13

Select the destination folder for the Injector program installation. Click “Next” to accept the default location or

“Browse” to an alternate location, then click “Next.”



Page 14

Enter the IP address of the server hosting the AppLoader Controller in the “AppLoader Base IP Address” field.

Enter the “AppLoader Base Port” address – use the same port that was assigned to the AppLoader “Web

Server Port” in the AppLoader installation.

Select the “Scenarios Folder” location. We recommend that you keep the default location of C:\Program

Files\NrgGlobal\Injector\scenariostore\.

Group Policy: screensaver must be disabled for proper Injector performance. If you uncheck this now,

screensaver must be disabled at some point prior to using the Injector in a load test.

Click “Next”.



Page 15

Enter the following Injector parameters:

Injector Alias – a unique name to identify the Injector. This is how the Injector will be identified by and

registered with the AppLoader Controller. (i.e. Injector1)

Injector Host – the computer name of the machine where the Injector is being installed.

Injector IP – the IP address of the machine where the Injector is being installed.

Injector Location – description field, usually the Injector Alias + Injector IP

Description

Click “Install” to initiate the Injector installation process.



Page 16

Click “Finish” to complete the Injector installation.



Page 17

3. INSTALL SCENARIOBUILDER

Run the setup.exe file in the ScenarioBuilder folder


version of ScenarioBuilder, a window will open indicating that “ScenarioBuilder version x.x is already installed

on this environment. This installer currently doesn’t support upgrades, if you wish to continue, please uninstall

the old version first and try again.” Click “OK” to close this window.)



Page 18




Page 19

Select the destination folder for the ScenarioBuilder program installation. Click “Next” to accept the default

location or “Browse” to an alternate location, then click “Next.”



Page 20

Enter the IP address of the server hosting the AppLoader Controller in the “Base Hostname / IP” field.

Enter the “Base Port” address – use the same port that was assigned to the AppLoader “Web Server Port” in

the AppLoader installation.

Check the “Base uses a secure web interface (SSL)” box if your web server uses the SSL protocol.

Click “Install” to initiate the installation process.



Page 21

Click “Finish” to complete the ScenarioBuilder installation.



Page 22

END USER LICENSE AGREEMENT

IMPORTANT -- PLEASE READ CAREFULLY BEFORE USING THIS SOFTWARE PRODUCT: This End-User

license Agreement ("EULA") is a contract between (a) you (either an individual or the entity you represent) and

(b) NRG Global, Inc. ("NRG GLOBAL") that governs your use of the software product (“Software”).

RIGHTS IN THE SOFTWARE ARE OFFERED ONLY ON THE CONDITION THAT YOU AGREE TO ALL

TERMS AND CONDITIONS OF THIS EULA. BY INSTALLING, COPYING, DOWNLOADING, OR

OTHERWISE USING THE SOFTWARE, YOU AGREE TO BE BOUND BY THIS EULA. IF YOU DO NOT

ACCEPT THIS EULA, YOU MUST DESTROY ALL COPIES OF THE SOFTWARE AND ACCOMPANYING

DOCUMENTATION WITHIN 1 DAY OF PURCHASE FOR A REFUND FROM NRG GLOBAL.

1. THIRD PARTY SOFTWARE. The Software may include, in addition to NRG GLOBAL proprietary software

(“NRG GLOBAL Software”), software under licenses from third parties (“Third Party Software” and “Third Party

License”). Any Third Party Software is licensed to you subject to the terms and conditions of the corresponding

Third Party License. Generally, the Third Party License is located in a file such as license.txt or read.me file;

you should contact NRG GLOBAL support if you cannot find any Third Party License.

2. LICENSE RIGHTS. You will have the following rights provided you comply with all terms and

conditions of this EULA: a. Use. NRG GLOBAL grants you a license to Use one copy of the NRG

GLOBAL Software. "Use" means installing, copying, storing, loading, executing, displaying, or

otherwise using the NRG GLOBAL Software. You may not modify the NRG GLOBAL Software or

disable any licensing or control feature of the NRG GLOBAL Software. Additional restrictions on Use

may appear in the User Documentation. You may not separate component parts of the NRG GLOBAL

Software for Use. You do not have the right to distribute the NRG GLOBAL Software. b. Copying. You

may copy the NRG GLOBAL software for back-up and archival purposes, provided any copy contains

all the original software’s proprietary notices.

3. UPGRADES. To Use NRG GLOBAL Software provided by NRG GLOBAL as an upgrade, update, or

supplement (collectively “Upgrade”), you must first be licensed for the original NRG GLOBAL Software

identified by NRG GLOBAL as eligible for the Upgrade. To the extent the Upgrade supersedes the original

NRG GLOBAL Software, you may no longer use such NRG GLOBAL Software. This EULA applies to each

Upgrade unless NRG GLOBAL provides other terms with the Upgrade. In case of a conflict between this EULA

and such other terms, the other terms will prevail.

4. TRANSFER. a. Third Party Transfer. The initial end user of the NRG GLOBAL Software may make a one-

time transfer of the NRG GLOBAL Software to another end user. Any transfer will include all component parts,

media, User Documentation, this EULA, and if applicable, the Certificate of Authenticity. The transfer may not

be an indirect transfer, such as a consignment. Prior to the transfer, the end user receiving the transferred

Software will agree to this EULA. Upon transfer of the NRG GLOBAL Software, your license is automatically

terminated. b. Restrictions. You may not rent, lease or lend the NRG GLOBAL Software or Use the NRG

GLOBAL Software for commercial timesharing or bureau use. You may not sublicense, assign or otherwise

transfer the NRG GLOBAL Software except as expressly provided in this EULA.

5. PROPRIETARY RIGHTS. All ownership and intellectual property rights in the Software and User



Page 23

Documentation are owned by NRG GLOBAL and/or its suppliers and are protected by law, including

applicable copyright, trade secret, patent, and trademark laws. You will not remove any product

identification, copyright notice, or proprietary restriction from the Software.

6. LIMITATION ON REVERSE ENGINEERING. You may not reverse engineer, decompile, or disassemble the

NRG Global Software, except and only to the extent that the right to do so is allowed under applicable law.

7. CONSENT TO USE OF DATA. NRG Global and its affiliates may collect and use technical information you

provide in relation to (i) your Use of the Software or the NRG GLOBAL Product, or (ii) the provision of support

services related to the Software or the NRG GLOBAL Product. All such information will be subject to NRG

GLOBAL’s privacy policy. NRG GLOBAL will not use such information in a form that personally identifies you

except to the extent necessary to enhance your Use or provide support services.

8. LIMITATION OF LIABILITY. Under no circumstances shall NRG GLOBAL or its suppliers or resellers be

liable to licensee or any other person for any special, incidental, indirect, or consequential damages, including

damages for loss of goodwill, lost profits, work stoppage, business interruption, personal injury, computer

failure, loss of privacy, or any and all commercial damages or losses, even if NRG GLOBAL or any supplier

has been advised of the possibility of such damages. Further, in no event shall NRG GLOBAL’s liability under

any provision of this agreement exceed the license fee paid to NRG GLOBAL. Some states or other

jurisdictions do not allow the exclusion or limitation of incidental or consequential damages, so the above

limitation or exclusion may not apply to you.

9. U.S. GOVERNMENT RESTRICTED RIGHTS AND COMPLIANCE WITH EXPORT LAWS. This software is

provided with RESTRICTED RIGHTS. Use, duplication, or disclosure by the Government is subject to

restrictions as set forth in subparagraph (c)(1)(ii) of The Rights in Technical Data and Computer Software

clause of DFARS 252.227-7013 or subparagraphs (c)(i) and (2) of the Commercial Computer Software-

Restricted Rights at 48 CFR 52.227-19, as applicable. You will comply with all laws, rules, and regulations (i)

applicable to the export or import of the Software, or (ii) restricting the Use of the Software.

10. RESERVATION OF RIGHTS. NRG Global and its suppliers reserve all rights not expressly granted to you

in this EULA.



Page 24

POST-INSTALLATION

For guidance in configuring your Windows server for optimal performance as an AppLoader Injector, please refer to the applicable document listed below:

Windows Server 2003 Optimization.pdf

Windows Server 2008 Optimization.pdf

http://nrgglobal.com/downloads/AppLoader7_InjectorServer2003Configuration.pdf

http://nrgglobal.com/downloads/AppLoader7_InjectorServer2008Configuration.pdf



Page 25

APPLOADER OVERVIEW

AppLoader’s ease and efficiency makes it possible for Quality Assurance teams to load test the performance of

their application(s) in just a few steps:

Build a Scenario using ScenarioBuilder;

Send Scenario to AppLoader Controller;



Page 26

Launch Controller Web Interface. AppLoader’s intuitive user interface steps you through the process of configuring and generating a load test;

TERMINOLOGY

AppLoader Controller: The central console from which tests are managed and monitored in real-time.

Includes the report writer which generates summarized and detailed test analyses through reports and graphs.

ScenarioBuilder: The tool for creating sequences of user actions on the application under test.

Injector: The Windows server that hosts rUser sessions which execute the Scenarios that were built in

ScenarioBuilder.

rUsers: The “real user” accounts which exist in the Windows environment and and are administered through

AppLoader.



Page 27

APPLOADER CONTROLLER

After building your Scenarios, launch the AppLoader Controller web interface to manage, run, and analyze

your tests. If you haven’t built your scenarios yet, please click here to learn more about building Scenarios.

After installing AppLoader, a shortcut will be created on the desktop. Launch the Controller by:

(1) clicking on the desktop shortcut, or;

(2) entering the Controller address + web port into the address bar of a web browser (http://<IP of controller

installation machine>:8888);

The Apploader Controller opens to the “Manage Injectors” page;



Page 28

LEFT PANE

The “Left Pane” of the AppLoader Controller provides a guide to creating, running, monitoring and analyzing

your tests. Simply follow from top to bottom:

Manage Injectors: Create rUsers; Start rUsers; Send Scenarios; Edit

Injectors.

Manage the Windows environment where real users (hereinafter refered to

as rUsers) interact with the application(s) under test.

All registered Injectors appear in the active window next to the buttons in the Left Pane.

Manage Backend Monitors: Create monitors that report the health of

application backend servers (CPU, Memory, Disk I/O, Network, processes,

services, etc.).

From the “Manage Backend Monitors” page, define “Systems” and specify their metrics. Add Systems to “Templates” for inclusion in “Test Plans”.

Templates allow for the correlation of the backend data with application response time data, through graphs and reports.

Plan and Run Tests: Create & Edit Test Plans; Attach Scenarios;

Parameterize Data; Define Rendezvous Points; Add Alerts

The Test Plan formulates and defines the parameters of the load test, including Scenarios, rUsers, Variables distribution, Backend Templates, Rendezvous Points, Ramp-up and Steady State times.

View Real Time Results: Monitor Test in Progress

View rUser status (connected, disconnected, playing, completed, etc).

View current test status (response times, graphs, error screenshots, backend alerts, number of failures, etc.) in real time.

Restart rUsers, add more rUsers to the test, end the load test, and reset the load test.

Test Analysis: Analyze Current or Previous Tests

View response time graphs and reports, from higher level to detailed level.

Export complete test data and graphs into Excel or PDF formats.



Page 29

MANAGE INJECTORS

Injectors are the Windows servers which host the rUsers that play back Scenarios,

thereby “injecting” a load on the application under test. Upon installation, Injectors

register themselves with the AppLoader Controller. Injectors that have been

properly configured during installation can be viewed from the “Manage Injectors”

page of the Controller. If an installed Injector is not listed on this page, please

reinstall the Injector ensuring that correct Controller address and port properties are

provided.

To open the “Manage Injectors” page, simply click the “Manage Injectors” button in the left pane of the

Controller.

Note that the AppLoader Controller defaults to the “Manage Injectors” page upon opening.

CREATE RUSERS

Creating rUsers is the first step in preparing a load test. Once rUsers are created, this step does not have to

be repeated, unless rUsers are deleted.

Click the “Create rUsers” button in the menu bar. “Injector rUsers” form opens;



Page 30

Mode: Select the type of rUser accounts to create (Incremental; From CSV File).

Incremental – Choose this mode if testing in a non-domain environment. Local user accounts will be

created simultaneously on both the Windows server and within AppLoader.

From CSV File – Choose this mode if your testing environment requires that you use authenticated

domain users. Domain user accounts must already exist on the Windows server. A CSV (comma

separated values) file containing username, password, and domain must be prepared in advance to

create users in this mode. Click here for more on how to format a CSV file for this mode.

CSV File to Import: If using “From CSV File” mode, navigate to the CSV file containing the user accounts to

be uploaded to Apploader.

The following fields are only accessible when “Incremental” mode is selected:

User Name: Provide a generic user name for rUser accounts. Apploader will add a sequential numeric suffix

to make each account unique (e.g. ruser001, ruser002, ruser003, etc…). User names must be compliant with

the Windows server requirements. Recommended value: ruser)

Start Index: Specify the numeric value from which User Name indexing should begin. For example, if 200 is

entered, indexing will be: ruser200, ruser201, ruser202, etc…. Leave this value at “1” unless you have a need

to manipulate the indexing.

User Count: Enter the number of rUsers you wish to create.

Password: Enter a password for the rUser accounts. Password must be compliant with the Windows server

requirements. Recommended value: Password123).



Page 31

Click “Submit” to issue the “Create rUsers” command.

FORMAT CSV FILE

For testing in an environment that requires authenticated domain users, select the “From CSV File” mode

when creating rUsers. Be sure that the domain user accounts have been created on the Windows server

hosting the Injector prior to “Creating” rUsers with AppLoader.

Create a CSV file (no header row, no spaces before/after commas) containing user names, passwords and

domain. Dedicate a row to each user. Below is a sample CSV file:

rwilliams,password2012,mydomain tjones,password2013,mydomain asmith,password2014,mydomain rclark,password2015,mydomain

CSV files can be created in Microsoft Excel - “Save as type” CSV (Comma Delimited) (*.csv) or with a word

editor like Notepad – “Save as type” All Files (*.*) and add the .csv extension to file name.

START RUSERS

The next step in the process is to “Start rUsers”. This process launches the rUserManager on the Injector and

initiates the rUser logins.

Click the “Start rUsers” button . The “Start rUsers” window pops up;

# of Users: Specify the number of users to start on the selected Injector.

Start Index: Specify the numerical index from where the users will be launched. For Injectors using CSV files,

this is the row number in the CSV file.

For example, if you would like to start from the 10th user to 200th user, you would need to set 190 on # of users

field and start index to 10. rUser10 to rUser200 would be started in the rUser Manager.

Launch Delay: Specify a launch delay in seconds. The system waits this amount of time after starting each

rUser. It is recommended to use a delay value to avoid overloading the system by logging in multiple user

accounts all at the same time. For example, a Launch Delay of 5 results in a new rUser logging in every 5



Page 32

seconds.

Click “Submit” to start rUsers.

Started rUsers can be viewed in the “View Real Time Results” page of the Controller;

RUSERMANAGER

AppLoader uses the rUserManager to house all of the rUser sessions on an Injector in a single window. The

rUserManager launches on the Injector when the “Start rUsers” command is issued from the Controller. Within

the manager, individual Windows sessions initiate and log in in accordance with the “Launch Delay” setting in

the “Start “rUsers” command (e.g. if the “Launch Delay” was set to 15, every 15 seconds an rUser session

would begin).

The rUserManager can be viewed throughout the test process, from the starting of rUsers through the

complete execution of the test. It can be minimized or maximized on the Injector as the test is in progress.

However closing the rUserManager during the test would end Scenario playback and disrupt the test.

To view the rUserManager, establish an RDP connection to the Injector server;

Once connected to the Injector, you will see the rUserManager with the started rUser sessions;



Page 33

Click the “Maximize” icon in the upper right corner of any rUser session to maximize that session within the

rUserManager;



Page 34

Click “Tile” in the “Windows” menu to minimize the individual session and return to the default rUserManager

view;



Page 35

Feel free to keep the RDP session active throughout the course of the test, as this will provide you with an

actual view of rUsers performing iterations of the test Scenarios in real time.

INJECTOR ACTIONS

MAKE INJECTOR “ACTIVE” OR “INACTIVE”

Inactive Injectors are not listed in the “rUser Status” pane of the “View Real Time Results” page. So, if you’re

running a test that requires the use of some, but not all of your Injectors, making the un-used Injectors

“Inactive” will keep them from cluttering the “Real Time Results” page without having to delete them.

To make an Injector “Inactive”, from the Actions column, click the “Active/Inactive” icon next to the desired

Injector;

Click “Yes” to confirm.

“Inactive” Injectors can easily be activated by simply clicking the same icon;

Click “Yes” to confirm.

DELETE INJECTOR

To delete an Injector, click the “Delete” icon next to the desired Injector;

Click “Yes” to confirm the deletion of the selected Injector.



Page 36

EDIT INJECTOR

To edit the properties of an Injector, click the “Edit” icon next to the desired Injector. “Injector rUsers” form

opens;

Name: A unique name identifying the Injector.

Host: The hostname or IP address of the computer hosting the Injector.

Domain: The Domain of the injector.

Location: An optional descriptive field to further identify the Injector

Resolution: Sets the resolution for the individual rUser sessions inside the rUser Manager. This value should

match the resolution of the machine on which the Scenarios were created. Recommended value is 1024x768.

(Note: Lower resolutions require fewer resources and allow for more rUsers per Injector)



Page 37

Color Depth: Sets the color depth for the individual rUser sessions inside the rUser Manager. This value

should match the color depth of the machine on which the Scenarios were created. Recommended value is

16bit.

Timeout: RDP connection time out.

Send Enter Key: Check if you wish to send <Enter> key upon login. This can be used to dismiss a security

message window that pops up after login.

Font Smoothing: When enabled, the rUserManager will activiate Cleartype font smoothing for rUser sessions.

Description: An optional descriptive field to further identify this Injector.

Click “Submit” to save your editing changes and update the Injector.

MANAGE INJECTOR RUSERS

To view or delete rUsers that have been created on an Injector, click the “Manage Injector rUsers” icon

next to the desired Injector. “Injector rUsers” form opens, displaying the list of created rUsers;

To delete all rUsers, click the “rUser Name” checkbox and all rUsers will be checked. To delete specific

rUsers, simply click the check box next to each rUsers to delete. Click “Delete Selected rUsers” button to

delete the rUsers from AppLoader.



Page 38

Click “OK” to confirm the rUser deletion.

Note that deleting rUsers completely is a two-step process. The above step deletes rUsers from AppLoader,

however the accounts still exist in Windows;

To delete rUsers from Windows server, open “Computer Management” snap-in and right-click on the user

accounts you wish to delete;

STOP RUSERS

Click the “Stop rUsers” button if you have a currently running test and you wish to stop Scenario

playback for all rUsers on the selected Injector.

DISCONNECT RUSER

Click the “Disconnect rUsers” button to disconnect the remote desktop sessions for all

rUsers on the selected Injectors. rUser manager on the injector machine will be terminated and will disconnect



Page 39

all rUsers from AppLoader Controller.

Note: Disconnecting users from the injector does not mean the users will be logged off.

ADD NEW INJECTOR

To add an Injector, run the installation program included in the AppLoader_Suite zip file.

SETUP

Setup – Manage users and groups in the AppLoader System. For example, adding more user, group, change

login password etc.

Click on User Management to see what Users and Groups the AppLoader Controller recognizes

Click on Users to see what users are in the system

User Groups - Click on User Groups to see the groups of users that have been created from the Manage

Injector section

Change Password – To change the password for the active users in the AppLoader Controller



Page 40

SEND SCENARIOS

Scenarios that were sent from ScenarioBuilder to the Controller, prior to the addition of a new Injector, must be

sent to the new Injector before they can be utilized by that Injector in a Test Plan. To send a Scenario, first

select the Injector to which you are sending the Scenario by clicking the check box next to it. To select all

Injectors, click the check box next to the Actions heading. Next, click the “Send Scenarios”

button in the Menu bar. The “Send Scenario to Injectors” form opens;

Select the Scenario to be sent from the “Scenario” drop down box and click “Submit”. Repeat this step for

each Scenario you wish to send. If the Scenario you are seeking is not listed in the drop down box, it most

likely has not yet been sent from ScenarioBuilder to AppLoader.

“Sent” Scenarios are added to the ScenarioStore folder on the Injector.

INJECTOR UTILITIES

The Injector Utils menu contains options to run remote command, kill process and log off all of your active

rUser on the selected Injector, click on the Injector Utils dropdown to do any of the following:

RUN REMOTE COMMAND

Run any windows command on all launched rUser sessions on the selected Injectors (e.g. notepad.exe).

KILL REMOTE PROCESS

Kill a running process on all launched rUser sessions on the selected Injectors.

LOG OFF RUSERS



Page 41

Log off all rUsers on the selected Injector(s).

RESTART RUSERS PROCESS

Restart rUsers on the selected Injector(s).

ADVANCED SETTINGS

Temporarily override the default settings from the Injector.ini file.

Click on Advanced Settings in Injector Utils menu to temporarily override the default settings from the

Injector.ini file:

Log Level: Recommended to be set to None.

Log File Name: (Optional) Specify a name of log file if Log Level is activated

Scenario Folder: Specify the path to where the Scenarios are saved on the injector

Send Block Status: Will notify the Controller whenever a Component starts/ends during playback.

Send Transaction: Will notify the Controller whenever a Transaction starts/ends during playback.

Send Screenshots: Screenshots will be taken and transferred to the Controller whenever Scenario playback

fails.

Check Interval: In milliseconds: This is the interval when checking for commands with Controller.



Page 42

Check Timeout: In milliseconds: This is the max time wait for response of commands issued from Controller.



Page 43

MANAGE BACKEND MONITORS

Backend monitors allow you to track the performance of your application or Injector

servers during the execution of your test. AppLoader provides standard

performance metrics - CPU, memory, disk (Windows) or file system (Linux &

Solaris), processes, services and network - which are pre-defined (not user-

defined). Monitoring of these performance metrics produces data regarding usage

of available capacity, which can be correlated with application response times to

give you a complete perspective of your application’s performance .

Monitoring backend systems is not necessary for all load tests (this step can be skipped if your load test does

not require the monitoring of backend systems).

ADD NEW SYSTEM

First, define the system you wish to monitor with AppLoader. Click the “Manage Backend Monitors” button in the left pane;

Click the “Add New System” button in the menu bar. The “New System” form opens;



Page 44

System Alias: Enter a meaningful name for the System in this field.



Page 45

Frequency: The check frequency for monitoring metrics for this System.

Connection Timeout: This is the amount of seconds AppLoader will keep trying to connect to the System

before it “times out” and reports no connection.

Profile: This determines the window during which monitoring occurs (i.e. 24/7).

System Type: Select the operating system type from the drop down box.

Connection Method: The method you will use to connect to this System. The options available depend on the

selection made for the “System Type”. Most of these connection methods are agent-less, however, for certain

system types, like Windows, an agent-based connection method is available. If you select an agent-based

connection method, then please ensure that the appropriate agent is already installed and running on the

target machine. For Oracle systems, please make sure that the appropriate client is installed on the machine to

be monitored.

Host name / IP: Enter the Hostname or Static IP address of the System. If the local system is being monitored

enter “localhost” or “127.0.0.1”. If the “Connection Method” is “Agent” and you are using non-default port, then

enter the system address as <hostname or ip address>:<port> (without the brackets). If you did not change

the default listening port (port: 9000) for the agent, you may enter just the server name or IP.

TNS Name: This parameter is only used for Oracle “System Types” (Oracle system 8.0 and previous versions),

and can be found in the TNSNames.Ora file (SQLPlus Client should be installed on the server being

monitored).

Login: Enter the login name. For Windows systems joined to domain, use: domain_name\username or

[email protected] (i.e. MYDOMAIN\john or [email protected]). For Windows systems not joined to

domain, use: hostname\username or username (i.e. 192.168.10.100\administrator or administrator). No Login

name is required for Windows if the host is “localhost” or “127.0.0.1”. For *nix systems the domain name is not

required.

Password: Enter the password for the username provided.

Monitor Top Processes: Select “Yes” if you wish to monitor the top processes by CPU or memory usage

(available in all operating systems except Windows 2000 and prior). If “Yes” is selected, AppLoader will

provide the top processes using most CPU and memory at any point of time on the CPU or memory graphs.

Metrics: Click the “Fetch Metrics” button in the left pane of the Metrics section;



Page 46

If login credentials have been configured properly, available Metrics will populate the left pane. Following is a

list of available Metrics:

CPU Metrics: CPU usage of the System is monitored. Usage of each CPU is listed separately, as well as total CPU usage. In the case of a single CPU system, CPU0 and Total CPU are equivalent.

Disk and File System Metrics: Disk and File System Metrics are conceptually the same test. Disk is for Windows and checks for available disk space usage on logical drives. File System is for Linux and Solaris.

Memory Metrics: The physical memory of the System is monitored.

Network Metrics: Packets sent and received across the network are monitored.

Process Metrics: Any running process on the System is monitored.

Service Metrics: Any running Service on the System is monitored.

Expand folders to reveal individual metrics. Drag and drop the metrics you wish to monitor from the left pane

onto the “System” icon in the right pane.



Page 47

Alternate method: Select (left click) the metrics you wish to monitor in the left pane, then click “Accept Selected

Metrics” button to add them to right pane. Use ctrl+left click to select multiple metrics prior to clicking “Accept

Selected Metrics”.

Selected metrics appear in the right pane under the System;

Description: Optional description field to further identify this System.

Complete the “New System” parameters and “Submit”.

TEMPLATES

Select from the Systems that you have set up, to create a custom Template that includes any or all of the

available metrics. A Template can then be attached to a Test Plan allowing for the correlation of backend

metrics with application response times via graphs and reports.

Click the “Templates” button in the menu bar. The “Templates” form opens;



Page 48

Template Name: Specify a meaningful name for a new Template or select an existing Template to modify from

the dropdown box.

With “System” displayed in the selection box, expand the defined Systems to reveal available metrics. Drag

and drop the desired metrics onto the Template icon (right pane) to add selected metrics to your Template.



Page 49

Use ctrl+left click to select multiple metrics to drag and drop from the left pane into the right pane all at once.

Click “Save” to save the Template and make it available to be attached to a Test Plan.

BACKEND MONITORS ACTIONS

Click the “Delete” icon next to a System to delete that System.

Click the “Edit” icon next to a System to edit the properties of that System.

Click the “Next” button to advance to the “Plan and Run Tests” page.



Page 50

PLAN AND RUN TESTS

Click the “Plan and Run Tests” button to design a Test Plan. Test Plans determine

how many rUsers are involved in the test, the pace at which rUsers are introduced

into the test, the number of iterations each rUser will perform, and the amount of

time that all rUsers will be concurrently active in the test. Most importantly, Test

Plans determine which Scenarios are performed and by which rUsers.

Once a Test Plan is created it can be run immediately. Be sure that rUsers have

been started (showing in “Connected” column of “View Real Time Results” page on

Controller, and sessions are visible in rUserManager on Injector) prior to running Test Plan.

The “Plan and Run Tests” page defaults to the “Test Plans List” view;



Page 51

CREATE NEW PLAN

Click the “Create New Plan” button in the menu bar. The “Test Plan” form opens;

The above image shows how the Test Plan form looks upon opening. Next, we’ll break down each section of

the form and provide an image with typical sample data entered, per section:



Page 52

DETAILS

Injector: Select the Injector on which the Scenario will play. After an Injector is selected, a new row appears

below it (add as many Injectors as necessary to complete your Test Plan). Select the same Injector multiple

times to play different Scenarios on the same Injector.

Scenario: Choose the Scenario to play on the selected Injector. If Scenario does not appear be sure that it

has been sent from ScenarioBuilder.

rUsers: Enter the number of rUsers to play the selected Scenario on the selected Injector.

Delay: The time, in seconds, that each rUser waits before playing the Scenario. A Delay value of "0" results in

all rUsers beginning simultaneously. A Delay value of "5" results in each rUser beginning 5 second after the

previous. A Delay value of "5+" results in the first rUser beginning 5 second after the test begins, and each

subsequent rUser 5 second after the previous. A Delay value of "1-5" results in each rUser beginning anytime

from 1 to 5 seconds after the previous (value chosen randomly by AppLoader).

Halt: Enable if you want rUsers to stop upon failure. Halted rUsers report a failure and exit the test. If disabled

(unchecked), rUsers will report a failure, but will advance to their next iteration after a failed iteration –

assuming the Scenario has an “On Failure” section which addresses the failure. For more about “On

Failure”, see the ScenarioBuilder User Guide.

TEST PLAN

Test Plan Name: Enter a name for the new Test Plan. Use standard alpha/numeric characters only.

Template: Attach a Template for backend monitoring (optional). Templates contain a set of system metrics



Page 53

configured in “Manage Backend Monitors”.

Steady State Time: The amount of time, in minutes, that all rUsers are actively playing Scenarios. Steady

State Time uses Scenario Reference Time to calculate the number of iterations per rUser. Test duration is

ruled by Iterations, not Steady State Time.

Ramp-up Time: The amount of time elapsed before all rUsers are actively playing Scenarios (rounded to the

nearest minute). This field is automatically calculated based on the Test Plan details provided (field cannot be

edited manually). Ramp-up time is the total time elapsed from when the first rUser starts playing a Scenario

until the last rUser starts playing. In other words it is the time from the initiation of the test to the point at which

the Steady State time begins.

Users / Min: The number of rUsers that begin playing Scenarios per minute. This field is automatically

calculated based on the Test Plan details provided (field cannot be edited manually). This is the maximum

number of rUsers injected, per minute during the Ramp-up time.

The Graph shown in this pane provides a visual representation of the Test Plan, including ramp-up time, ramp-

down time and total test run. This graphical view can also give you an idea of how many users are in the

system and testing at any given time during the test.

SCENARIOS

Scenario: Automatically populated (autofilled) from the “Scenario” field in the Test Plan “Details” section.

Duration (Min.): The length of time for a Scenario to complete an iteration. This value is automatically derived from the Scenario’s “Reference Time” (established in ScenarioBuilder). This field cannot be edited.

Iterations: Value calculated from the “Steady State Time”, “Ramp-up Time”, “Users / Min”, and “Reference Time”. Iteration value may be edited (double-click and enter new value), however, note that “Steady State Time” does not adjust to reflect an override to this value.

CSV FILES



Page 54


CSV File: Automatically populated field, shows the CSV (variables) file(s) attached to each Scenario.

Access Rule: Method by which values are retrieved from the CSV file:

Fixed Row Per rUser: One row per user for all iterations. Values are retrieved in row-order.

Sequential: Values are retrieved in row-order and used repeatedly as required to complete the total number of iterations.

Random: Values are retrieved and used repeatedly, without regard for order.

Unique: Values are retrieved in row-order and used one time only.

Start Line: The row in the CSV file from which the first value is retrieved. This setting is ignored when the Random "Access Rule" is used.

Rows / User: The number of rows from the CSV file assigned to each rUser per iteration. When a Loop Action is used to call a variable mutliple times per iteration, this value must be the same as the number of times through the loop to achieve the desired result.

RENDEZVOUS POINTS


Rendezvous Pt.: A Rendezvous Point is a step in the Scenario where rUsers hold until a defined number of

rUsers have arrived. This field is automatically populated based on Rendezvous Points defined in a Scenario.

To add Rendezvous Points, edit Scenarios in ScenarioBuilder and re-send to AppLoader.

Release Policy: The number of rUsers to be released, in unison, from the Rendezvous Point. Double-click this value to edit.

Timeout: The length of time (in seconds) that each rUser will wait for the Release Policy to be met. rUsers will

release either in accordance with the Release Policy or after the Timeout period elapses, whichever comes

first.

Delay: The length of time each rUser waits (after the previous rUser) before continuing from the Rendezvous

Point. Use to stagger the release of rUsers.



Page 55

SLA

This section allows you to set up Alerts for your backend servers’ behavior. The metrics in the Template attached to the Test Plan appear in the SLA section. Here, you can assign Alerts of varying degrees of severity tied to backend metrics performance.

Double-click on the “Operator” field to reveal the list of available Operators, then double-click on the “Value”

field to enter a threshold value.

Monitor: The individual backend metrics associated with the Template attached to this Test Plan.

Operator: The method by which the backend metric is compared to the threshold established om the Value” field.

Value: The threshold at which an Alert is triggered.

Alert Severity:

Red: Error

Yellow: Warning

Blue: Information

SAVE / REVIEW / RUN TEST PLAN

After completing all of the applicable sections of the Test Plan, click the “Save” button above the

“Details” window to save your Test Plan. The following message will appear:

Click “Yes” to open the “Execution Plan” form for viewing and editing;



Page 56

The Execution Plan will show you each rUser name, Host, assigned Scenario, assigned number of Iterations,

and Start Delay.

Click on icon beside each rUser to view the “Assigned CSV Rows” popup. CSV row assignment are

shown for each Scenario the rUser is playing.

Click on icon next to each Scenario listed in the “Assigned CSV Rows” popup window to view the

actual value which will be pulled from the CSV file for that rUser..

Close the Execution Plan window to continue. The following message will appear:

Click “Yes” to Run your test or click “No” to exit without running. If you clicked “Yes” the following window

opens. Specify a name for your load test and click “Submit.”



Page 57

AppLoader shifts to the “View Real Time Results” page automatically after submitting the test.

TEST PLANS LIST

Click the “Test Plans List” button to toggle from the Test Plan details page back to the Test

Plans List page.

Click the check box next to a Test Plan to select it, then click the “Run” button to execute it;

DELETE TEST PLAN

To delete a Test Plan, click the “Delete” icon next to the desired Test Plan;

Click “Yes” to confirm the deletion of the selected Test Plan.

EDIT TEST PLAN

To edit the properties of an existing Test Plan, click the “Edit” icon next to the desired Test Plan. The “Test



Page 58

Plan” form opens. Make editing changes and “Save” Test Plan.

CLONE TEST PLAN

To clone a Test Plan, click the “Clone” icon next to the desired Test Plan. A duplicate of the Test Plan will

be added to the “Test Plan List” page.



Page 59

VIEW REAL TIME RESULTS

When a test is in progress you can view summary or detailed results in real time

from the AppLoader Controller. AppLoader provides statistical and graphical

results on a general or detailed level. Graphical results are updated every minute.

See your rUsers status as they progress through connected, waiting, playing, and

completed stages. Click status icons to drill down to rUser by rUser up-to-the-

minute performance results, including where they are in their current Scenario and

screenshots of any errors they’ve encountered.

VIEW LOAD TEST STATUS - SUMMARY

Click the “View Real Time Resluts” Button. “Real Time Results” page opens in Summary view;

In the rUsers Status pane you get an overview of the rUsers’ status on each Injector;



Page 60

Below is a brief description of the Summary page columns and the information contained therein:

Connected: Shows the number of rUsers, by Injector, that are connected to AppLoader Controller and

ready to perform load test.

Disconnected: Shows the number of disconnected rUsers, by Injector. Disconnected rUsers may be a

result of limitations of the AppLoader license or limitations in resources on the injector machine.

Playing: Shows the number of rUsers that are playing the scenario on each Injector.

Waiting: Shows the number of rUsers that are waiting to play on each Injector.

Rendezvous: Shows the number of rUsers held at Rendezvous Points, waiting to be released in

accordance with the Test Plan Release Policy.

Completed: Shows the number of rUsers that have completed their test execution without any error or

failure.

Completed with Errors: Shows the number of rUsers that have completed their test execution with errors.

Click on the icon to see the error message and the screenshot for failed steps.

Total: Shows the total number of rUsers on each Injector.

In the Graph pane, you will see a real-time graphical representation including number of rUsers, number of

failures, and response times of each Scenario playing. The graph is updated every minute. Place your mouse

over any point on the graph to see the time and value for that occurance;



Page 61

VIEW LOAD TEST STATUS - DETAIL

Click any of the status icons to view details for each rUser. For example, clicking the “Playing”

icon will display details for rUsers currently playing Scenarios, including their Scenario, current Transaction

or Component, last Execution Time, Last Update, etc….

Details include:

Status: Icon indicates the current status or the rUser, such as Playing, Connected, etc.

rUser: Displays the name of the rUser.

Scenario: Displays the Scenario that the rUser is currently playing.

Trx/Blk/Sub-Scr: Displays the Transaction, Component, or Sub-Scenario that the rUser is currently playing.

Rdv Pt.: Displays the point at which the rUser is holding until the conditions of the “Release Policy” are met.

Iteration: Displays the number of iterations the rUser has performed.

Exec. Time (in seconds): Displays the Execution Time of the last completed iteration for that rUser.

Delay: Displays the difference between the Reference Time and the actual response time. Negative values

indicate that the actual response time is faster than the Reference Time; Positive values indicate that the

actual response time is slower than the Reference Time.

Last Message: Displays the latest update message for the rUser.

Last Update: Displays the time of the last update message for the rUser.

Click the “Summary” button to return from the “Details” view to the “Summary” view.

VIEW THE EXECUTION PLAN

Click “View Plan” button to see the current Test Plan details. How many rUsers are performing the

test? What Scenario(s) are being used in the test? How many iterations will each rUser execute? How soon

will each rUser begin their iterations? Which variables will be allocated to which rUsers?. Click here for a

detailed explanation of Execution Plan.



Page 62



Page 63

VIEW RUNTIME INFORMATION

Click the “Runtime Information” button to get a more detailed look at how rUsers are

performing on the Injector. This page groups errors together, providing screenshots of the errors. View

screenshot individually by rUser, or in a gallery where they are segregated and grouped by occurance,

allowing you to easily isolate trends and red flags in your application.

Test Start Time: Date and Time when the test started.

Total Time of Ramp-up: Time that it takes for all rUsers to enter the test.

Start Time of Peak: Time when all users are in system, this is the start of the Steady State Time.

Total rUsers in System: Total number of playing rUsers.

Elapsed Time: Time elapsed since the test began.

Ramp-up Policy: The number of users per minute, injected into the system.

Number of Failures: Number instances where steps either failed to execute or failed to get the expected

response from the application under test.



Page 64

TEST FAILURES / SCREENSHOTS

View screenshots of application slowness or errors occurring during the playback in real-time. The left pane

shows the count of failures per each unique failure (multiple occurances of the same failure are cataloged

under one screenshot). The right pane shows the screenshot associated with the failure selected. Click the

link in the screenshot pane to view the images in gallery mode .

ALERTS

View backend server Alerts, if any occurred. This section allows you to view high/medium/low severity behavior

of the application’s backend servers based on the settings you specified in the SLA section.

VIEW ALERTS

Click the “Alerts” button to view backend server Alerts in a popup window, if any occurred. This

section allows you to view high/medium/low severity behavior of the application’s backend servers based on

the settings you specified in the SLA section.

RESTART RUSERS

Click “Restart rUsers” button to restart finished and/or failed rUsers. After restarting, rUsers

start playing from the beginning of the Test Plan.



Page 65

END LOAD TEST

Click “End Load Test” to end the current test and save the reports. When clicking on End

Loadtest, please wait and make sure that the test has changed to or from in the “Test Analysis”

Page.

RESET

Click “Reset” button to stop test playback; remove disconnected Injector; delete any active load test;

reset the attributes of rUsers status.

VIEW RUN LOG

Click “Run Log” button to view load test execution logs from the first load test to the latest one.

EXPORT SUMMARY

Click “Export Summary” button to export currently running test results summary to CSV file.

Choose from rusers status or failure summary reports. Export it to CSV file and re-graph it using Microsoft

excel for report.



Page 66

TEST ANALYSIS

In this section you can analyze your current or previous test(s), get response time

graphs and reports, and export complete test data and graphs into excel format.

You can see the timing of your transactions, Scenarios, and how your users

performed after the test was completed. View standard reports.

TEST ANALYSIS PAGE LAYOUT

The Test Analysis page contains four panes: Load Test; Pie Chart; Stats; Graph.

LOAD TEST

The Load Test pane contains the test “tree” showing the current test. The tree can be expanded to drill down

through each Scenario, to the Component and Transaction levels. Also, the tree includes rUsers

and Failures . If a backend template was included in the test, the metrics associated with the template

would also be viewable here.



Page 67

PIE CHART, STATS AND MIXED DATA GRAPH

The Pie Chart pane displays success/failure rate of the current load test. The Stats pane containes summary

information including max number of users reached during the playback, total number of Scenario iterations

played, # of successes, # of failures, and total duration of the test. And the Mixed Data Graph allows you to

show response times, number of rUsers, failures and backend metrics for the load test in a single graph in any

combination that you desire.

Click on the overall test plan (top of the tree in the Load Test pane) to display the chart, stats and graph for the

test;



Page 68

Hold down ctrl on the keyboard while clicking on multiple items in the Load Test tree to select a combination of

mixed data to graph;

Then click the refresh icon to create a graph comparing the selected data;



Page 69

The graph’s “X” or horizontal axis displays the span of time over which the data was collected. The default

setting displays the full length of the test evenly proportioned in the graph pane. However, this setting can be

adjusted in the “Duration” field above the graph;

The “Y” or vertical axis displays digits reflecting seconds or measurement values depending on the metric

shown in the graph. In cases where mixed data is displayed, the “Y” value can represent a different type of

measurement, depending on the item graphed. For example, in the graph shown below, the green line and

orange lines, represent rUsers and Failures respectively. Therefore , the corresponding “Y” value would

represent a number (number or rUsers; number of Failures). However, the purple line represents the

Scenario, in which case the corresponding “Y” value represents the seconds taken for the Scenario to

complete.



Page 70

You can scroll to the next/previous hour by clicking on Next/ Previous buttons, reload the graph by

clicking on Refresh button, export the graphs to CSV file by clicking on Graph Export and toggle to

scale the graph during comparisons by clicking on Scale.

Click the “Analyze Current Test” button to see saved summary results (same as Runtime

Information found on “View Real Time Results” page).

SEARCH PREVIOUS TEST

Click the “Search Previous Tests” button to search for a previous test and view the

graphs and reports. “Search Load Tests” form pops up;



Page 71

Test Name: Enter the full or partial test name. Note that the string of characters you enter in this field must

match tests exactly, so use a short string or keyword if unsure of how tests were named.

From Date: Select a beginning date range to filter search results.

To Date: Select an ending date range to filter search results.

Use Archived data: Check this box if you wish to search from only tests which have been archived (if

unchecked, search will exclude archived tests).

Number of Results: Choose one of the pre-defined values to limit search results (5;10;15;20;30;50;75;100).

Recent Searches: Select from the list of recent searches to execute a previously defined search.

Clear Recent Searches: To clear the list of “Recent Searches”, click this button.

Complete the form and click “Search” to execute your search.

In the below example, the search should return up to 20 tests containing the word “Test” in the name;



Page 72

Here’s the result set;

Click on any one of the tests in the results window to display its test statistics, pie chart and performance graph

in the “Test Analysis” window;



Page 73

DELETE SELECTED TESTS

Select a test by clicking on it, then click the “Delete Selected Tests” icon to delete. Use Ctrl + click to

select multiple tests to delete at once.

ARCHIVE SELECTED TESTS

Archive selected tests to remove them from the load test database and store them in the archive database.

Use the “Delete” and “Archive” functions to remove tests from the “Load Test” pane and keep it manageable

and easy to navigate. Archived tests can be accessed at any time by simply searching previous tests and

checking the “Use Archived Data” box.

To archive, select a test by clicking on it, then click the “Archive Selected Tests” icon. Use Ctrl + click to

select multiple tests to archive at once.

To change the archived test folder location, open the chroniker.ini file with a text editor like notepad;

\Program Files\NrgGlobal\AppLoader\apps\includes\chroniker.ini

In the [database] section, modify the ArchivesFolder="C:\Program Files\NrgGlobal\AppLoader\archives" line;

ArchivesFolder="YOURFOLDER\archives"

Save the chroniker.ini file and close.



Page 74

ANALYZE SELECTED TEST

Click on any one of the tests in the results window and click the “Analyze Selected Test”

button. “Test Stats” for that test will open in a a new window (displayed in the same format as Runtime

Information form from “View Real Time Results” page).

EXPORT SELECTED LOAD TEST DATA

Click on any one of the tests in the results window and click the “Export Selected Load Test Data”

button to export raw test result data to a comma separated values file. Test data

includes Response Time and Delay (difference between Scenario “Reference Time” and actual “Response

Time”) by rUser, per Scenario, Transaction and Component.:

REPORTS

Click on any one of the tests in the results window and click the “Reports” button to expand the

Reports menu. Select one of the following pre-defined reports:

LOAD TEST SUMMARY REPORT

The Load Test Summary Report provides statistical analysis (minimum value, average value, maximum value,

standard deviation and 90th percentile), summarized at the Scenario, Component and Transaction levels. This



Page 75

test is a good indicator as to how good your test results are (grouped close to the average or spread out over a

wide range).

Min Value: The shortest amount of time it took to play the Scenario.

Avg. Value: The Average time taken to play the Scenario.

Max Value: The Maximum amount of time taken to play the Scenario.

Std. Dev.: The amount of how much variation exists from the average. A low Std. Dev. would mean the values

are close to the Avg Value. A high Std. Dev. would indicate the values are spread over a large range of values.

90th Percentile: Shows the results of rUsers in the 90% percentile.

rUsers: Number of users in the test.

Iterations: Number of iterations played in this Load Test.

Failures: Number of Failures in this Load Test.

Click the “PDF” button in the lower right hand corner of the report to create a pdf file for printing or

attaching to an email.

Click the “CSV” button in the lower right hand corner of the report to export the report to a comma

separated values file.

LOAD TEST FAILURE ANALYSIS REPORT

This report groups and summarizes failures, and shows the highest iteration at which a particular failure

occurred.



Page 76

Scenario Name: Name of the Scenario that was played.

rUsers: The number of rUsers performing the test.

Max Iteration #: The highest iteration at which a particular failure occurred.

# of Failures: The total number of times a particular failure occurred.

Error Message: The message describing where and why a particular failure occurred.





LOAD TEST ERRORS REPORT

This report lists errors by Scenario, by iteration.



Page 77

Scenario Name: Name of the Scenario that was played

Iteration #: The iteration at which a particular error occured

# of Failures: The number of times a particular failure occurred at a particular iteration.

Error Message: The message describing where and why a particular failure occurred.





GO TO CURRENT TEST

Click the “Go to Current Test” button to go back to current test. The pie chart, stats and

graphs for the current test will display on the “Test Analysis” page.

ANALYZE CURRENT TEST

Click the “Analyze Selected Test” button. “Test Stats” for the test will open in a a new

window (displayed in the same format as Runtime Information form from “View Real Time Results” page).

EXPORT LOAD TEST DATA

Click the “Export Load Test Data” button to export raw test result data to a comma

separated values file. Test data includes Response Time and Delay (difference between Scenario “Reference

Time” and actual “Response Time”) by rUser, per Scenario, Transaction and Component.



Page 78

CREATE SCENARIOS IN SCENARIOBUILDER TO SEND TO APPLOADER

With ScenarioBuilder, you can emulate real user activities, including mouse events, keyboard inputs, and

more. Here’s a sampling of the user Actions you can create in a test Scenario:

Type text, including function keys

Mouse moves and clicks

Move and resize windows

Find and click on an image (for Windows objects that don't have a "handle”, ScenarioBuilder’s image

Actions provide the flexibility to navigate to virtually anywhere on the desktop)

Define a specific image “Search Area” on the desktop

In addition to basic user Actions, ScenarioBuilder includes advanced conditional statements and variables, so

you can easily create complex Scenarios without any scripting:

If/Then/Else statements

Loops

Case/Switch statements

Variables retrieved from a CSV file allow you to change values with each Scenario execution

Determine Actions to take if your Scenario is interrupted by timeouts, pop-ups, or other failures

ScenarioBuilder’s tools and options provide flexible, user friendly functionality

Create Components to encapsulate Action sequences for reuse throughout the current, or in future

Scenarios

Record Actions in real time with Scenario Recorder

Edit and update recorded Scenarios in either “Graphical” or “Tree” views

For an overview of each Action group in ScenarioBuilder, go to Appendix A-1.



Page 79

LAUNCH SCENARIOBUILDER

When you launch ScenarioBuilder, a splash screen will appear;

Click “New Scenario” to create a brand new Scenario. Load an existing Scenario from the current Project by

clicking the link listed below the Project selection box. Select a different Project and the Scenarios contained

in that Project will be listed.

Projects organize Scenarios and their associated elements into six folders: scenarios, images, components,

favorites, variables and documents. Use Projects to keep related Scenarios together, and to share their

elements among each other.

For a comprehensive examination of ScenarioBuilder’s features, please see the ScenarioBuilder User Guide.



Page 80

CREATE A SIMPLE CLIENT/SERVER SCENARIO

In this Scenario, we will launch a desktop medical application, login with parameterized username/password,

and run a patient report.

In the Actions (left) pane, expand the Images group and double click the Action. Once the

Action is shown in the Scenario (middle) pane, it will automatically become highlighted. Right click on the “Find

Image #” field in the Properties (right) pane to show menu;

Notice in the lower right corner, the Description pane contains a brief description and/or tip for the highlighted

Action.

Choose “Capture Image”. ScenarioBuilder minimize to reveal the most recently launched window. The screen

capture tool hovers over the desktop;

Press the CTRL key once to enable the screen capture tool. Left click while dragging the mouse around the



Page 81

image you wish to capture, then release the click to capture the image. Enter a meaningful name for the

image. Saved image will appear in the Properties pane;

Click the “Mouse Click” child Action below the “Click on Image” Action;

Change the “Click Type” property to “Doubleclick”;

Click “Play Scenario from Start” icon and ScenarioBuilder will execute the step, double-clicking the

desktop shortcut and opening the “Patient Manager” login window;

To ensure that the “Username” field has cursor focus, use the “Click on Image” action to click into the



Page 82

Username field. Capture only the name of the field (not the field itself);

Click the “Mouse Move” child Action below the “Click on Image” Action;

Click the “Mouse Coordinates” selection icon in the properties window;

With the tool enabled, click into the Username field on the “Patient Manager” login window;

The “Mouse Move” Action will locate the mouse relative to the captured image. This allows us to use an

image as an anchor, but not necessarily a destination for a mouse click. In this example, the “Username”

field could contain a blinking cursor or cached data, which would prevent it from being recognized by a “Find

Image” Action. Whereas, the field’s name, “Username” is static (it will consistently render the same). By

capturing the field name and using the “Mouse Move”, we ensure that our Scenario will succeed regardless of

what residual data or cursor blinking may occur inside the field.

Let’s now parameterize the login.

Click on the CSV tool in the ScenarioBuilder menu bar. Enter a name for your file and click .

Create a file containing column headers in the first row. These are the variable names. The subsequent rows

are for the data values. Click on the following image to view a sample CSV file.



Page 83

users.csv

Save the CSV file and it will automatically be stored in your Project’s “Variables” folder.

To link the file to your Scenario, with the Scenario name row highlighted, click the icon in the “Variables

File Name” properties field and browse to the desired CSV file;

To pull variable data from the CSV file into the form use a “Type Text” Action. Expand group and

double click on Action. Right click on Text property field in the Properties pane and select the

“username” variable (notice that variables are framed with percentage (%) signs);

Double click on and select Tab key.

Double click on again and select the “password” variable.



Page 84

Double click on and select Enter key. Your Scenario should look like below at this point;

With this script, the application can now launch a parameterized user login;

Let’s add a “Transaction” to measure the login time of the application. Transactions mark a starting and ending

point, between which are any number of Actions. The Transaction will total the execution time of its framed

Actions.

Expand group and double click on to mark the start point of the Transaction.

Provide a meaningful name for the Transaction in the name field of the Properties pane.

Next, we’ll double click and capture an image of the icon in the Folders window of

the application. This success of this Action will serve two purposes, as it will expand the Reports tree (we want

to run a report for our test) as well as provide evidence that the application has loaded.

Now, double-click on and choose the same Transaction we just created from the “Transaction

Name” field in the Properties pane.

To recap, we’ve created a Transaction that measure the amount of time that will elapse from submitting the

login request to the launch of the application, which is verified when the “Click on Image” Action finds the

“Reports” link.

Moving along, double-click on and capture an image of the link.



Page 85

This loades the Patient Daily Report page;

Double click on and capture an image of the field.

Double click on and type the name of the patient on which to report (you may use a CSV file to

parameterize patient names if necessary).

Double click on and select “Tab” key to tab over to the “Date” field.

Let’s populate the date field with last month’s date. Expand group and double click on

. In the Properties pane, provide a meaningful “Variable Name” (e.g. “lastMonth”). Choose a

“Variable Value” from the dropdown. The dropdown includes an assortment of predefined variables (date,

time, random numbers, and various environment parameters). (Note: you may modify date fields by adding -/+

signs followed by an integer. For example, MM-1/dd/yyyy will result in last month’s date).

Next, double-click on , right click on the text field and select “%lastMonth%” variable.

Lastly, double-click on and select “Enter”, which in this case will launch the report.



Page 86

Our completed Scenario looks like below;

Save the Scenario. Now click on File Send Scenario to Controller (alternately, you can click on the “Send”

icon in the menu bar) to send the Scenario to the AppLoader Controller where it can now be used in a

Load Test.



Page 87

CREATE A SIMPLE WEB SCENARIO

In this Scenario we will launch Amazon.com, click on the “Books” menu and search for a book.

Expand group and double click on . This Action opens a file/application with

its associated program. Type the website name www.amazon.com.

Now let’s check that the home page has loaded successfully before going into the books menu. If the books

menu is on the homepage, then we can use it as a check and click on it at the same time.

Expand the “Images” group and double click on Action. Right click on the “Find Image #” field in

the Properties pane to show menu;

Select “Capture Image”. Press [CTRL] key to enable the screen capture tool;

http://www.amazon.com/



Page 88

Click and hold while dragging the mouse around the “Books” link on the Amazon page, then release. Enter a

meaningful name for the image.

Once you click the “Books” link, a sub menu appears;



Page 89

Double-click on and choose . A new page loads;

Double click on and capture the heading. The “Advanced Search” form opens.

To ensure that the page loads successfully before our Scenario attempts to type in search date, we’ll capture

the image, and use a “Mouse Move” action to drop our cursor in the “Title” field.

Double-click on and capture ;

Click on the child Action beneath the “booksearch” image. In the Properties pane,

click the selection icon;



Page 90

Scenario Builder minimizes and the mouse changes to + shape. Click inside the “Title” text field.

Notice, now the (x, y) coordinates are populated with (-4,133) pixels. Again, this is a case where an image is

an anchor, but not a destination for a mouse click;

Let’s parameterize our search so that we search for a different book each time.

Click on the CSV tool in the ScenarioBuilder menu bar. Enter a name for your file and click .

Create a file containing one column header called “bookname”. This is the name of our variable. Add a

number of book names below the column header and Save the CSV file. It will automatically be stored in your

Project’s “Variables” folder.

To link the file to your Scenario, with the Scenario name row highlighted, click the icon in the “Variables

File Name” properties field and browse to the CSV file.

To pull variable data from the CSV file into the form use a “Type Text” Action. Expand group and

double click on Action. Right click on Text property field in the Properties pane and select the

“bookname” variable;



Page 91

Double click on and select “Enter” key to initiate the search. A page is loaded with many books

related to the title entered from the CSV file.

Finally, we’ll add a Transaction to measure the amount of time take to complete the search. Expand

group and double click on to establish the Transaction starting point. Enter a

meaningful name for the Transaction in the Properties pane.

Double click on and capture (since this image will appear upon the completion of every

search).

Now, double-click on and choose the same Transaction we just created from the “Transaction

Name” field in the Properties pane.

Our completed Scenario looks like this;

Save the Scenario. Now click on File Send Scenario to Controller (alternately, you can click on the “Send”

icon in the menu bar) to send the Scenario to the AppLoader Controller where it can now be used in a

Load Test.



Page 92

APPENDIX A-1: SCENARIOBUILDER ACTION GROUPS

ScenarioBuilder segregates Actions under group headings. Here is a brief explanation about each Action

group and the types of Actions contained therein:

Application: Launch application (web, desktop, client/server, RDP, or custom application); establish

Transactions to record response times for specifiied events; define Components to organize Actions into

manageable, exportable blocks; run commands from the Windows “Command Line”; call (nest) another

Scenario.

Images: Find and click on images. An image is any collection of pixels captured with ScenarioBuilder’s screen

capture tool. Images can be used to keep rUsers synchronized with the application under test by finding icons,

logos or buttons and thus ensuring that pages have loaded prior to executing keyboard or mouse Actions.

(Note that if an image is hidden on the rUser desktop, ScenarioBuilder may not be able to find or click on that

image.)

Keyboard: Trigger keystrokes and function keys (including function key combinations e.g. Ctrl+Alt+Del).

Mouse: Trigger mouse clicks and mouse moves.

Window: Wait for, close, maximize, minimize or resize an active window on the desktop. Window Actions are

resource intensive and should be used with discretion (if the same result can be accomplished with another

Action, use the other Action).

Conditionals and Variables: Establish pre-defined variables (e.g. date, random numbers, etc.); increment

variables; create IF/Then/Else and Switch/Case statements; insert rendezvous points, add “On Failure”

Actions.

Miscellaneous: A handful of less commonly used Actions.

Favorites: A user-defined group of Actions. Actions added to Favorites will hold their established properties.

AppLoader User Guide

Technology

Transcript of AppLoader User Guide