SMP Enterprise Scale Mobility Webinar Series · SAP Mobile Platform Testing Mostly Online...
Transcript of SMP Enterprise Scale Mobility Webinar Series · SAP Mobile Platform Testing Mostly Online...
John Polus / SAP Mobile RIG
August, 2013
Load Testing Mostly Online Applications
SMP Enterprise Scale Mobility – Webinar Series
Brought to you by the SAP Mobile Rapid Innovation Group (RIG)
© 2013 SAP AG. All rights reserved. 2 Public
SAP Mobile Platform: Enterprise Grade Mobility
On SCN pages and a series of
webinars, we drill down into Enterprise
Readiness aspects of the SAP Mobile
Platform (SMP).
• On the SCN Pages, you find links to
White Papers, How-To Guides, Blogs
and other resources.
http://scn.sap.com/docs/DOC-43424
• Webinars complement these
published resources. The Webinar
schedule is also published on SCN.
http://scn.sap.com/docs/DOC-43425
Load Testing Mostly Online Applications
SMP Enterprise Grade Mobility – Webinar Series
© 2013 SAP AG. All rights reserved. 4 Public
Agenda
1. Introduction
2. Script Generation
3. Configuration & Parameterization
4. Compile and Run Test
5. More Information
© 2013 SAP AG. All rights reserved. 5 Public
SAP Mobile Platform
Testing Mostly Online Applications
This presentation ‘Testing Mostly Online Applications’ provides you with the
understanding of the tools and process steps to:
1. Create LoadRunner Scripts for SUP load testing with Hybrid Web Container
(HWC) and Open Data Protocol (ODP) apps.
2. Configure & parameterize the LoadRunner scripts for multi-user testing
3. Run the tests
Script Generation Configuration &
Parameterization Compile and Run Test
© 2013 SAP AG. All rights reserved. 6 Public
Load Testing with HP LoadRunner
Our tool of choice for load testing mobile apps is LoadRunner from HP:
• Load testing before deployment can provide confidence that problems will be avoided during mobile application
rollout
• LoadRunner is a robust framework for high volume server load testing
• Enterprises that already have experience with LoadRunner can utilize their existing expertise when load testing
SUP
LoadRunner Extension for the Sybase Unwired Platform
An available extension supports LoadRunner-based load testing to simulate clients:
1. Hybrid Web Container (HWC)
2. Open Data Protocol (ODP) via Messaging Channel and the OData Proxy
The extension has been developed and certified for use with LoadRunner versions 11 and 11.5.
Note: Only available with SUP 2.2 and above
Licensing
To use the extension, a LoadRunner license with the C# script type (Microsoft .NET protocol) is required.
A 10 day trial version supports the C# scripts but with a limit of 25 virtual users.
© 2013 SAP AG. All rights reserved. 7 Public
Agenda
1. Introduction
2. Script Generation
3. Configuration & Parameterization
4. Compile and Run Test
5. More Information
© 2013 SAP AG. All rights reserved. 8 Public
Offline Process overview for Loadrunner
Script Generation
Record Application
& Generate Code
Create & Save Script
Replace Files
Configuration & Parameterization
Edit Runtime Settings & Run Vuser
Connection Properties
Multi User Testing
Example Scripts
Additional Parameterization
Compile and Run Test
Adding References
LoadRunner 11.0 Compatibility
Create Scenario & Add Vuser Script
Run Single &
Multi User Test
Troubleshooting
© 2013 SAP AG. All rights reserved. 9 Public
Script Generation
Record the Application & Generate Code (1/2)
In this process step, you create a separate recording for each application connection.
Recording Location: UnwiredServerDir\logs\XMLRecording
Code Generation
• Reads the recording file for the application connection and generates a LoadRunner C# test script that simulates the device application
• Two Generators:
• vugen-hwc – for HWC applications
• vugen-odp – for ODP applications
Runtime Library
• Provides additional APIs for use by LoadRunner C# scripts
• These APIs are referenced by the generated C# scripts
• Primary APIs:
• SAP.Mobile.LoadRunner.HwcApi for HWC applications
• SAP.Mobile.LoadRunner.OdpApi for OData/ODP applications
• API Documentation
• UnwiredServerDir\LoadRunnerAPI\api\index.html
© 2013 SAP AG. All rights reserved. 10 Public
Script Generation
Record the Application & Generate Code (2/2)
Start Recording
Run Application
Stop Recording
Generate Code
© 2013 SAP AG. All rights reserved. 11 Public
In the HP Virtual User Generator, create a new Single Protocol Script.
Script Category: Microsoft .Net.
Script Generation
Create & Save Script (1/2)
Important: Cancel the dialog
‘Start Recording’
© 2013 SAP AG. All rights reserved. 12 Public
Script Generation
Create & Save Script (2/2)
Save the script in the LoadRunner directory
© 2013 SAP AG. All rights reserved. 13 Public
Script Generation
Replace Files
Saving a LoadRunner script creates a directory for the script,
e.g.”c:\LoadRunner\scripts\my-test”.
• Close Virtual user generator
• Copy the C# files that were generated by the virtual user generator from the server
machine into the scripts directory.
This overwrites LoadRunner generated vuser_init.cs, Action.cs and vuser_end.cs
• Open the virtual user generator
Note: SAP and HP are working together to streamline the integration in a future release so that
session recording and script generation can be accomplished directly by LoadRunner Virtual
User Generator
© 2013 SAP AG. All rights reserved. 14 Public
Agenda
1. Introduction
2. Script Generation
3. Configuration & Parameterization
4. Compile and Run Test
5. More Information
© 2013 SAP AG. All rights reserved. 15 Public
Offline Process overview for Loadrunner
Script Generation
Record Application
& Generate Code
Create & Save Script
Replace Files
Configuration & Parameterization
Edit runtime Settings & Run Vuser
Connection Properties
Multi User Testing
Example Scripts
Additional Parameterization
Compile and Run Test
Adding References
LoadRunner 11.0 Compatibility
Create Scenario & Add Vuser Script
Run Single &
Multi User Test
Troubleshooting
© 2013 SAP AG. All rights reserved. 16 Public
Configuration & Parameterization
Edit Runtime Settings & Run VUser
As a Process
• Must be used for HWC and
ODP using the Messaging
Chanel
As a Thread
• Only for ODP using the HTP
Channel ( More efficient than
running as a process. Uses
less RAM on the LoadRunner
Load Generator Machines)
Edit Runtime Settings Run VUser
© 2013 SAP AG. All rights reserved. 17 Public
Configuration & Parameterization
Connection Properties
© 2013 SAP AG. All rights reserved. 18 Public
Configuration & Parameterization
Multi User Testing
The generated script is based on a single user recording, so it may be inappropriate for running
a multi-user test
When the test is run from the LoadRunner Controller with multiple virtual users, multiple virtual
users my try and create, update or delete the same data object which could cause resource
contention or operational failure
It is the testers responsibility to Parameterize the generated script using the various LoadRunner
parameterization techniques.
In general this means changing constant strings into variable references or other expressions.
© 2013 SAP AG. All rights reserved. 19 Public
Configuration & Parameterization
Example Scripts (1/2): Creates Customers with the same Name
© 2013 SAP AG. All rights reserved. 20 Public
Configuration & Parameterization
Example Scripts (2/2): Creates Customers with the new Name
© 2013 SAP AG. All rights reserved. 21 Public
Configuration & Parameterization
Additional Parameterization
Other helpful techniques for parameterization
• Loading parameter values from a data file
(e.g. using standard LoadRunner lr.eval_xxx API calls)
• Saving the result of one request to be used to extract values that are passed as parameters to
a subsequent request.
Note that for HWC / ODP code generation, each online request result is saved into a class field
so it may be conveniently referenced when formulating subsequent requests.
Due to the inherent complexity of HWC and ODP requests, typically returning complex (nested)
structures and lists, it is not usually possible for the code generator to intelligently auto-wire such
correlation parameters.
That is left to the responsibility of the test author.
© 2013 SAP AG. All rights reserved. 22 Public
Agenda
1. Introduction
2. Script Generation
3. Configuration & Parameterization
4. Compile and Run Test
5. More Information
© 2013 SAP AG. All rights reserved. 23 Public
Offline Process overview for Loadrunner
Script Generation
Record Application
& Generate Code
Create & Save Script
Replace Files
Configuration & Parameterization
Edit runtime Settings & Run Vuser
Connection Properties
Multi User Testing
Example Scripts
Additional Parameterization
Compile and Run Test
Adding References
LoadRunner 11.0 Compatibility
Create Scenario & Add Vuser Script
Run Single &
Multi User Test
Troubleshooting
© 2013 SAP AG. All rights reserved. 24 Public
Compile & Run Test
Adding References
Adding Link-Time References
To ready the generated script, which is based on a single user recording, for multi user testing:
• Open the LoadRunner generated C# project (Script.csproj) in Visual Studio
• Add reference to SAP.Mobile.LoadRunner.dll from the extension lib library, e.g.
Sybase\UnwiredPlatform\Server\UnwiredServer\LoadRunnerAPI\lib.
Alternative: edit the Script.csproj file.
• Save the project and exit Visual Studio
Adding Runtime References
The Sap.Mobile.LoadRunner.dll has dependencies on other DLLs to execute your LoadRunner Script.
• Copy DLLs from the SUP Mobile SDK Win32 subdirectory (e.g. \Sybase\UnwiredPlatform
\MobileSDK22\Win32 ) to the “bin” subdirectory of your LoadRunner script directory (e.g.
\LoadRunner\scripts\my-test\bin)
© 2013 SAP AG. All rights reserved. 25 Public
Compile & Run Test
Compatibility and Workaround: LoadRunner 11.0
Version compatibility and workaround:
Visual Studio & LoadRunner 11.0
• Situation: Visual Studio 2008 automatically upgrades the C# project when opening
the Script.csproj file, making it unusable for LoadRunner 11.0 (LoadRunner
invokes incompatible vs. 2.0 of “msbuild” tool).
• Workaround: Edit the Script.sln file with a text editor and change the Format
Version to 9.00
© 2013 SAP AG. All rights reserved. 26 Public
Compile & Run Test
Create Scenario and add Vuser Script
Launch the LoadRunner controller, create a Scenario & add the VUser script
1. Open the Runtime settings
i. Set Miscellaneous -> Run VUser as a process". Running as a thread will NOT work.
ii. Uncheck Miscellaneous - >Define each action as a transaction.
iii. In General->Log, check ‘Enable Logging’ and select ‘Always Send Messages’.
iv. In General->Think Time, select ‘Replay Think Time”, use random percentage of recorded think
time, and set the percentage.
v. To run a certain number of iterations for each VUser, set the number in General ->Pacing-
>Number of iterations.
vi. Configure the scheduler for test
2. Specify the number of Vusers and how quickly to ramp up to the required value
3. Specify the duration of the test by clicking on Duration.
To run a certain number of iterations (configured in runtime settings) instead of certain duration, set
the Duration to Run until completion.
© 2013 SAP AG. All rights reserved. 27 Public
Compile & Run Test
Run Single & Multi User Test
• Compile the script from the Virtual User Generator
• Preform a single user test run from he Virtual User Generator to ensure the
script is functioning correctly
• Once a single user test is successful, you are ready to run the load test using
the LoadRunner Controller
© 2013 SAP AG. All rights reserved. 28 Public
Compile & Run Test
Troubleshooting
• Reduce the number of VUsers
to avoid overloading the Load
Generator host.
• Enable detailed logging
• Use “Process per VUser”
(gives you one log file per
process) when doing detailed
logging to avoid difficulty in
interpreting the log file contents
© 2013 SAP AG. All rights reserved. 29 Public
Agenda
1. Introduction
2. Script Generation
3. Configuration & Parameterization
4. Compile and Run Test
5. More Information
© 2013 SAP AG. All rights reserved. 30 Public
More Info on Testing Mobile Applications
SAP Infocentrer Documentation
Path: Sybase Unwired Platform 2.2 SP04 System Administration 2.2 SP04 Performance Tuning and Testing
Direct Link:
http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01205.0224/doc/html/den1345124609233.html
SAP Community Network:
SAP Mobile Platform - Load Testing: Overview & Resources: http://scn.sap.com/docs/DOC-40850
Load Testing Introduction Video: http://scn.sap.com/docs/DOC-40963
Load Testing Mostly On-line Applications: http://scn.sap.com/docs/DOC-40965
Load Testing Mostly Off-line Applications: http://scn.sap.com/docs/DOC-40964
© 2013 SAP AG. All rights reserved.
© 2013 SAP AG. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft Corporation.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation.
Linux is the registered trademark of Linus Torvalds in the United States and other countries.
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries.
Oracle and Java are registered trademarks of Oracle and its affiliates.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems Inc.
HTML, XML, XHTML, and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.
Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc.
IOS is a registered trademark of Cisco Systems Inc.
RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited.
Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik and Android are trademarks or registered trademarks of Google Inc.
INTERMEC is a registered trademark of Intermec Technologies Corporation.
Wi-Fi is a registered trademark of Wi-Fi Alliance.
Bluetooth is a registered trademark of Bluetooth SIG Inc.
Motorola is a registered trademark of Motorola Trademark Holdings LLC.
Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH.
SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.
Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company.
Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company.
Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company.
All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.
The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.