Introduction to Mobile Performance Testing
description
Transcript of Introduction to Mobile Performance Testing
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Introduction to Mobile Performance TestingWilson Mar, Angel Tran, Shlomi SalmaHOL330 in Sands 305, June 13, 2013
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2
Mobile apps are now:Why performance test mobile apps?
• how customers locate stores and information• how brands get exposure many times a day• the “face” of the organization
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3
Slow app = Slow product?
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4
Baseline vs. impact of network latency, etc.
[P01]from HP Network Virtualization by Shunra
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5
Possible points of risk in landscape
StorageDB
ServersApp
ServersWeb
ServersLoad
Balancers
3rd Party Cloud Services
CDNs & CoLo.
Carriers InternetBrowsers
Platforms& Versions Firewalls
VPNsDevice Features
2G,3G,4GCellular
Mobile Components
ScreenSizes
Web Services
LANs
ISPs
DB Indexes& Logs
ResourceUsage
Latency& Loss
Pooling &Threading
ServerConfig.
SysAdmin: DBA:Back-end:Dev:Network:DataCaching
mDev:Encrypt,Compress.
Load Bal. Algorithm
Network:Network:
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6
HP Live Network
Collaborate2
HP LoadRunner components and workflow
Virtualize Network5
End User Application Server(s)Internet/WAN Database
Analyze, Compare, Report8
Client(s)
Generate Load6
Author Scripts1
Measure Impact7
Test Scenario
Plan and Execute Tests3
Virtualize Services4
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7
Scan for more information
LoadRunner
NetworkVirtualization
ServiceVirtualization
MobileTesting
HP Codescan app
http://www.hp.com/go/...
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8
AppleApp
Store
Play.googl
e.com
iPhone, iPadAndroid“featurephones”
WindowsPC
Mobile technology terms
Safari browser
URL
m.ft.com
mobilesite
ft.com/wsweb
services
ft.comHTML5
responsive design
Native app
internetexplore
r
ft.comHTML4
URL
minimal
browserURL
Hybridapp
Chromebrowser
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile LoadRunner Scripting Demos
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10
http://code.google.com/p/android-screen-monitor/Android Screen Monitor
Download to folder …\sdk\platform-tools\ASM.jarWithin Windows Explorer, double click on ASM.jar
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11
Create a New Script, Mobile Category
HP LoadRunner VuGen 11.5 SP1+
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12
Multiple Approaches Available
Browser or hybrid app:Ajax? > TruClient Ajax Mobile scripting protocol using User Agent
spoofing in FirefoxSimple > Mobile HTTP/HTML scripting protocol
Native app: > Mobile HTTP/HTML scripting protocolAndroid:
No device > Record Emulator using Android Virtual Device (AVD)
Have rooted device > Mobile Recorder generate lrpcap to Analyze
iOS & others:> LoadRunner Proxy on same subnet/Wi-Fi hub
(the newest technique) > Record & Analyze WinPcap capture on the
mobile app server
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.13
Recording Method TruClient Ajax
TruClient Ajax HP LoadRunner VuGen 11.51+
App web server
User Agent in Firefox >
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14
TruClient recording
HP LoadRunner VuGen 11.51+
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15
Devices with User Agent strings
Count: 27
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16
Sample User Agent for Samsung Galaxy S3User Agent string example
Mozilla/5.0 (Linux; U; Android 4.0; xx-xx; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
HP LoadRunner VuGen11.52More at http://www.useragentstring.com/
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17
Android Versions
4.1 (16)4.2 (17)
2.3 (9,10)
4.0 (15)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.18
Mobile Device Manager Tool in VuGen
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.19
VuGen Scripting Process
from HP Virtual User Generator User Guide 11.52 page 73
1. Configure recording settings2. Consider run-time settings3. Object identification using
script levels4. Specify Transaction Names 5. Specify Completion End
Events Logic6. Insert flow control7. Run-time logic8. Vary data based on
parameters9. Using C script functions in a
library
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.20
Random Think-time in TruClient
Arguments:
http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/
Math.floor( 11 * Math.random()+5 );
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21
TruClient Toolbox drag and drop
window.alert( “randWait=” + randWait );
Arguments:
http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/
Math.floor( 11 * Math.random()+5 );
var randWait =
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Mobile Recording
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23
Mobile HTTP vs. HTML
HP LoadRunner VuGen
mode=“HTML”mode=“HTTP”Parse HTML to automatically identify additional resources to download
For use with web services calls.Does not parse for resource requests.
• CSS• JavaScript
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.24
Record Mobile HTTP/HTML
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.25
HTTP/HTML
TruClient
HTTP/HTML Recording Methods Wizard
Proxy Recordi
ng
Record Emulato
r
AnalyzeTraffic
Record &
Analyze Traffic
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Proxy Recording
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.27
HTTP/HTML
TruClient
HTTP/HTML Recording Methods Wizard
Proxy Recordi
ng
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.28
Proxy Recording
Android
LoadRunner
Proxy
HTTP/HTML
Proxy Recordi
ng
HP LoadRunner VuGen11.5x
App web server
Port 80/443
VuGenProxy IP:Port 8080
APK
iOS IPA
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.30
Proxy Recording
HP LoadRunner VuGen11.52
HTTP/HTML
Proxy Recordi
ng
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.31
Proxy Recording Setup on iOS
1.
Source: http://www.amsys.co.uk/2012/blog/how-to-setup-proxy-servers-in-ios/
4.3. Scroll down2.
Eleanor
Password: ncjm-q3ey-t1dd
Eleanor 192.168.1.2255.255.255.0
192.168.1.1
192.168.1.38080
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.32
Proxy Recording Setup on Android Galaxy S3 (4.1.2)1. 4.3.2. Long press
Eleanorncjm-q3ey-t1dd
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record Emulator
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.34
HTTP/HTML
HTTP/HTML Recording Methods Wizard
Record Emulato
r
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.35
Record Emulator Android Recording Method
Android
deviceemulator
HTTP/HTML
Record Emulato
r
APK
HP LoadRunner VuGen11.5x
App web server
Async.
APK
@device ADT file
Android… \sdk\tools
File manager
Backup
Play.Google.com
Install
Copy
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.36
Record Emulator Requirements
1. USB cable to device2. USB driver for device 3. Android app APK file for Android app under test4. Android file download manager app to extract APK file from device
5. JDK (Java Development Kit)6. ADT (Android Development Toolkit) plug-in for Eclipse7. AVD (Android Virtual Device) within Eclipse for device under test8. JAVA_HOME and PATH Windows environment variables
9. HP LoadRunner 11.50 and SP2 patch to upgrade to 11.52
HTTP/HTML
Record Emulato
r
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.37
Emulator
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record & Analyze Traffic
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.39
HTTP/HTML
HTTP/HTML Recording Methods Wizard
Record &
Analyze Traffic
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.40
Record & Analyze Traffic
Androidnetworksniffer
iOS
HTTP/HTMLRecord
& Analyze Traffic
Ad hoc network
.cap,.pcap
HP LoadRunner VuGen11.5x
App web server
APK
IPA
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Analyze Traffic
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.42
HTTP/HTML
HTTP/HTML Recording Methods Wizard
AnalyzeTraffic
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.43
Analyze Traffic
MobileRecorder
Android
.lrpcap
HTTP/HTML
AnalyzeTraffic
Android 4.1+(rooted)
App web server
HP LoadRunner VuGen11.52
APK
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.44
Android Mobile Recorder1. Start Recording 3. Stop and send email2.
Perform user steps.
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.45
Android Mobile Recorder Settings
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.46
Recording Methods Recap
MobileRecorder
Android
.lrpcap
LoadRunner
Proxy
deviceemulator
networksniffer
iOSVuGenProxy IP:Port 8080
HTTP/HTML
Proxy Recordi
ng
Record Emulato
r
AnalyzeTraffic
Record &
Analyze Traffic
Port 80/443
User Agent in Firefox >
Android 4.1+(rooted)
Ad hoc network
APK
APK
.cap,.pcap
TruClient Ajax HP LoadRunner VuGen11.52
App web server
@device ADT file IPA
Play.Google.com
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile scenarios
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.48
How can we help?
Run test cases
Provision & deploy environment
Execute automated build
Application Security Assessment
Test results
Security Code ScanFortify Source Code Analysis
UFT / BPTLoadRunner
WebInspect
Continuous Delivery Automation (CDA)
(Jenkins)
HP integrated testing suite
ALM Analysis
Executive Scorecard
Performance Center
Service VirtualizationNetwork Virtualization
DevelopmentAgile Manager
Business Service ManagementMonitoring &
Diagnostics
Analytics
App. Lifecycle Intelligence
ALM Lab Management
Tear down environmentContinuous Delivery Automation
(CDA)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.49
Shopping processInterconnected component services
PayBuy Ship Track
Product
Search
Customer profiles Payment Shipping
Composite
Managed cloud
Public cloud
Traditional
Track package
Privatecloud
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.50
Concurrency and rendezvous
[P10]Illustrative data
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.51
Key take-aways
• For mobile browser apps (jQuery Mobile, Sencha, PhoneGap, etc.), little scripting is needed using User Agent spoofing via Firefox TruClient Mobile
• JavaScript executes against a DOM for each virtual user• Traffic captured between native apps and server are converted to
VuGen scripts• Android native app traffic on rooted devices can be recorded
on device using LoadRunner Mobile Recorder
• iPhone and iPad (iOS) -- all non-rooted apps – can be recorded using LoadRunner Proxy
• Monitor and Diagnose using HP Business Service Management• Consider HP Network Virtualization and Services Virtualization
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.52
For more information
Attend these sessions
Mobile Performance Hands-on Lab HOL3330Sands Room 305
• Tuesday 1p – 3p• Thursday 3p-5p
After the event
• Contact your sales rep• Visit: http://
www.hp.com/go/mobiletesting http://www.hp.com/go/loadrunner
Your feedback is important to us. Please take a few minutes to complete the session survey.
Visit these demos
Applications: mobile performance testing RT3112• Wednesday June 12, 1030 am
Lido 3101A• Thursday June 13, 930 am
Lido 3101B
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you