Testing Ajax, Mobile Apps the Agile Way
-
Upload
appvance-inc -
Category
Technology
-
view
5.097 -
download
1
description
Transcript of Testing Ajax, Mobile Apps the Agile Way
Testing Ajax, Mobile Apps the Agile Way
March 4, 2014 Frank Cohen, [email protected], (408) 364-5508
PepsiCo Needs Deep Reach To Consumers
•High User Experience Web App Campaigns
• Do Us A Flavor, Every2Minutes using Ajax, Web, Flash/Flex,
•Broad Reach Mobile App Campaigns
• iOS/iPhone, iPad, Android, Tablets Native Apps
•Time-On-Page Using Publishing Platforms
• Tumblr, WordPress, Google+, Facebook, Twitter
•Partners Via ERP, Workflow, Integration Platforms
• Oracle, TIBCO, IBM, Microsoft, Software AG, Redhat BPM, EMS, MQ
Testers Are Under Big New Pressure
•Agile Software Methods
•Ajax Environments
•Mixed Mobile Environments
“The Business Managers Tell Us They Need 1000 VUs in 10 Mins”
“The Devs used JMeter”
“The App Runs on WebKit Browsers and Android Native”
How Does It All Fit Together
•Agile Environment featuring Agile Stories
• Story < UseCase < Test < Schedule < Deploy < Build
• Continuous Integration, Source Repository, Test Scripting, Deploy Manager, and Agile Test Management
• CloudBees, Travis CI+Deploy, JMeter, Rally, Appvance
•Select Test Tools, Test Management Tools, And Create Operational Test Data
Test Runtime Architecture
Appvance Performance Cloud
Virtualized Environment
Shared ServicesEnvironment
AppvanceController
Map Story to TestsDeploy to Cloud
Log Results
Test Repository
Clear
Continuous Integration
JenkinsDevelopers
Business,DevOps
Test Management
Clear
Instance 2
Instance 1
BrowserChrome, IE
App Under Test
DPL RepositoryOracle
Scalability Index
Dashboard
Results RepositoryOracle
DriverSOAP, REST
Test InjectorScript Runner, Monitor, DPL
Test InjectorScript Runner, Monitor, DPL
Service Under Test
App Under Test
Service Under Test
BrowserChrome, IE
DriverSOAP, REST
Aric New User SLA WarningAverage perf: 32 Seconds
Modern Deployment Requires Modern Testing
•Desktop browsers, iOS and Android native apps, WebKit and Mobile Web browsers
•End-To-End means HTTP Protocol level testing
•Beginning-To-End means Driving The Entire App
•Combination Testing Surfaces Root Case To Bottlenecks
The Hybrid Model
•End-To-End Component Testing at API/Protocol Level
•Beginning-to-End App Integration Testing
•Scalability Testing of End-User Experience
•Data Driven Tests from live operational data sources
•Functional Tests Repurposed To Performance Tests
•Validate Digital Agency Tests (normally not LoadRunner)
Typical Solution Combines Process and Teams
•LoadRunner Best Practices for Ajax, Mobile
• Tool and license for LoadRunner team to do its own testing
• Train testers and business managers on Test Methodology
•Consulting Practice, Integration Expertise
• Build upon the tools
•Identify Requirements for Digital Agencies, Product Teams
• 10-15 best practices required by digital agencies for app development
• Server Configuration Requirements Document
• SLA Requirements Definition for Publishing PAAS Providers
Missing Integration Expertise
•App Integration Bridges Digital Agencies
Agency Architecture
Diagram
Host Implements
Frank: “Where’s the Node.js load balancer?”
Host: “Oh, We missed that.”
Digital Agencies Making Same Mistakes
•Frank asks: “Why is the deployed app not accepting 13 digit cap codes?” Agency: “We missed that.”
•Frank asks: “Why is network limiting throughput to PHP requests to Node.js?” Host: “The F5 load balancer does that by design. We’ll remove it.”
•Frank asks: “Does the delivered app have testing APIs?” Agency: “We will write some.”
Mistakes Happen At App Integration Time
•Frank asks: “How could more than 1 person log-in per second using Gigya?” Gigya: “Oh, we didn’t think of that.”
•Frank asks: “Why isn’t the Log In service scalable above 100 log-ins per second?” Agency: “You’re running it on Windows, all our experience is on Linux.” Host: “We don’t know.”
•Frank asks: “Why is Apache set to only use 25 worker threads, when the app needs hundreds?”
Example Architecture - Integrated App
Distributed Edge Cache
Browser
App Web 1Apache, PHP, Zend
MySQL MasterUtility
Apache, PHPEmail Services
NodeJSBid History
NodeJSView Counts
Load Balancer Load Balancer
MySQL Read
MySQL Read
Load Balancer
App Integration
App Integration Test Readiness Requirements
App code from Agency
Installation at Host
Voting Service (SAAS)
Configured
Preference Center
Configured
Database Created and Populated
Apache, PHP, Zend, .NET,
Java optimized
Last Minute Changes from the Business
Number of Server
Machines Identified and
Provided
Best Practices for LoadRunner on Ajax, Mobile
•Beginning-to-End App Integration Testing
• Run Ajax App In Real Browsers: Ie, Firefox, Chrome, Safari, Webkit, Mobile Web. Tests 100% Of The End-User Experience.
• LoadRunner TruClient does not work reliably with Ajax apps, does not test on Chrome, Safari, WebKit, and older IE, Firefox
• LoadRunner works great for end-to-end and protocol level tests. For example, making HTTP/JSON requests to a Node.js server through a load balancer
•Combine Beginning-To-End and LoadRunner for test of app’s integration at app and protocol tiers
Models That May Work
•Product and License - Augment What You Already Have, Teach and Train your team
•Services, Experts in Load Runner and other products, need outsource, short staff
•We know your product, we know your tools, we know our tools, we know what reduces app performance
•Maestro Service between agencies
•App Integration and Performance Test Services
Appvance
•When You Build Modern Apps And Wonder “How Risky Is That App Launch Going To Be?”
•Call Us At (855) 254-1164 Or Email Us [email protected] We Will Show You