Post on 24-May-2015
description
1© 2014 SOASTA CONFIDENTIAL - All rights reserved.
The Mobile Testing Checklist
Ensure success of your mobile apps - don't overbuild or under test.
2© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Today’s Presenters
Lee BarnesFounder and CTO Utopia Solutions
If you have questions during the webinar, submit in chat box during event
Mike OstenbergDirector Solutions EngineeringSOASTA
Moderator: Brad JohnsonVP of Product MarketingSOASTA
3© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Agenda
1. Introduction
2. The Landscape of Mobile Testing
1. Why is Mobile Testing Different
2. Checklist for Mobile Testing
3. Mobile Testing Framework
3. SOASTA Solutions
1. TouchTest Key Capabilities
2. CloudTest for Mobile Load Testing
3. mPulse RUM for Mobile Monitoring
4. Resources and Conclusion
4© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Is In the Enterprise2014 GigaOm Survey
Source: “Mobile In the Enterprise”, Zendesk 2014
5© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Software Test Professionals Polls
“Do you currently have a mobile initiative”
1 Year of “Continuous Mobile Testing”
Spring 2013
Fall 2013 Spring 2014
25 attendees 28 attendees 21 attendees
56%
44%
6© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Gartner On Mobile
CHALLENGES
o Most traditional dev practices don’t work with mobile
o Apps require constant refinement to tune the user experience
o Rapid change requires rapid development & deployment
Driven From Enterprise Requirements
Source: Gartner “Traditional Development Practices Will Fail for Mobile Apps” April 15, 2014
7© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Gartner On Mobile
CHALLENGES
o Most traditional dev practices don’t work with mobile
o Apps require constant refinement to tune the user experience
o Rapid change requires rapid development & deployment
RECOMMENDATIONS
o Collaborate with business, users, devs…and observe work practices
o Use all test types: functional, performance & load, and user experience
o Analyze how apps are used in production by real users
o Adopt agile development and deployment
Driven From Enterprise Requirements
Source: Gartner “Traditional Development Practices Will Fail for Mobile Apps” April 15, 2014
8© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Poll Question
9© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Lee BarnesFounder and CTO
The Landscape of Mobile Testing
10
Mobile testing is…
…different
www.motivationalmemo.com
11© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Multiple OS / Fragmentation
Source: OpenSignal Android Fragmentation Report August 2014
iOS 8iOS 7Earlier
4.44.34.24.14.03 - 4.042.3.3 - 2.3.72.2 & earlier
12© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Device Diversity
13© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Network Diversity
14© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Diversity Increases Test Burden
Mobile Testing Iceberg
App Functionality
15© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Diversity Increases Test Burden
App Functionality
Mobile Specific Test Conditions
16© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Diversity Increases Test Burden
App Functionality
Mobile Specific Test Conditions
17© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Diversity Increases Test Burden
App Functionality
Mobile Specific Test Conditions
18© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Specific ChecklistTest Conditions
Installation
Interrupt
Network
Device Integration
Performance
19© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Testing ChecklistInterrupt Conditions
Incoming / Outgoing Calls
Popup Alerts (Calendar, Alarm, SMS, etc.)
Power Cycle
Extended Background
20© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Testing ChecklistInstallation Conditions
Install
Uninstall
Reinstall
Update
21© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Testing ChecklistNetwork Conditions
Network Type (Mobile variations and WiFi)
Network Quality
Network Loss
Network Transition
22© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Testing ChecklistPerformance Conditions
Speed / Responsiveness
CPU and Memory Usage
Battery Drain
On-Device / Memory Card Storage
23© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Testing ChecklistDevice Integration Conditions
Location / GPS
Video / Still Camera
Screen Size, Resolution, Orientation
Accelerometer
24© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Test CaseSelection
App Type
App Usage
App/Device Integration
25© 2014 SOASTA CONFIDENTIAL - All rights reserved.
AppFunctionality& Technology
User Profile
BusinessObjectives
Device Selection
• Deployment platforms & versions
• App interaction with device• Integrated systems
• Prevalent Devices• Preferred Browser• Concurrent usage /
load profile
• Application value to the business
• Risk of application failure
26© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Testing
27© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Test Case Matrix
28© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mobile Test Case Matrix
29© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Test Automation
Execute test cases across a set of diverse devices
30© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Mike OstenbergDirector Solutions Engineering
SOASTA Solutions
31© 2014 SOASTA CONFIDENTIAL - All rights reserved.
SOASTA SolutionsQuality Across the Mobile Delivery Lifecycle
Expertise
32© 2014 SOASTA CONFIDENTIAL - All rights reserved.
SOASTA Solutions
Expertise
Run local and remote private device clouds. Compress testing from weeks to overnight.
Quality Across the Mobile Delivery Lifecycle
33© 2014 SOASTA CONFIDENTIAL - All rights reserved.
SOASTA Solutions
Expertise
Quality Across the Mobile Delivery Lifecycle
Performance test the mobile backend to any
scale or global distribution.
Continuously.
34© 2014 SOASTA CONFIDENTIAL - All rights reserved.
SOASTA Solutions
Expertise
Quality Across the Mobile Delivery Lifecycle
Measure real mobile users in production to determine scale and usage patterns. Test better.
35© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Questions?
Lee BarnesFounder and CTOUtopia Solutions
Email:lee.barnes@utopiasolutions.com
Twitter:twitter.com/USI_LeeBarnes
LinkedIn: linkedin.com/in/leebarnesBlog:
www.RaiseYourMQ.com
Mike OstenbergDirector Solutions EngineeringSOASTA
Email:mostenberg@soasta.com
36© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Poll Question
Thanks
Contact SOASTA:www.soasta.com/cloudtest/
info@soasta.com866.344.8766
Follow us:
twitter.com/cloudtest
facebook.com/cloudtest
RESOURCES
Next Webinar: Oct. 22, 2014“’Accelerating Web and Mobile Testing for Continuous Delivery”
Register: Webinars future and past:
http://www.soasta.com/knowledge-center/webinars/
Want to evaluate any SOASTA product or service?- email info@soasta.com
- You’ll hear from us tomorrow (maybe today)
38
www.SOASTA.com
www.utopiasolutions.com