The Mobile Testing Checklist

38
1 © 2014 SOASTA CONFIDENTIAL - All rights reserved. The Mobile Testing Checklist Ensure success of your mobile apps - don't overbuild or under test.

description

Testing mobile apps is different. There are more form factors, more combinations, more complexity and more users. You need a checklist to be sure you don't overbuild or under test. SOASTA and Utopia have the experience and technology you need to be successful. Join this free webinar and learn: The most common mobile app issues Missed areas like app interrupts, poor connections and device settings When to automate for functionality and performance Technology for end-to-end mobile testing How to collect mobile user information for continuous improvement Utopia Solutions Founder and CTO, Lee Barnes and the SOASTA team will share customer experiences and demonstrations that will help you cross off every critical element of your mobile testing checklist.

Transcript of The Mobile Testing Checklist

Page 1: The Mobile Testing Checklist

1© 2014 SOASTA CONFIDENTIAL - All rights reserved.

The Mobile Testing Checklist

Ensure success of your mobile apps - don't overbuild or under test.

Page 2: The Mobile Testing Checklist

2© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Today’s Presenters

Lee BarnesFounder and CTO Utopia Solutions

If you have questions during the webinar, submit in chat box during event

Mike OstenbergDirector Solutions EngineeringSOASTA

Moderator: Brad JohnsonVP of Product MarketingSOASTA

Page 3: The Mobile Testing Checklist

3© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Agenda

1. Introduction

2. The Landscape of Mobile Testing

1. Why is Mobile Testing Different

2. Checklist for Mobile Testing

3. Mobile Testing Framework

3. SOASTA Solutions

1. TouchTest Key Capabilities

2. CloudTest for Mobile Load Testing

3. mPulse RUM for Mobile Monitoring

4. Resources and Conclusion

Page 4: The Mobile Testing Checklist

4© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Is In the Enterprise2014 GigaOm Survey

Source: “Mobile In the Enterprise”, Zendesk 2014

Page 5: The Mobile Testing Checklist

5© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Software Test Professionals Polls

“Do you currently have a mobile initiative”

1 Year of “Continuous Mobile Testing”

Spring 2013

Fall 2013 Spring 2014

25 attendees 28 attendees 21 attendees

56%

44%

Page 6: The Mobile Testing Checklist

6© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Gartner On Mobile

CHALLENGES

o Most traditional dev practices don’t work with mobile

o Apps require constant refinement to tune the user experience

o Rapid change requires rapid development & deployment

Driven From Enterprise Requirements

Source: Gartner “Traditional Development Practices Will Fail for Mobile Apps” April 15, 2014

Page 7: The Mobile Testing Checklist

7© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Gartner On Mobile

CHALLENGES

o Most traditional dev practices don’t work with mobile

o Apps require constant refinement to tune the user experience

o Rapid change requires rapid development & deployment

RECOMMENDATIONS

o Collaborate with business, users, devs…and observe work practices

o Use all test types: functional, performance & load, and user experience

o Analyze how apps are used in production by real users

o Adopt agile development and deployment

Driven From Enterprise Requirements

Source: Gartner “Traditional Development Practices Will Fail for Mobile Apps” April 15, 2014

Page 8: The Mobile Testing Checklist

8© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Poll Question

Page 9: The Mobile Testing Checklist

9© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Lee BarnesFounder and CTO

The Landscape of Mobile Testing

Page 10: The Mobile Testing Checklist

10

Mobile testing is…

…different

www.motivationalmemo.com

Page 11: The Mobile Testing Checklist

11© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Multiple OS / Fragmentation

Source: OpenSignal Android Fragmentation Report August 2014

iOS 8iOS 7Earlier

4.44.34.24.14.03 - 4.042.3.3 - 2.3.72.2 & earlier

Page 12: The Mobile Testing Checklist

12© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Device Diversity

Page 13: The Mobile Testing Checklist

13© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Network Diversity

Page 14: The Mobile Testing Checklist

14© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Diversity Increases Test Burden

Mobile Testing Iceberg

App Functionality

Page 15: The Mobile Testing Checklist

15© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Diversity Increases Test Burden

App Functionality

Mobile Specific Test Conditions

Page 16: The Mobile Testing Checklist

16© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Diversity Increases Test Burden

App Functionality

Mobile Specific Test Conditions

Page 17: The Mobile Testing Checklist

17© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Diversity Increases Test Burden

App Functionality

Mobile Specific Test Conditions

Page 18: The Mobile Testing Checklist

18© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Specific ChecklistTest Conditions

Installation

Interrupt

Network

Device Integration

Performance

Page 19: The Mobile Testing Checklist

19© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Testing ChecklistInterrupt Conditions

Incoming / Outgoing Calls

Popup Alerts (Calendar, Alarm, SMS, etc.)

Power Cycle

Extended Background

Page 20: The Mobile Testing Checklist

20© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Testing ChecklistInstallation Conditions

Install

Uninstall

Reinstall

Update

Page 21: The Mobile Testing Checklist

21© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Testing ChecklistNetwork Conditions

Network Type (Mobile variations and WiFi)

Network Quality

Network Loss

Network Transition

Page 22: The Mobile Testing Checklist

22© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Testing ChecklistPerformance Conditions

Speed / Responsiveness

CPU and Memory Usage

Battery Drain

On-Device / Memory Card Storage

Page 23: The Mobile Testing Checklist

23© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Testing ChecklistDevice Integration Conditions

Location / GPS

Video / Still Camera

Screen Size, Resolution, Orientation

Accelerometer

Page 24: The Mobile Testing Checklist

24© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Test CaseSelection

App Type

App Usage

App/Device Integration

Page 25: The Mobile Testing Checklist

25© 2014 SOASTA CONFIDENTIAL - All rights reserved.

AppFunctionality& Technology

User Profile

BusinessObjectives

Device Selection

• Deployment platforms & versions

• App interaction with device• Integrated systems

• Prevalent Devices• Preferred Browser• Concurrent usage /

load profile

• Application value to the business

• Risk of application failure

Page 26: The Mobile Testing Checklist

26© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Testing

Page 27: The Mobile Testing Checklist

27© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Test Case Matrix

Page 28: The Mobile Testing Checklist

28© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mobile Test Case Matrix

Page 29: The Mobile Testing Checklist

29© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Test Automation

Execute test cases across a set of diverse devices

Page 30: The Mobile Testing Checklist

30© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Mike OstenbergDirector Solutions Engineering

SOASTA Solutions

Page 31: The Mobile Testing Checklist

31© 2014 SOASTA CONFIDENTIAL - All rights reserved.

SOASTA SolutionsQuality Across the Mobile Delivery Lifecycle

Expertise

Page 32: The Mobile Testing Checklist

32© 2014 SOASTA CONFIDENTIAL - All rights reserved.

SOASTA Solutions

Expertise

Run local and remote private device clouds. Compress testing from weeks to overnight.

Quality Across the Mobile Delivery Lifecycle

Page 33: The Mobile Testing Checklist

33© 2014 SOASTA CONFIDENTIAL - All rights reserved.

SOASTA Solutions

Expertise

Quality Across the Mobile Delivery Lifecycle

Performance test the mobile backend to any

scale or global distribution.

Continuously.

Page 34: The Mobile Testing Checklist

34© 2014 SOASTA CONFIDENTIAL - All rights reserved.

SOASTA Solutions

Expertise

Quality Across the Mobile Delivery Lifecycle

Measure real mobile users in production to determine scale and usage patterns. Test better.

Page 35: The Mobile Testing Checklist

35© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Questions?

Lee BarnesFounder and CTOUtopia Solutions

Email:[email protected]

Twitter:twitter.com/USI_LeeBarnes

LinkedIn: linkedin.com/in/leebarnesBlog:

www.RaiseYourMQ.com

Mike OstenbergDirector Solutions EngineeringSOASTA

Email:[email protected]

Page 36: The Mobile Testing Checklist

36© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Poll Question

Page 37: The Mobile Testing Checklist

Thanks

Contact SOASTA:www.soasta.com/cloudtest/

[email protected]

Follow us:

twitter.com/cloudtest

facebook.com/cloudtest

RESOURCES

Next Webinar: Oct. 22, 2014“’Accelerating Web and Mobile Testing for Continuous Delivery”

Register: Webinars future and past:

http://www.soasta.com/knowledge-center/webinars/

Want to evaluate any SOASTA product or service?- email [email protected]

- You’ll hear from us tomorrow (maybe today)

Page 38: The Mobile Testing Checklist

38

www.SOASTA.com

www.utopiasolutions.com