3 tips to increase mobile test coverage

May 13, 2015

Three Tips to Increase your mobile test coverage

Dan Boutin – Senior Product Evangelist

[email protected]

Mobile (404) 304-9529

@DanBoutinSOASTA

© 2014 SOASTA. All rights reserved. March 3, 2015 2

Agenda

o What is Test Coverage, anyway?

o Functional Testing Overview

o How did the game, and the focus, change

o So how do I determine the optimal test coverage for my app?

o Step outside the box with me!

o Let’s show you how you can do it!

o Questions & Final Trivia for #swag


So what is test coverage anyway?

o Answers:

• A. The degree in which tests exercise the software features

• B. Code Coverage

• C. Device Coverage

• D. How many tests are automated


Functional Testing Overview

What is Functional Testing?

Functional Testing is testing the functionality of an

app to meet a business requirement.

Ex: Sears testing their app that a user can log in their

account, search for shoes, select the size, go to the

checkout and purchase.

What is Test Coverage?

A measure of the proportion of an application exercised

by a test suite, usually expressed as a percentage.

Many different ways to measure. Here are some:

Feature/function

Scenario

Transition Path (e.g. “click thru”.)

© 2014 SOASTA. All rights reserved. May 13, 2015 7

How did the game, and the focus, change?

© 2014 SOASTA. All rights reserved.

Mobile is a customer acquisition strategyMobile market is Frantic

○ How many business apps do you have on your phone?

○ How many applications do you use on your desktop?

© 2014 SOASTA. All rights reserved. May 13, 2015 9

Complex Ecosystem

o Fragmentation and Plethora of Devices,

OS/Platforms, & Screen Footprint

o New capabilities to test

o Regression Testing Spikes

o More Network Considerations

o More complicated Apps

What was the fallout?


Testing cycles and development methodologyAgile decreases cycle time but increases regression suites

Agile or Waterfall?

Agile regression tests increase as you progress the

project

Functional Testing Optimal Test Coverage

When is it “done”?

12

Automating the test fabrication process is Fundamental for functional

aka “The Assembly Line”

I tested 60% (141 out of 236) of the paths (picked by risk, as the most

critical paths) and 80% (4 out of 5) of the Data Sets in the software, for a

total of 53% (624 out of 1180) overall coverage.

“I tested it.”

Program Manager’s Translation:


Mobile has caused an explosion!

Testing matrix has exploded

Desktop App

300 test cases x 2 browsers = 600 test cases

600 tc * 30 mins per test case = 300 hrs

300 hrs / 6 hrs per day = 50 person days

50 pd / 5 people = 10 days of testing

Mobile App

300 test cases x 26 devices = 7800 test cases

7800 tc * 30 mins per test case = 3900 hrs

3900 hrs / 6 hrs per day = 650 person days

650 pd / 65 people = 10 days of testing – 2 weeks

650 pd / 5 people = 130 days of testing – 26 weeks

Functional Testing

Background


Functional Testing History is manual

Manual Testing Automation Testing

100%100%

Functional Testing

Needs Innovation

CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015

Build functional tests that are based on user

behavior on your application

First you need to determine the coverage model for the application.


Let your customers be your guide…


Mobile TestingThe Road to Faster Testing

Manual

Testing

Mobile Device

Clouds

Automation TouchTest

Analytics mPulse


Analyze the most common devices of real users

mPulse


Analyze the most common session paths of real

users : Data Science Workbench

TouchTest DEMO – The path to automation

22


We will close with some Mobile Trivia

@DanBoutinSOASTA


Step out of the Box….

oSOASTA TouchTest

•FREE DownLoad: http://goo.gl/6CErMN

oVisit our Blog: http://goo.gl/TA2HKP

oQuestions? @DanBoutinSOASTA

http://goo.gl/6CErMN

http://goo.gl/TA2HKP

May 13, 2015

Three Tips to Increase your mobile test coverage

Dan Boutin – Senior Product Evangelist

[email protected]

Mobile (404) 304-9529

@DanBoutinSOASTA

3 tips to increase mobile test coverage

Technology

Transcript of 3 tips to increase mobile test coverage