Welcome.@propelics

Building a World-Class Mobile Testing StrategyMay 7th, 2015

We will start at 1:02pm EDT

Propelics creates mobile strategies and world-class Apps for the Enterprise.

Propelics: Background

• 15+ year track record• Founded in 2011• Fortune 500 focus• Global delivery experience

Trusted by some of the largest global brands:

Pre Built Mobile Apps

Mobile Strategy Research & Advisory

Custom AppS & Managed Services

• 100% focused on digital and mobile for the Enterprise

• San Jose, Boston, Pittsburgh, Guadalajara

2

MOBILE RESEARCH COUNCIL

We will give you the edge.

Our Objective:Driving innovation and business value with mobile software and solutions.

Our executive guidance helps members to quickly and effectively deliver high-impact mobile experiences to employees, partners and customers.

ü MRC has no sales angle or vendor bias. ü Members benefit from insider access to real-

world mobile experience.ü Members include: American

Airlines, AmerisourceBergen, Bank ofMontreal, Carnival Corporation, Family Dollar, UPS, and many more…

From Market Trends to Action at the Speed of Mobile

Research. Advisory. Community.

Shahab Choudhry: Partner - Propelics

Shahab is a visionary thought leader who has been at the forefront of trying to make sense of the various technology shifts in the past two decades.

He has advised several startups, early stage companies and Fortune 500 companies in making the most of these technology shifts. He continues to be a “Trusted Advisor” to some of the leading brands around the world. Shahab is frequent speaker at conferences related to mobile.

4

Q:What are the key considerations for my MobileTesting Strategy for the Enterprise?

5

The mobile reality…

6

Methods of customer interaction will change

Source:  Morgan  Stanley,  20117

8

Testing is not a new concept…

…Mobile Testing is another storyWorld Quality Report 2012 - 2013

• What is the readiness of the team to test new features in rapid sprints?

• How ready is the organization to test every app against new OS releases?

• What aspects of our testing plan can be automated?

• What type of tools are available to help with the various types of testing?

• How can organizations test for user centric design?

• How can we test every app in a BYOD environment?

• How do you plan on integrating Mobile Testing into your Project Plan?

• Are the testing methodologies consistent and repeatable across OSes and devices?

• Will our app drain a user’s battery life or use up all its memory?

• How can we procure and maintain the devices needed for testing?

• How do we test for mobile specific considerations such as phone calls, switching apps etc.

• How can we test gestures?

• …and many more10

Testing Math101

11

12

Mobile Testing: The Complexity Equation

# of supported devices # of supported OSesOS updates per year

Number of Apps # of test cases per appApp release frequency

The “I can barely keep up” syndrome

X

Disruptive Change

Ready?

Q:What are companies doing to address this need to focus on quality?

14

Enterprise Mobility Testing

Strategy

Business Drivers

mobile testing philosophy

Supported mobile

Devices / OS

Mobile App Portfolio

Reliability

App Adoption

Continuous development

Security

Quality ImprovementEliminate manual work

Mobile specific testing tools

Mobile testing skill sets

Mobile Testing: The strategy

6 Considerations for Enterprise Mobile Testing Strategy

1. Be mindful of the unforgiving attitude of the mobile user

The Unforgiving UserQuick to download - Quick to delete

74% of downloaded apps are opened only once40% of apps are deleted after first use

Average expected wait time for a app to launch is 2 secTop reason for deleting apps – app crashed

Enterprise Expectation = Consumer Experience

Gift of Consumerization

2. Know the types of testing

20

Installation Testing

• New App Install• Uninstall and Reinstall

• Upgrade testing

Interoperability (IOP)

• Voice / SMS interrupts• Notifications

• Battery /Cable Removal

Functional• Validation of Functionality

• Smoke / Regressions Testing

• Offline access testing• Negative Testing

Usability Testing

• User Experience• Competitive Analysis

• Expert Review

Security Testing• OWASP Vulnerabilities

• Dynamic Testing• Static Code Analysis

• Data Encryption

Non Functional• Network Strength / Outage /

Recovery• Different NW Types• Peripheral Testing

Language Testing

• Validation for Locales• Images and Text

• Currencies, time zones etc.• Context

Mobile Testing: the big blocks

Memory Leak• Memory Usage• Memory Leaks

• Garbage Collection

Performance Testing

• CPU Usage testing• Network Usage

• Page Render time or activity Render time

3. Understand the impact of the “New Math”

22

Mobile Testing: The Complexity Equation

# of supported devices # of supported OSesOS updates per year

Number of Apps # of test cases per appApp release frequency

The “I can barely keep up” syndrome

X

# of Supported Devices

23

Source:  Open  Signal

Diversity of Supported OSes

24

iOS Android

Devices and OSes

25

• B2C• Select based on analytics based on

% of sessions from each device type• Cutoff at devices with less than 1%

of sessions• Cutoff at 75% - 80%

• B2B• ALL Corporate Owned• BYOD – Approved devices

•

• Maintain a Pool of Devices• Setup a Device Cloud• Simulator as a last resort

# of Apps

26

4. Establish a baseline of where you stand

Mobility Testing Readiness: Details

28

Sta

ge

4

Sta

ge

3

Sta

ge

2

Sta

ge

1

Sta

ge

5

Mobility Testing Strategy

SDLC

Mobile Testing Tools

Types of TestingFunctional

Non-FunctionalMemory Leak

Installation TestingPerformance Testing

Security TestingInteroperability Testing

Usability TestingLanguage Testing

Mobile Testing Skills

Mobile Testing Organization

Security & compliance

Testing Maturity Ranking

Testing Readiness Report Card

5. Build a comprehensive plan of attack

App Portfolio

Testing capabilities

Testing Roadmap

implementation

Mobile Testing Roadmap

Business Drivers

Philosophy and Quality Posture

Types and Number of Apps

Complexity of Apps

Best Practices

Skills, Roles & Responsibilities

Testing Tools

Testing Types

Alignment with SDLC

Ongoing Testing

EnterpriseMobility Testing

Strategy

Mobile Testing: Philosophy

Mobility Testing Roadmap

31

• Tactical Roadmap to Improve Maturity

• Each Tactic Ranked On: 1. Value and Alignment to Goals2. Organizational Readiness3. Ease of Implementation

• All Tactics Prioritized & Phased• Prioritize Apps for mobile testing

based on• Criticality• Complexity• Effort

• Select Apps for initial phase of testing

Mobile Testing Roadmap

32

Required

Short  Term

Mid/Long  Term

Future

Quick  Wins

6. Automated functional testing is essential

Mobile Testing Automaton Roadmap

34

Mobile Testing Automaton Roadmap

35

Business Criticality (H,M,L)Core/frequently used functionality, Security/data breach/privacy/legal, Dependence on 3rd Party data or technology, User experience impact

Value of Automation (H,M,L)Increases quality, Reduces resource load, High volume/Repetitive/Data driven, Feature life-span, Smoke test/Build verification

Automation Readiness (H,M,L)Test Script Complexity, Stable Functionality, Measurable results, Recordable actions, Explicit list of observable steps, Test Script Maintainability

Ease of Automation (H,M,L)Data or logic driven, Object level user interface, Known expected results around user experience, Clear acceptance criteria per test case, Minimal performance issues interacting with external end points or services

1. Be mindful of the unforgiving attitude of the mobile app user

2. Know the types of testing

3. Understand the impact of the “New Math”

4. Establish a baseline of where you stand

5. Build a comprehensive plan of attack

6. Automated functional testing is essential

6 Considerations for Enterprise Mobile Testing

Propelics Mobile Testing Services

• Mobile Testing Readiness Kickstart• Mobile Testing Roadmap Implementation• Mobile Test Automation Kickstart• Mobile Testing as a Managed Service

Action Plan and testing strategy recommendations

Mobile Testing maturity

assessment

Step I: Direction Setting

Step IV: Next Steps and

Recommendations

Mobile Testing Readiness Kickstart: The methodology

Step III: Prioritization & Phasing

Mobile Test

strategy scope

Mobile Testing

Roadmap

1-2 WeeksSize and Complexity of App Portfolio

Types of Testing

Testing Scope

Finalized

Step II: Current State Assessment

& Recommendations

Recommended Apps for initial

Phasing

Business Drivers

App Portfolio

App portfolio Phasing

Mobile Test Automation kickstart

Ongoing Testing

Testing Setup &

Initiation

Step I: Direction Setting

Step IV: Ongoing Testing

Mobile Test Automation: The methodology

Step III: Setup,

Initiation & Initial Testing

Step II: Planning &

Phasing

Number & complexity

of Apps

Device Types

Test Automation

Scope

Supported OS

Mobile Test

Automation Roadmap

Release frequency

User stories

Testing Tools

5-10 Days

Test Plan

1- 2 Months

Ongoing

Mobile testing as a managed service

Webinar Attendee Special Offer

40

Special OfferFree 1 Hour Mobile

Advisory Callinfo@propelics.com

888-405-2820

TestingStrategy for

MobileKickstart

App Scoping & Prototype

EnterpriseMobile Roadmap

IT Strategy for Mobile

Mobile UI/UXDesign

Mobile COE

41

Thank you for attending!