Enough is not enough - Test Strategy for Mobile

32
Enough is not enough Test Strategy for Mobile Jatin Bhasin @jatin_bhasin

description

This talk was presented at VodQA Gurgaon 3rd edition (11 July 2013) Talk Abstract: No matter how many devices, platforms or screen-sizes you test your mobile app on, your testing may still not be enough. In this era of ever increasing mobile devices and varied platforms, this is bound to happen unless you have a test plan tailored for the mobile world. The intent of this talk is to brace ourselves for this challenge and envision a test strategy for mobile that covers these widespread avenues. About Speaker: Jatin has been breaking software systems and helping in re-building them better since last 5+ years. Having worked for clients ranging from small startups to big enterprises, he has worked on variety of domains including telecom, retail, e-commerce, mobile and consulting. He works as Senior Quality Analyst at ThoughtWorks, Gurgaon.

Transcript of Enough is not enough - Test Strategy for Mobile

Page 1: Enough is not enough - Test Strategy for Mobile

Enough is not enough Test Strategy for Mobile

Jatin Bhasin @jatin_bhasin

Page 2: Enough is not enough - Test Strategy for Mobile

Who am I ?

!   Tester ! Agilist !   Learner !   Traveller ! ThoughtWorker

Page 3: Enough is not enough - Test Strategy for Mobile

What is Test Strategy ?

Page 4: Enough is not enough - Test Strategy for Mobile

Definition

A test strategy is an outline that describes the testing approach of the software development cycle. It is created to inform project managers, testers, and developers about some key issues of the testing process.

Page 5: Enough is not enough - Test Strategy for Mobile

Test Strategy

!   Scope !   Roles and Responsibilities !   Environment !   Testing Tools !   Risks and Mitigation !   Schedule !   Priorities !   and more…

Page 6: Enough is not enough - Test Strategy for Mobile

Why all the buzz about Mobile?

Page 7: Enough is not enough - Test Strategy for Mobile

Did you know?

(Source: Mobile Marketing Association Asia)

World Population ~ 7 Billion

Toothbrushes ~ 4.2 Billion

Cell Phones ~ 5.1 Billion

Page 8: Enough is not enough - Test Strategy for Mobile

Yes, that’s true!

!   Smartphone usage grew by 50% from 2011 to 2012 !   91% of Americans have their mobile device within reach 24x7 !   This year, mobile phones will surpass PCs as the primary means of

accessing the internet !   In 2011, Apple sold 48m mobile devices

V/S 4.9m desktops/laptops. !   Only 20% of businesses have invested in a mobile solution

(Source: various)

Page 9: Enough is not enough - Test Strategy for Mobile

Types of Mobile apps

Native Hybrid Mobile Web

Page 10: Enough is not enough - Test Strategy for Mobile

Big challenges on small devices

!   Environment !   Application !   Users !   Devices !   Network !   Automation

Page 11: Enough is not enough - Test Strategy for Mobile

Environment

!   Frequent OS changes !   Domain Constraints

Page 12: Enough is not enough - Test Strategy for Mobile

Application

!   Native/web/hybrid Apps !   Behavior on different devices !   Tech stack differs for each OS !   Comparison with web counterparts

Page 13: Enough is not enough - Test Strategy for Mobile

Users

!   Intuitive UI !   Accommodating maximum functionality

Page 14: Enough is not enough - Test Strategy for Mobile

Devices

!   Browser limitations !   Touch and non-touch devices !   Memory Constraints !   Battery Drainage !   Keypad lock

Page 15: Enough is not enough - Test Strategy for Mobile

Network

!   Sans network !   Offline support !   Wi-Fi !   Interruptions

Page 16: Enough is not enough - Test Strategy for Mobile

Automation

!   Device OS specific tools !   Testing across platforms !   Who will do it?

Page 17: Enough is not enough - Test Strategy for Mobile

Test thy app

!   Physical Device !   Simulators !   Testing in the ‘Cloud’ !   Crowd Source testing !   Automation

Page 18: Enough is not enough - Test Strategy for Mobile

Physical Devices

!   Benefits

!   Challenges

Page 19: Enough is not enough - Test Strategy for Mobile

Simulators

!   Benefits

!   Challenges

Page 20: Enough is not enough - Test Strategy for Mobile

Testing in the ‘cloud’

!   Benefits

!   Challenges

!   Examples !   Device Anywhere !   Perfecto Mobile ! Testdroid

Page 21: Enough is not enough - Test Strategy for Mobile

Crowd Source testing !   Benefits

!   Challenges

!   Examples !   Mob4Hire ! uTest ! Testcovery

Page 22: Enough is not enough - Test Strategy for Mobile

Automation - Android

! Robotium !   Bot-Bot ! Monkeyrunner ! DroidPilot ! AndroidDriver (Selenium/WebDriver)

Page 23: Enough is not enough - Test Strategy for Mobile

Automation - iOS

!   Frank !   KIF (Keep It Functional) ! UIAutomation !   Zucchini

Page 24: Enough is not enough - Test Strategy for Mobile

Automation - cross platform

! Appium !   Calabash ! NativeDriver !   And many more…

Page 25: Enough is not enough - Test Strategy for Mobile

Types of testing

!   Usability !   Performance !   Security

!   Interface Testing !   Services Testing !   Low level resource testing !   Compatibility !   Operational

Page 26: Enough is not enough - Test Strategy for Mobile

Usability Testing

!   Ease of use !   Single handed use !   How intuitive is it

Page 27: Enough is not enough - Test Strategy for Mobile

Performance Testing

!   Need for speed !   Get real !   Business impact !   3rd party integrations !   Performance tuning

Page 28: Enough is not enough - Test Strategy for Mobile

Security Testing

!   Penetration test !   Vulnerability Assessment !   Encryption

Page 29: Enough is not enough - Test Strategy for Mobile

Mobile means faster feedback!

!   User Ratings

Page 30: Enough is not enough - Test Strategy for Mobile

Forget me not !

!   Budget !   Time to market !   Target users

Page 31: Enough is not enough - Test Strategy for Mobile

Recap

Challenges in Mobile Testing

Test thy app

Devices Application Users Environment

Network Automation

Automation Tools

Testing in Cloud

Physical device Simulators

Testing Types

#2

#3

#4

#1

Crowdsourcing

Performance Usability

Security

Cross-platform

iOS Android

Page 32: Enough is not enough - Test Strategy for Mobile

Questions