From Robotium to Appium: Choose Your Journey

Dmitry VinnikLead Software Engineer, Salesforce

GoalsMobile End-to-End Testing: What, Why, How and When

Robotium vs Appium

Find Right E2E Tool For Your Team

What are our goals?

2

Do you do manual testing?

3

Do you write automation?

From the Top Level

4

How do we organize QA Effort?

Hint: It has 4 sides

Agile Testing Quadrants

5

Testing Quadrants

From “Agile Testing: A Practical Guide for Testers and Agile Teams”by Lisa Crispin and Janet Gregory

How do we usually test?Hint: It has 3 sides

6

Let’s go in more details

Test Pyramid↑ Fidelity↓ Speed↑ Cost

↓ Fidelity↑ Speed↓ Cost 7

Target PlatformsiOS

Android

Responsive Apps

Hybrid Apps

Target Platforms?

8

Android iOS

Mobile is taking overMobile vs DesktopDesktop

Mobile

Let’s see some dataBut “Mobile” is so broadAndroid

iOS

Android is leadingFocus: Android

Test Pyramid↑ Fidelity↓ Speed↑ Cost

↓ Fidelity↑ Speed↓ Cost 12

Mobile Test Pyramid↑ Fidelity↓ Speed↑ Cost

↓ Fidelity↑ Speed↓ Cost 13

Many E2E Testing Solutions

Calabash

UIAutomator MonkeyTalk

KIF

Journey of Android E2E Testing

Android E2E Testing

Robotium Selendroid Appium

Our Testing JourneyRobotium Selendroid

Appium

Selendroid

Appium

Robotium: Architecture

RobotiumDevice

App Package

Test Package

Test Runner

Robotium

Android Oriented

Blackbox Testing

User Centric ↓ Verbosity

Robotium: Demo

Robotium

Our Testing JourneyRobotium Selendroid

Appium

Selendroid

AppiumAppium

SelendroidRobotium

Selendroid: Architecture

Selendroid

Selendroid

Selenium Based

Multiple Devices

Gesture Support

Older Android API

Selendroid: Demo

Selendroid

Our Testing JourneyRobotium Selendroid

Appium

Selendroid

AppiumAppium

SelendroidRobotium

Robotium vs Selendroid

Robotium vs SelendroidRobotium Category SelendroidRobotium Category Selendroid

Architecture

Robotium Category SelendroidActivity Architecture

Robotium Category SelendroidActivity Architecture App

Robotium Category SelendroidActivity Architecture App

Locator Strategy

Robotium Category SelendroidActivity Architecture App

By Android IDs

Locator Strategy

Robotium Category SelendroidActivity Architecture App

By Android IDs

Locator Strategy

By Selenium Locators

Robotium Category SelendroidActivity Architecture App

By Android IDs

Locator Strategy

By Selenium Locators

Gesture Support

Robotium Category SelendroidActivity Architecture App

By Android IDs

Locator Strategy

By Selenium Locators

Limitted Gesture Support

Robotium Category SelendroidActivity Architecture App

By Android IDs

Locator Strategy

By Selenium Locators

Limitted Gesture Support Extensive

Our Testing JourneyRobotium Selendroid

Appium

Selendroid

AppiumAppium

SelendroidRobotiumRobotium Selendroid

Appium

Appium: Architecture

AppiumDevice

Appium Client

Appium Server

Native Libraries App

Appium

Selenium Based

Platform Agnostic

Language Agnostic

Developer Oriented

Appium: Demo

Appium

Appium: Four Pillars

AppiumProduction ->Test Ready

Loose Coupling

Standards Driven

Open Source

Our Testing JourneyRobotium Selendroid

Appium

Selendroid

AppiumAppium

SelendroidRobotiumRobotium Selendroid

Appium

Selendroid

Selendroid vs Appium

Selendroid vs AppiumSelendroid Category AppiumSelendroid Category Appium

Platform

Selendroid Category AppiumAndroid Platform

Selendroid Category AppiumAndroid Platform Any*

Selendroid Category AppiumAndroid Platform Any*

Android API Version

Selendroid Category AppiumAndroid Platform Any*

All* Android API Version

Selendroid Category AppiumAndroid Platform Any*

All* Android API Version Post API16

Selendroid Category AppiumAndroid Platform Any*

All* Android API Version Post API16

Advanced Web Tests

Selendroid Category AppiumAndroid Platform Any*

All* Android API Version Post API16

Limitted Advanced Web Tests

Selendroid Category AppiumAndroid Platform Any*

All* Android API Version Post API16

Limitted Advanced Web Tests Extensive

Android is leadingBut iOS as important

Our Testing JourneyRobotium Selendroid

Appium

Selendroid

AppiumAppium

SelendroidRobotiumRobotium Selendroid

Appium

SelendroidSelendroid

Appium

Robotium

Robotium vs Appium

Robotium vs AppiumRobotium Category AppiumRobotium Category Appium

Coupling

Robotium Category AppiumTight Coupling

Robotium Category AppiumTight Coupling Loose

Robotium Category AppiumTight Coupling Loose

Device Support

Robotium Category AppiumTight Coupling Loose

Single Device

Device Support

Robotium Category AppiumTight Coupling Loose

Single Device

Device Support

Multiple Devices

Robotium Category AppiumTight Coupling Loose

Single Device

Device Support

Multiple Devices

Record & Play

Robotium Category AppiumTight Coupling Loose

Single Device

Device Support

Multiple Devices

Paid Plugin Record & Play

Robotium Category AppiumTight Coupling Loose

Single Device

Device Support

Multiple Devices

Paid Plugin Record & Play

Free, Desktop App

Right Tool for the Right Problem

36

Avoid Tool Mixups

37

No DDD

No Common Language

↑ Complexity

How to Choose?

38

Github Repos

Twitter Activity

External Integrations POC

But More Importantly

39

Flexibility

Use CaseUse Case

Flexibility

40

Cost of Transition ROI

Ability to Replace

Ability to Customize

But More Importantly

41

Flexibility

Use CaseUse Case

Flexibility

Use Case

Use Case

42

Team Expertise

Application Framework

Test Infrastructure

Scaling Up

43

BrowserStack

Devices

SauceLabs

OS Version

Going Forward

Call For Action

Evaluate Test Architecture

Unify Test Strategy

Adapt to Change Experiment!

44

Q/A

45

About SpeakerTwitter: @DmitryVinnikLinkedIn: in/dmitry-vinnik/Email: dmitry@dvinnik.com

46