iOS and Android apps automation

50
iOS and Android Apps automation By, Sridhar Ramakrishnan Life long travel geek @ Hotwire Inc.

Transcript of iOS and Android apps automation

Page 1: iOS and Android apps automation

iOS and Android Apps automation

By, Sridhar Ramakrishnan

Life long travel geek @ Hotwire Inc.

Page 2: iOS and Android apps automation

Agenda• How is Native app testing is different from Web testing• Kind of Native app bugs• Test automation• Different framework for native apps automation• What is appium & how it works• Android & iOS frameworks – Espresso & Xcode UI testing• Sample test code• Demo• Code Coverage• Our CI process• Screenshot testing – Image comparison tests• Tracking• So why we are doing all this??

Page 3: iOS and Android apps automation

How is Native app testing is different from Web testing

• Usability & interaction• Faster & more reactive• Involves Gestures• No unnecessary content

• Data Entry Keyboard Types• Numeric vs. Alphanumeric

• Phone Settings• Changing Regions• Changing Location Services• Changing Notifications

• Devices vs. Simulators

• Multiple Devices and Operating Systems

Page 4: iOS and Android apps automation

Bugs Types

• Crashes• Missing data from the API• Missing data in the app• Wrong keyboard displayed• Rendering differences across devices• Views overlaying one another• Views are getting locked up• Gesture related – swipe, double tap

Page 5: iOS and Android apps automation

More Bugs ?

• Limited manual regression testing = Spot check ; less boring

• More manual exploratory testing = More bugs;

“Simultaneously learning about the system while designing and executing tests, using feedback from the last test to inform the next.” - Elisabeth Hendrickson

Page 6: iOS and Android apps automation

Test Automation

Page 7: iOS and Android apps automation

Frameworks comparison

Frameworks for comparison

Test the actual

appMany

languageStandard API ( Webdriver)

Open source &

active BDD iOS Android HybridFirefox

OSAppiumXcode UI Automation calabash-iOSFrankios-driverKIFRobotiumEspressoselendroid

Page 8: iOS and Android apps automation

Test Automation @ Hotwire

• Test Framework = Hotwire BDD• Tools =

Xcode UI testing

• API = Spoofer (mock) vs API calls

Page 9: iOS and Android apps automation

Appium – Multi OS Support

And More…

Page 10: iOS and Android apps automation

Appium Architecture

Test Script ( Java )

Webdriver (Seleium Java)

Apple Instruments

(Instruments.js)

iPhone / iPad[Simulator /

Device]APP

Android[Simulator /

Device]APP

Android (adb, UIAutomator

(AppiumBootstrap.jar …)

FireFox OS / Hybrid Apps

Firefox OS / Web browser

BDD (Cucumber, Webdriver,

Spring, maven)

CI – Jenkins (Build xcode, start Appium server, mvn install

Appium server (Node.js) JSJSON

Page 11: iOS and Android apps automation

BDD – Cucumber & Gherkin

Feature: Hotel Search and Book

Scenario: Search and book hotel as Sign In user Given I'm searching for hotel and attempt to book When I book a hotel as valid signed in user Then I should see confirmation that my hotel is booked

Page 12: iOS and Android apps automation

BDD – Cucumber @tags

@iOS @Android @SMOKEFeature: Hotel Search and Book

Scenario: Search and book hotel as Sign In user Given I'm searching for hotel and attempt to book When I book a hotel as valid signed in user Then I should see confirmation that my hotel is booked

Page 13: iOS and Android apps automation

BDD – StepDefs//Spring context file@ContextConfiguration("classpath:cucumber.xml")

//hotel object gets input from Spring@Autowired private Hotel hotel;

@Given("^I'm searching for hotel and attempt to book$") public void search() {

hotel.search(); }

Page 14: iOS and Android apps automation

Java code

public interface Hotel {void search();}

// iOSpublic class IosHotelImpl implements Hotel {

@OverridePublic void search() {IosSearchScreen iosSearchScreenObj = new IosSearchScreen(getWebDriver());}

// Androidpublic class AndroidHotelImpl implements Hotel {

@OverridePublic void search() {AndroidSearchScreen androidSearchScreenObj = new AndroidSearchScreen(getWebDriver());}

Page 15: iOS and Android apps automation

Screen Objects - iOS

// iOS Screen Object

public class iOSSearchScreen extends AbstractScreen {

@FindBy(name = ”Search") private WebElement searchBtn; public void search() { searchBtn.click(); }

}

Page 16: iOS and Android apps automation

Screen Objects - Android// Android Screen object

public class AndroidSearchScreen extends AbstractScreen {

@FindBy(id= ”Search") private WebElement searchBtn;

public void search() { searchBtn.click(); }

}

Page 17: iOS and Android apps automation

Screen Objectspublic class AbstractScreen {

private final WebDriver webdriver;

public AbstractScreen(WebDriver webdriver) { PageFactory.initElements(webdriver, this); this.webdriver = webdriver; }

protected WebDriver getWebDriver() { return webdriver; }}

Page 18: iOS and Android apps automation

BDD – StepDefs//Spring context file@ContextConfiguration("classpath:cucumber.xml")

//hotel object assigned by Spring@Autowired private Hotel hotel;

@Given("^I'm searching for hotel and attempt to book$") public void search() {

hotel.search(); }

Page 19: iOS and Android apps automation

Dependency Injection using Spring Framework

Context file - Cucumber.xml

<context:component-scan base-package="com.test" />

<import resource="cucumber-${mobileOS}.xml"/>

Page 20: iOS and Android apps automation

Dependency Injection using Spring Framework

Cucumber-ios.xml

<bean id="hotel" class="com.test.ios.hotel.IosHotelImpl"></bean>

<bean id="desiredCapabilities" class="org.openqa.selenium.remote.DesiredCapabilities">

Page 21: iOS and Android apps automation

Dependency Injection using Spring Framework

Cucumber-android.xml

<bean id="hotel" class="com.test.android.hotel.AndroidHotelImpl"> </bean> <bean id="desiredCapabilities" class="org.openqa.selenium.remote.DesiredCapabilities">

Page 22: iOS and Android apps automation

Managing Dependency

3rd Party Jars

CucumberSpringReportingJunit / testng…

Tests Jars

FeaturesStepDefsiOS-Screen-ObjectsAndroid-Screen-Objects…

Page 23: iOS and Android apps automation

Hotwire Apps BDD architecture

Screen Objects (Objects representing screen /

fragment)

Appium server (Node.js)

Apple Instruments

with UI Automation

iPhone / iPad[Simulator /

Device]APP

JSON

JS

Page 24: iOS and Android apps automation

Appium locator & tools

WebDriver Locators :• ID• Name• XpathTools :• Appium inspector (iOS)• Uiautomatorviewer (Android)

Page 25: iOS and Android apps automation

Appium Sample Code – Send Text and Tap

@FindBy(name = "Email address") private WebElement emailAddress;

@FindBy(name = "Password") private WebElement password;

@FindBy(name = "Sign in") private WebElement signIn; public void enterSignIn(String emailAddress, String password) { this.emailAddress.clear(); this.emailAddress.sendKeys(emailAddress); this.password.clear(); this.password.sendKeys(password); this.signIn.click(); }

Page 26: iOS and Android apps automation

Sample Code – Swipe functionalityprivate void swipeFunctionality(WebElement startPoint, WebElement endPoint) {

HashMap<String, Double> args = new HashMap<>(); double x1 = startPoint.getLocation().getX(); double y1 = startPoint.getLocation().getY(); double width1 = startPoint.getSize().getWidth(); double height1 = startPoint.getSize().getHeight(); args.put("touchCount", (double) 1); args.put("startX", (double) (x1 + width1) - 5); args.put("startY", (double) (y1 + height1 / 2));

double x2 = endPoint.getLocation().getX(); double y2 = endPoint.getLocation().getY(); double width2 = endPoint.getSize().getWidth(); double height2 = endPoint.getSize().getHeight(); args.put("touchCount", (double) 1); args.put("endX", (double) (x2 + width2) - 5); args.put("endY", (double) (y2 + height2 / 2));

args.put("duration", (double) 1); executeScript("mobile: swipe", args); }

public Object executeScript(String script, Object... args) { return ((JavascriptExecutor) getAppiumDriver()).executeScript(script, args); }

Page 27: iOS and Android apps automation
Page 28: iOS and Android apps automation

Sample code (contd.) if (checkInDate.equals(LocalDate.now().plusDays(1))) { LocalDate currentDate = LocalDate.now(); String currentMonth = new SimpleDateFormat("MMMM").format(currentDate.toDateTimeAtStartOfDay().toDate()); String currentDay = currentDate.dayOfMonth().getAsString(); String currentWeekName = currentDate.withDayOfWeek(currentDate.getDayOfWeek()).dayOfWeek().getAsText(); String currentCheckin = "S_Today, " + currentWeekName + ", " + currentMonth + " " + currentDay; WebElement currentElmt = getAppiumDriver().findElement(By.name(currentCheckin)); swipeFunctionality(currentElmt, checkOutElmt); } else if (checkInMonth.equals(checkOutMonth) || nextToCheckInMonth.equalsIgnoreCase(checkOutMonth)) { checkInElmt.click(); if (tapOrDrag.equalsIgnoreCase("tap")) { checkOutElmt.click(); } else if (tapOrDrag.equalsIgnoreCase("tap and drag")) { swipeFunctionality(checkInElmt, checkOutElmt); } else { swipeFunctionality(nextToCheckInElmt, checkOutElmt); } } else { checkInElmt.click(); if (tapOrDrag.equalsIgnoreCase("tap")) { fillCalendarMonth(checkOutMonth); checkOutElmt.click(); } else if (tapOrDrag.equalsIgnoreCase("tap and drag")) { WebElement middleElmt = findMiddleElement(checkInDate, checkInDay, checkInMonth); swipeFunctionality(checkInElmt, middleElmt); fillCalendarMonth(checkOutMonth); swipeFunctionality(middleElmt, checkOutElmt); } else { WebElement middleElmt = findMiddleElement(checkInDate, checkInDay, checkInMonth); swipeFunctionality(nextToCheckInElmt, middleElmt); fillCalendarMonth(checkOutMonth); swipeFunctionality(middleElmt, checkOutElmt); } }

Page 29: iOS and Android apps automation

iOS Demo

Page 30: iOS and Android apps automation

Native frameworks

• Espresso – Android• Xcode UI testing - iOS

Page 31: iOS and Android apps automation

Espresso• Espresso is an open source Android test automation

framework from Google.• Espresso is API for UI Testing on Android.• Espresso core API is small, predictable and easy to learn.• Espresso is a thin layer on top of Android Instrumentation

that makes it easy to write reliable UI tests.• It supports all Android versions.• Tests run optimally fast as there is no need for any sleeps

(tests run on same millisecond when the app becomes idle)

Page 32: iOS and Android apps automation

Espresso: Code Examplepublic class EspressoTest extends ActivityInstrumentationTestCase2<HomeScreenActivity> { public EspressoTest() { super(HomeScreenActivity.class); } @Override public void setUp() throws Exception { super.setUp(); getActivity(); }

@Test public void testSearchHotel() { //finding a view with onView and perfoming an action onView(withId(com.hotwire.hotels.R.id.current_location_edittext))

.perform(click()); onView(withId(com.hotwire.hotels.R.id.destination)).perform(typeText("San Franciso\n")); onView(withId(com.hotwire.hotels.R.id.search_hotels_button)).perform(click()); }}

Page 33: iOS and Android apps automation

Android Demo

Page 34: iOS and Android apps automation

Xcode UI testing

• Available from Xcode 7• New feature added to XCTest framework• UI recording• Ease to locate elements• Run tests on multiple iOS versions• Integrates with Xcode server• Code coverage

Page 35: iOS and Android apps automation

But why? We already have Appium

- Faster execution time (1m45sec vs 5m45sec)

- Faster to write too (Record feature)- No Appium/3rd party dependencies- Free code coverage reports- Handles wait times implicitly - Snapshot of failed tests- Consolidated reports

Page 36: iOS and Android apps automation

UI testing : Code Exampleimport XCTest

class HotwireUITests: XCTestCase { override func setUp() { super.setUp() continueAfterFailure = false XCUIApplication().launch() }

func testIPadHotel() { //HomeScreen verification let homeScreen = HomeScreen() homeScreen.verifyHomeScreenElements() //Hotel Search with Signed In User homeScreen.navigateToHotel() //Hotel Farefinder verification let hotelFF = HotelFareFinder() hotelFF.verifyFareFinderElements() hotelFF.findHotel() }

import Foundationimport XCTest

class CarBookingScreen { let elementsQuery = XCUIApplication().scrollViews.otherElements let toolbarsQuery = XCUIApplication().toolbars func goToSignIn(){ elementsQuery.buttons["SIGN IN"].tap() } func completeBooking(){ enterCCV() elementsQuery.switches["AgeRestriction"].tap() elementsQuery.buttons["Book Now"].tap() }}

Page 37: iOS and Android apps automation

Xcode UI testing for iPad : Demo

Page 38: iOS and Android apps automation

Code coverage - iOS

• Tool : LCOV- Collects data from multiple

source files , creates HTML pages containing source code annotated with coverage

• GitHub - XCodeCoverage- Instrument classes- Run tests- Generate reports(./getcov)

Page 39: iOS and Android apps automation

Code coverage - iOS

Page 40: iOS and Android apps automation

public class Main {

private String s;

public static void main(String[] args){ //some code goes here if (something != null) { //body } }}

public class Main {

private String s; //magic infused

public static void main(String[] args){ //report that method was executed //some code goes here if (something != null) { //report that condition was met //body } }}

Original class Instrumented class

Code coverage – Android using Emma

Page 41: iOS and Android apps automation

iOS / Android CIGoalTo provide rapid feedback so that if a bug is introduced into the code base, it can be identified and fixed as soon as possible.

JenkinsWe are using Jenkins for all our CI runs at the moment, triggered on each Git check-in

iOS / Android CI Pipelines• Master Pipeline• Develop Pipeline • Release Pipeline• Feature Pipeline

PollingNodes/hosts from an iOS and Android host-cluster, polled for each leg of the pipeline execution. Artifacts are zipped and transferred between jobs.• Less wait time for resources• Parallel job execution

Page 42: iOS and Android apps automation

iOS CIBuild * Unit Tests * iPad Smoke Car/Hotel, iPhone Smoke Car/Hotel * Regression

Page 43: iOS and Android apps automation

iOS CI • Reporting & Snapshots

Page 44: iOS and Android apps automation

iOS CI

Page 45: iOS and Android apps automation

Screenshot testing

• Image comparison – Screen by screen. Why ?– Saves time testing UI layout on various sized

screens– New UI bugs caught faster– Helps reduce Regression Cycle Time

• ImageMagick tool

Page 46: iOS and Android apps automation

CI – Screenshot comparison

Page 47: iOS and Android apps automation

Analytics automation

• Log analytics / tracking information• Read Appium log file• Assert for expected values

Page 48: iOS and Android apps automation

So why are we doing all this?

• Feedback loop – faster @ story level• Regression – 0.5 to 1 day from 5 days• Speed with quality• Visibility

Page 49: iOS and Android apps automation

Hotwire automation framework

• To be Open sourced soon. Watch out -> https://github.com/HotwireDotCom/

Page 50: iOS and Android apps automation

Questions ?