ios-driverUsing Selenium to automate your iOS

applications

2Introduction

• Technical Test Lead at Paddypower• graham.abell@paddypower.com• https://github.com/nicegraham• Twitter: @nicegraham

• The Technical Test Team• SET – test frameworks, tools and mocking services• Performance Testing• Manage QA environments

• Involved with several OSS projects

3Testing mobile solutions

• The Problem - Several iOS applications in production, more on the way• Objective-C• Sencha• Hybrid• Pure web apps

• The Solution - Add automated functional testing to our CI process for mobile

4Our wishlist

5Apple’s Instruments framework

• Instruments is Apple’s iOS automation framework• It uses the UIAutomation JS library to interact with applications

• The good• Decent interface for running tests• The JS UI Automation library is extensive, well documented• Tight integration to Xcode• Does the heavy lifting of interacting with an application

• The not so good• Tied to a mac for running / debugging tests• No abstraction layer to run tests• Doesn’t scale• Browsing the Element Tree [DOM] is not fun

6Our wishlist

7Why Selenium?

• Clean Design• Supports any language or framework

• Stable API• Well on it’s way to becoming a w3c standard

• Already part of our framework• Minimum rework to add support for ios

• The team are used to working with it• BDD used – QAs and BAs create automated scenarios

• Scalability• Selenium provides Grid integration out of the box

8ios-driver

• What is ios-driver?• Used to drive user journeys on mobile UI• Exposes the underlying Element Tree in a navigable form• Implementation of Selenium API to drive iOS applications• Allows reuse of all existing Selenium infrastructure

• We tried NativeDriver – Google project to solve this problem

• We looked at several other projects in the space but ios-driver• Had the most interesting road map • Written in Java - suits our skills

9Our wishlist

10Who develops ios-driver?

• Francois Reynaud (owner Grid project, Se committer)

• Graham Abell, Darragh Grace (NativeDriver)

• Luke Inman-Semerau (iPhoneDriver, Se committer)

• Kevin Menard (Se committer)

• Daniel Wagner-Hall, Micahel Klepikov (Se committer, chromedriver)

• Martin Szulecki (libimobiledevice)

• Jason Evans

11How ios-driver works

Test Client

Java

C#

Ruby

Python

Selenium

Server

ios-driver

Instruments

WebDriver API

12Writing tests – Plain old Selenium

13Our wishlist

14Localisation

• You can only locate elements by their content

15Localisation

• Accessing the content

16Localisation

• Localizing the script

17Our wishlist

18Element Inspection

19Our wishlist

20Hybrid applications

• Web content can not be automated properly with Instruments

21Accessing web content in a native app

• Webkit Remote Debug Protocol provides the Webkit Web Inspector over the wire

22Accessing web content in a native app

Test Client

Java

C#

Ruby

Python

Selenium

Server

ios-driver

Instruments

WKRDPWebDriver API

23Writing tests for hybrid applications

Interact via Instruments

Switch context from native to web

Interact viaWKRDP

24Web applications - Mobile Safari

25Our wishlist

26Let’s go faster!

Client

HubClient

CI

Selenium Grid

nodenode

node

nodenode

node

nodenode

node

nodenode

node

27Selenium Grid Extras

28Our wishlist

29Adding Native Applications to CI

• The standard process for Selenium testing within CI• Application is deployed by CI tool• Tests are run via a browser • Browser versions change infrequently

• Therefore• There is low volatility with registered Nodes• Traditional nodes are immutable

• Native applications are different• They change on every commit

• An ios-driver node handles this volatility by• Monitoring for new applications• Stops accepting new test sessions• Registers the new capability• Re-opens for business

30Our Mobile Build Pipeline

• Jenkins slave running on a mac• XCode Jenkins plugin used to

• Build app in release and debug configurations• Unlocks keychain of Jenkins slave to sign app with correct Certs

• Xctool used to • Run the application unit tests• Run static code analysis

• Clokins plugin used to track lines of code• PMD used to detect duplicate code• GCOVR and Cobertura Jenkins plugin used for code coverage analysis• Successful builds kick off functional test with ios-driver• Testflight plugin to distribute good builds via Test Flight

31Our wishlist

32What’s next? – ios-driver 0.7.0 release

• New libimobiledevice integration• Better device support, more control over real device state• ios-driver nodes will be able to run on windows & linux• airplay

• Performance logging for MobileSafari• Will be able to generate HAR files

• New inspector• Limited record and playback

• iOS7 Support

33What’s next? – Selenium 3.0

• “a tool for user-focused automation of mobile and web apps” – Simon Stewart

• The Selenium Project will

• Ensure as much commonality in approach as possible

• Host a suite of tests to facilitate interoperability

• Add extensions to W3C

34

• Ios-driver• https://github.com/ios-driver/ios-driver• http://ios-driver.github.io/ios-driver/• https://groups.google.com/forum/#!forum/ios-driver• #ios-driver on freenode irc tends to be the best place for help

• Selenium Grid Extras• https://github.com/nicegraham/Selenium-Grid-Extras

• Selenium 3• http://seleniumhq.wordpress.com/2013/08/28/the-road-to-selenium-3/

• Jenkins• https://github.com/facebook/xctool• http://jenkins.quiedeville.org/job/Clokins/• http://pmd.sourceforge.net/• https://github.com/gcovr/gcovr• https://www.testflightapp.com

Links

35Any questions?