Mobile Testing at Gilt

Matt Isaacs@haveahennessy

Gilt

What is Gilt?

Luxe brands up to 60% off

Inventory Changes DailySales last 36 hours.

Sales start at noon ET.Bonus sales at 9PM, Sundays and Wednesdays.

Gilt MobileThe simplified story

iPhone • iPad • Android

Developers focus on building the App(s). QA handles testing - All of it.

No automation.

Informal Process

When feature sets were simple. When the team was small.

When the codebase was small.

No Big Deal

– Some App Store Reviews

“Perfect App !!! Well done”“Awesome app! Love it!”

“OMG! Amazing!”“Your app is great and keeps getting better."”

We added more features. The codebase grew.

The team grew.

Some time passed

5-stars become harder to getYou can’t please everyone - But still…

Informal process doesn't scale

Add some structureCI + Nightlies with Jenkins. Dogfooding with Hockey.

Code + git branching conventions.

And tests…

– NSHipster

“Objective-C developers have, for the most part, remained relatively apathetic to Unit

Testing.”

How can this be?Objective-C is statically typed.

The compiler is awesome. TDD antipatterns - Cocoa makes heavy use of singletons.

~40% revenue comes from native mobile. Bad reviews hurt more than ever.

Increased pressure on QA.

Mounting cost of issues

So help out QAWrite some tests!

Bad habits are hard to breakMake time - Monthly testing hack-days.

Visibility - Make a scene when tests catch issues. Gentle, but firm approach.

Integration TestsUI + Functional tests. Notoriously difficult.

Provides the most benefit for QA.

AppiumSelenium webdriver for native mobile.

Actively developed. Open source.

Test

Appium

Instruments

App

HTTP

Unix Sockets

DTrace + Magic

Accessibility makes it work.Elements located via accessibility labels.

Be careful with container views. UIAccessibilityIdentifier is for tests.

Why we chose AppiumWe’ve already built Selenium infrastructure.

We already have Selenium skills. No SDK to compile in.

– Someone at Gilt

“Selenium is still too flaky. You can't trust the results. We need more dependable tests"

When should I wait? How long should I wait for?

Waiting

Where am I? Is it where I should be?

When should I be checking this?

State

Staging or Production? Painful issues with either.

Environment

Convention on top of Actions and Locators. Page validated during construction. Actions return new page objects.

Page Objects

When to wait? - Page object construction. Page objects represent state. Actions → State transitions

Solved!

Sore points.Accessibility.

Overlays and pass-throughs. Partially obscured controls.

End of the dayYou’ll need some process eventually.

It’s never too early - or late. TDD is not a religion.

Thankstech.gilt.com

@gilttech !

gilt.com/careers