10 things you didnt know about appium + whats new in appium 1.5
Automating Mobile Testing at Gilt with Appium
-
Upload
sauce-labs -
Category
Technology
-
view
3.995 -
download
1
description
Transcript of Automating Mobile Testing at Gilt with Appium
![Page 1: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/1.jpg)
Appium Better PracticesLessons learned the hard way
Matt IsaacsGilt
![Page 2: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/2.jpg)
What is Gilt
![Page 3: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/3.jpg)
Short story
Luxe brands up to 60% off
Inventory changes daily. - Sales last 36 hours.
Sales start at noon ET. - Bonus sales at 9PM, Sundays and Wednesdays.
![Page 4: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/4.jpg)
Gilt Mobile
![Page 5: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/5.jpg)
Goals
Understand how users use their devices.
Understand the capabilities of each device.
Understand our users and how the want to interact with Gilt.
Put these all together to create experiences that surprise and delight our users.
![Page 6: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/6.jpg)
A brief tour
![Page 7: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/7.jpg)
How we roll
By hand, with rigorous QA.
Engineers focus on app features + performance + fixes.
QA crew in Jordan takes over at the end of the day.
Repeat.
![Page 8: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/8.jpg)
Seems to be working
"Perfect App !!! Well done"
"Awesome app! Love it!"
"OMG! Amazing!"
"Your app is great and keeps getting better."
"I LOOOOOOOVE YOU GUYS AND YOUR APP!"
![Page 9: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/9.jpg)
Or is it..
"Since your update this app runs slower than ever"
"How do you get rid of this. It is like a incurable disease."
"Never using again. Sister and mom and friends have removed also. Horrible"
Some issues aren't Gilt Mobile's fault. But still..
![Page 10: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/10.jpg)
It gets worse
![Page 11: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/11.jpg)
Mobile First. So hot right now
Revenue from Mobile at ~40% and growing.
Future of Gilt is in our hands.
Increasingly involved with other teams at Gilt.
Negative experience becomes more costly.
![Page 12: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/12.jpg)
Informal Process
Doesn't scale!
Leads to questions: - Who's doing the build tonight? - What's in this build? - Which build must QA test?
Unit tests are run at the discretion of today's hand roller.
Surprise! Tests never get run.
![Page 13: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/13.jpg)
What does it mean?
Potential for wasted QA time is high.
Expensive resource being used improperly.
Tough for outsiders to work with Gilt Mobile.
More shocking: We're not the only ones.
![Page 14: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/14.jpg)
How can this be?
Tools have not kept up.
Apple tools lean in favour of the IDE.
xcodebuild test issues.
UIAutomation is part of intsruments.
XCode Bots is nice, but not quite ready.
Our way is easier.
![Page 15: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/15.jpg)
Time for change
![Page 16: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/16.jpg)
Road to automating
Change gradually.
CI - Nightly builds with Jenkins.
Internal distribution. Dogfooding!
Dust off old unit testing efforts.
UI Testing
![Page 17: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/17.jpg)
What's the big deal with UI tests
Special time saving powers for QA.
Sanity tests are time consuming.
STR can also be time consumig.
UI tests are QA enablers.
![Page 18: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/18.jpg)
Appium
![Page 19: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/19.jpg)
Why?
Philosophy works for us.
Test the app you release
Language/Framework independent.
Uses popular standard API - Selenium WebDriver.
Thriving open source community.
Support for multiple mobile platforms.
![Page 20: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/20.jpg)
Sounds great but...
Sometimes tests work. Sometimes they don't.
Reliability issues widespread enough to prompt development of Flaky tool.
UIAutomation is the root of a lot of the chaos. - "Instruments unexpectedly quit" - Random delays when searching tableviews.
Sometimes Apple breaks it altogether - Xcode 5.0.1.
Expect some misery on iOS + XCode major releases.
One test at a time. Sauce has you covered.
![Page 21: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/21.jpg)
The setup at Gilt
Basic set of sanity tests. - New user registration. - Existing user sign on. - Product purchase flow. - Product Waitlist flow.
Scala + ScalaTest. Allows us to leverage existing Selenium infrastructure. Why reinventthe wheel?
Still a little too brittle.
![Page 22: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/22.jpg)
Some lessons learned.
![Page 23: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/23.jpg)
Accessibility
Not just good for your user base.
UIAutomation leverages accessibility to drive your app.
Accessibility = Testability.
Setting AccessibilityLabel and isAccessibilityElement only takes you so far.
Container views are troublesome. If marked as accessible, their contents will bemasked. UIAccessibilityContainer has you covered.
![Page 24: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/24.jpg)
The special case
Table cells are containers and so need the aforementioned special care.
UIAccessibilityContainer might not be necessary.
Set the accessibility label of the cell, unless you like surprises.
![Page 25: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/25.jpg)
Product Detail Page Example
PDP Navbar title AccessibilityLabel = Product Name (which is what it should be)
How can a test identify this page?
Dynamic Accessibility labels? UIAccessibilityIdentification is for you.
"Using an identifier allows you to avoid inappropriately setting or accessing anelement’s accessibility label."
![Page 26: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/26.jpg)
Complex actions.
What if we want to search for unavailable products?
For each sale: - Wait for the product list and inventory to load, - Search elements for "Sold Out" or "Reserved" - Do some string processing to determine which of the products for a given cell isunavailable.
Or we could just use our API.
Complex navigations and searches with many moving parts can be susceptible toreliability issues.
![Page 27: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/27.jpg)
Patterns
Action - Locator has a couple issues that are more promininent on mobile. - No state. What page am I on? - No convetion: When do I check page? Before or after? - Found ourselves fine tuning delays.
Page Objects formalize some conventions on top of Actions and Locators. - All actions return page objects - No assertions in page object actions.
Never reuse page objects. Element IDs sometimes change.
Some page actions dont always return the same page. Use page source to identifypage.
Since we dont have the luxury of navigating by URL. - Restart the driver - Recovery calls
![Page 28: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/28.jpg)
Final nugget of sage wisdom
Be wary of "clever" UI hacks. Expect testing + accessibility misery.
Pass throughs and overlays: - Underlying views may be visible by you, but can't be acted upon because they are"hidden". - Tests become an error prone jumble of "mobile: tap" calls.
![Page 29: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/29.jpg)
Appium needs you
Use it.
File those radar issues for Instruments and UIAutomation.
Contribute to Appium! - https://github.com/appium
IRC: #appium on freenode
![Page 30: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/30.jpg)
Special Thanks
Sauce for hosting.
Appium community.
Everyone that joined and is still with us.
![Page 31: Automating Mobile Testing at Gilt with Appium](https://reader035.fdocuments.us/reader035/viewer/2022081403/554d2bf2b4c905c5208b51bb/html5/thumbnails/31.jpg)
Thank you
Matt IsaacsGilthttp://gilt.com (http://gilt.com)
@gilttech (http://twitter.com/gilttech)
@haveahennessy (http://twitter.com/haveahennessy)