Addressing the Challenges of Mobile Test Automation
Transcript of Addressing the Challenges of Mobile Test Automation
W8 Session 10/26/2016 11:30:00 AM
Addressing the Challenges of Mobile Test Automation
Presented by:
Pradeep Macharla
Inmar Inc.
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073 888---268---8770 ·· 904---278---0524 - [email protected] - http://www.starcanada.techwell.com/
Pradeep Macharla Inmar Inc. Pradeep Macharla is a passionate technical and business leader with over 13 years of experience managing, coaching and growing teams. Pradeep currently leads DevOps & Automation Practices at Inmar Inc. His background and experience has made him a highly sought after technical leader and mentor, consulting or working at the senior leadership and executive level. Pradeep has an MBA degree from Duke University that complements his technical ability to engage both business leaders and technical architects that brings clarity and simplicity to conversations.
Automate All The Things()
Mobile Test Automation
- Pradeep Macharla
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
What is the first word that comes to mind?
2
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Internet of Things (IoT)
3
If we think testing is hard
now, what about 50biillion
smart objects interconnected
on the planet.
Sooner or later, someone has
to test the solutions in the IoT
space.
Manual Testing cannot scale
at the rate of things added to
internet
Source: Amsix
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Mobile Device(s) – Consumer Interface to IoT
4
~4B – Mobile users
~51% - Mobile Penetration
~7.09B – Total mobile
subscriptions
Increasing exponentially..
Source: We are social 2015
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
‘Mobile’ doesn’t mean mobile (Mobile devices are used everywhere, not just when people are mobile)
5
Primary interface – Social, Cloud, Voice, Text, Identity (MFA)…
Can you live without a smart phone? (adjust for demographics)
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Form Factors
6
Getac
Time Clocks
Fitbit
Industrial devices
Smart Phones
Source: Flurry Analytics
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Mobile Software
7
Ecosystems – Apple, Google,
Windows etc.
iOS, Android, FirefoxOS,
Windows etc.
Native, WebApp, Hybrid
API levels, SDKs etc.
Source: Gartner, Statista
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Mobile Test Strategy
8
Manual vs. Automation
Scalable (think of AWS, Netflix)
Volume, Variety, Velocity
Cost (Open Source vs. Non)
Virtualization (Emulators)
Test Pyramid
Design & Frameworks
Source: www.seleniumframework.com
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Mobile Test Automation tools
9
Programming language – Start with
known, tend towards most collaborative
Inside-out / Outside-in (focus on end-user)
Don’t boil the ocean – Pick one and pivot
Interoperability - Cloud & on-premises infra
Tend towards using Real Devices
Appium, espresso & XCTest
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Mobile Test Execution Environment - CSFs
10
Hardware – Variety,
accessibility, form factors,
own vs. lease
Software – On-premises,
cloud, open standard, lightly
coupled
Support – Stack Overflow,
self-service
Source: Suresh Vadigi, Capital One
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Scope for Improvement
11
External app context support (e.g. upload photos)
Parallel execution on multiple devices/OSes
Richer Test Execution Reports
GPS location Simulation
Global coverage with devices and operating systems
Performance Testing
Simplify networking (understand and isolate network segments)
Security Testing
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Reference Pattern #Appium CI/CD/CT
12
Build-Deploy-Test Feedback
loop
Keep the automated tests
updated with changing
requirements
Self-service execution (&
feedback)
Access to source code
Version your app
Centralize Dependency
management
Source: https://leanpub.com/ci-mobile-app
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Reference Pattern #Appium Architecture
13
Client-Server Architecture
Session based
Desired Capabilities (JSON over
wire)
Appium.app, Appium.exe
Multi-language support client
side
Relies on Android OS and SDK
support for interaction
GUI vs. headless server
Source: https://leanpub.com/ci-mobile-app
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Reference Pattern #Appium Device Lab
14
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Reference Pattern #Appium BDD framework
15
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Reference Pattern #Appium Code Behind
16
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Demo
17
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Reference Architecture #Appium iOS
18
Appium server talks to UI
Automation
UI Automation translates into
click, set, get
GUI vs. headless server
Future UIAutomation support
dropped by Apple, but still
available from Instruments
Source: https://leanpub.com/ci-mobile-app
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Parallel Execution – At Scale
19
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
Recommendation(s)
20
Automation has little value without being idempotent
Mobile Automation requires increased focus on details (compared to web)
Utilize Cloud based solutions before buying devices (trials are free)
Build and Deploy tasks should be self-serviced (automated)
Be curious but do your groundwork upfront
Participate and contribute to online forums (people are nicer if you can
articulate with context)
Most of the technology challenges are easier to solve
Focus on collaboration and democratic sharing of information within your
team
Inmar confidential – do not copy, distribute or use without Inmar written permission, 2016
References
21
Speaker Pradeep Macharla
Email [email protected]
Github https://github.com/machzqcq
Twitter @macharla
Book https://leanpub.com/ci-mobile-app
Cell +1.919.491.1239
Website https://www.seleniumframework.com