Stop guessing, start testing – mobile testing done right - Timo Euteneuer
-
Upload
jaxlondonconference -
Category
Software
-
view
285 -
download
1
Transcript of Stop guessing, start testing – mobile testing done right - Timo Euteneuer
![Page 1: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/1.jpg)
![Page 2: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/2.jpg)
About Me
TestObject GmbH Co-Founder I Sales
Mobile Quality Crew’s Co-Founder I Organizer Contact: @testobject I timo@testobject
![Page 3: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/3.jpg)
Session
1) Mobile app testing: Why it is important
2) Challenges
3) Different testing approaches
4) Automation as key factor for success
![Page 4: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/4.jpg)
Familiar?
48 % of users won‘t use an
app again after encountering a
major issue!
Source: Compuware Global Study Mobile Apps 2013
![Page 5: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/5.jpg)
Better Find Bugs Before Users Do
![Page 6: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/6.jpg)
Bad Quality = Less Revenue
Quality
Ratings
Downloads
Revenue
![Page 7: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/7.jpg)
App Testing Challenges
![Page 8: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/8.jpg)
Device Fragmentation
http://opensignal.com/reports/2015/08/android-fragmentation/
![Page 9: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/9.jpg)
OS Fragmentation
http://opensignal.com/reports/2015/08/android-fragmentation/
![Page 10: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/10.jpg)
Screen Size Fragmentation
http://opensignal.com/reports/2015/08/android-fragmentation/
Android iOS
![Page 11: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/11.jpg)
... More Challenges
§ Mobile means your users are not in a protected environment
§ Network (Wifi, 3G, GPRS ...) § Sensors, Bluetooth ...
![Page 12: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/12.jpg)
Learnings
§ Your users are not very patient
§ Quality has direct impact on apps‘ success
§ Challenges: Fragmentation and mobile
use of apps
§ Implication: Mobile QA is a central part of development
![Page 13: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/13.jpg)
There must be an easy solution to handle this ?!
![Page 14: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/14.jpg)
...nope!
![Page 15: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/15.jpg)
What Do I Have to Test?
§ What to test: § Usability § Functionality § Performance § Edge Cases
§ Approx. 50 use cases (per platform) § Approx. minimum 20 target devices
![Page 16: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/16.jpg)
Testing Framework
Development QA Production
Unit Tests
Functional Tests
Usability
Sanity
Monitoring
Automation
Manual Edge Cases
Checkups
![Page 17: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/17.jpg)
Manual Approaches
§ Checkups, Usability, Edge Cases, Performance § Pros: You can test the actual user behavior
§ Cons: Time consuming, inefficient
![Page 18: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/18.jpg)
Continuous Delivery = Continuous Testing
Build
Test
Deploy
Enhance
![Page 19: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/19.jpg)
UI Automation Approaches
§ Regression, End-to-end, Integration § Pro: High test coverage, scalable, time-to-market,
releases resources for manual testing
§ Con: Initial investment, requires technical know-how, maintaining of scripts, no edge cases
![Page 20: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/20.jpg)
Automated Testing
ROI
Investment
Test Cycles
Resources
![Page 21: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/21.jpg)
UI Automation Components
§ CI & Build: Jenkins, Bamboo, Travis CI, Gradle, Ant, Maven
§ Scripting Tool: Proprietary or Open Source Tools § Infrastructure: e.g. Spoon, Selenium Grid,
SauceLabs, TestObject ...
![Page 22: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/22.jpg)
UI Automation Frameworks
iOS Android Web Language
Calabash
x x - Ruby
Robotium - x - Java
Espresso
- x - Java
Appium
x x x Java, JavaScript, Python, Ruby ...
![Page 23: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/23.jpg)
![Page 24: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/24.jpg)
Code Example Appium
![Page 25: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/25.jpg)
Example Test Execution
![Page 26: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/26.jpg)
Learnings
§ Quality is crucial but mobile QA is complex
§ Different automation and manual testing approaches are required
§ For decent test coverage and time-to-
market UI test automation is key
![Page 27: Stop guessing, start testing – mobile testing done right - Timo Euteneuer](https://reader034.fdocuments.us/reader034/viewer/2022051707/58ed9c6b1a28ab16568b46d3/html5/thumbnails/27.jpg)
Thank you!