Calabash, an open-source automated testing technology for native mobile, by Karl Krukow
-
Upload
codemotion -
Category
Technology
-
view
4.545 -
download
1
description
Transcript of Calabash, an open-source automated testing technology for native mobile, by Karl Krukow
![Page 1: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/1.jpg)
Introducing Calabash
automated functional testing for mobile native apps
Karl Krukow, [email protected],
LessPainfulCodeMotion 2012, Rome
1
fredag den 23. marts 12
![Page 2: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/2.jpg)
About me
2
fredag den 23. marts 12
![Page 3: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/3.jpg)
About me
• PhD, Computer Science, University of Aarhus, 2006.
2
fredag den 23. marts 12
![Page 4: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/4.jpg)
About me
• PhD, Computer Science, University of Aarhus, 2006.
• Developer at Trifork for about 6 years mostly Java enterprise, last two years on iOS.
2
fredag den 23. marts 12
![Page 5: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/5.jpg)
About me
• PhD, Computer Science, University of Aarhus, 2006.
• Developer at Trifork for about 6 years mostly Java enterprise, last two years on iOS.
• Co-owner, iOS responsible at
2
fredag den 23. marts 12
![Page 6: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/6.jpg)
About me
• PhD, Computer Science, University of Aarhus, 2006.
• Developer at Trifork for about 6 years mostly Java enterprise, last two years on iOS.
• Co-owner, iOS responsible at
• Spare-time Hickey & Clojure fan-boy!
2
fredag den 23. marts 12
![Page 7: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/7.jpg)
Agenda
3
fredag den 23. marts 12
![Page 8: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/8.jpg)
Agenda
• Automated functional testing for native mobile
• Some desirable properties for a functional testing tool
3
fredag den 23. marts 12
![Page 9: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/9.jpg)
Agenda
• Automated functional testing for native mobile
• Some desirable properties for a functional testing tool
• Introduce Calabash
• Focus on iOS only
3
fredag den 23. marts 12
![Page 10: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/10.jpg)
Agenda
• Automated functional testing for native mobile
• Some desirable properties for a functional testing tool
• Introduce Calabash
• Focus on iOS only
• Live Demo:
• Calabash iOS
• LessPainful: test service and device cloud
3
fredag den 23. marts 12
![Page 11: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/11.jpg)
Functional testing
4
fredag den 23. marts 12
![Page 12: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/12.jpg)
Functional testing
4
• Functional and Acceptance tests
• Actual app, as opposed to an isolated component
• Often based on use-cases written in natural language (for domain)
• Visual appearance of screens matter! (Design guidelines, branding,..)
• As realistic an environment as practically possible
fredag den 23. marts 12
![Page 13: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/13.jpg)
Functional testing
4
• Functional and Acceptance tests
• Actual app, as opposed to an isolated component
• Often based on use-cases written in natural language (for domain)
• Visual appearance of screens matter! (Design guidelines, branding,..)
• As realistic an environment as practically possible
• For mobile apps, in particular
• Many devices, screens, OS versions, languages
• often a manual process: repetitive, expensive
• regressions: why should I test that again?
fredag den 23. marts 12
![Page 14: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/14.jpg)
5
Functional testing desiderata
fredag den 23. marts 12
![Page 15: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/15.jpg)
• Minimize distance between use cases and actual test code (DSLs?).
• Expressive and efficient to write.
• Extensible
• High-level, declarative (robustness against “minor” UI changes).
• Support testing in realistic environments (multiple real devices, on multiple OS versions, languages).
• Support Continuous integration.
5
Functional testing desiderata
fredag den 23. marts 12
![Page 16: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/16.jpg)
Cucumber and Calabash
fredag den 23. marts 12
![Page 17: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/17.jpg)
• Cucumber provides
• a framework for writing software specifications
• a tool for executing those specifications
• Specifications are written in a business readable language that is close to natural language.
• Extremely popular tool for test and specs of web applications.
• http://cukes.info/
fredag den 23. marts 12
![Page 18: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/18.jpg)
Cucumber Example
Feature: As an administrator. I want to be able to add and remove users,so I can control access to the application
Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr"
Scenario: ...
fredag den 23. marts 12
![Page 19: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/19.jpg)
Cucumber Example
Feature: As an administrator. I want to be able to add and remove users,so I can control access to the application
Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr"
Scenario: ...
fredag den 23. marts 12
![Page 20: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/20.jpg)
Cucumber Example
Feature: As an administrator. I want to be able to add and remove users,so I can control access to the application
Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr"
Scenario: ...
fredag den 23. marts 12
![Page 21: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/21.jpg)
Cucumber Example
Feature: As an administrator. I want to be able to add and remove users,so I can control access to the application
Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr"
Scenario: ...
fredag den 23. marts 12
![Page 22: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/22.jpg)
Cucumber Example
Feature: As an administrator. I want to be able to add and remove users,so I can control access to the application
Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr"
Scenario: ...
fredag den 23. marts 12
![Page 23: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/23.jpg)
Cucumber Example
Feature: As an administrator. I want to be able to add and remove users,so I can control access to the application
Scenario: Add test user When I touch the Add User button And I fill in text fields as follows: | field | text | | Last Name | Knorr | | Username | knorr | And I touch "Save" Then I should be on the Users screen And I should see a table containing "Knorr"
Scenario: ...
fredag den 23. marts 12
![Page 24: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/24.jpg)
Step Definitions• Make the cucumber tests “come alive”
• Written in ordinary programming languages
• Mostly Ruby (but cucumber-jvm: Java, Clojure,...)
Feature Step definitions
fredag den 23. marts 12
![Page 25: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/25.jpg)
Step Definitions• Make the cucumber tests “come alive”
• Written in ordinary programming languages
• Mostly Ruby (but cucumber-jvm: Java, Clojure,...)
Scenario: Add test user When I touch the Add User button ...
Feature Step definitions
fredag den 23. marts 12
![Page 26: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/26.jpg)
Step Definitions• Make the cucumber tests “come alive”
• Written in ordinary programming languages
• Mostly Ruby (but cucumber-jvm: Java, Clojure,...)
Scenario: Add test user When I touch the Add User button ...
When /^I touch the Add User button$/ do btn_txt = 'Add user' touch("button text:#{btn_txt}") end
Feature Step definitions
fredag den 23. marts 12
![Page 27: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/27.jpg)
Calabash• One interface: Cucumber, for Android and iOS tests.
• Predefined and custom steps (Ruby + soon: JVM).
• Reuse of tests across platform possible.
• Run on physical devices and simulators.
• Support for hybrid apps (embedded webviews)
• Free, open source with commercial add-ons:
• Run tests in a device cloud using the LessPainful service,
• we can write and run your tests,
• setting up continuous integration, consulting, training, support.
10
fredag den 23. marts 12
![Page 28: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/28.jpg)
Architecture iOS
iOS App--------------------Calabash library
fredag den 23. marts 12
![Page 29: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/29.jpg)
Architecture iOS
iOS App--------------------Calabash libraryCucumber
(ruby api)
features
Test report(cucumber)
fredag den 23. marts 12
![Page 30: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/30.jpg)
LessPainfulTest Execution in the Cloud
• Execute Calabash tests concurrently on many devices, OS’es, languages.
• Authentic: Not jailbroken, iOS and Android devices, rotation.
• Visual test reports.
• Comparison across models and operating systems.
• Continuous integration.
fredag den 23. marts 12
![Page 31: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/31.jpg)
fredag den 23. marts 12
![Page 32: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/32.jpg)
fredag den 23. marts 12
![Page 33: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/33.jpg)
fredag den 23. marts 12
![Page 34: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/34.jpg)
Calabash iOS: more detail
15
fredag den 23. marts 12
![Page 35: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/35.jpg)
Calabash: more detail• Very easy to get started for iOS developers/QAs.
• Declarative query language for finding views.
• Based on UISpec, but simplified and extended. (New Implementation, EPL licensed).
• Touch synthesis, supports (multitouch) gestures. Extensible. (pan, swipe, pinch, your own)
• Full power of Ruby programming language for test logic
• Interactive, exploratory development experience.
• Can use device accessibility for identifying views.
16
fredag den 23. marts 12
![Page 36: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/36.jpg)
Queries
17
fredag den 23. marts 12
![Page 37: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/37.jpg)
Queries • Queries are like CSS selectors or XPath
• label text:‘Hello‘
• label index:2
• view marked:‘thepane’ label
• view:‘MyClassName’
• label {text LIKE ‘Hel*’}
• webView css:‘#header a.cssclass‘
• webView xpath:’//node()’
17
fredag den 23. marts 12
![Page 38: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/38.jpg)
Demo: - Calabash iOS
- LessPainful Device Cloud
18
fredag den 23. marts 12
![Page 39: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/39.jpg)
iOS Comparisons• Several options available. To my knowledge:
• Calabash
• UIAutomation, Apple
• Zucchini, iOS Testing Framework
• Frank, Pete Hodgson, ThoughtWorks
• UISpec, http://code.google.com/p/uispec/
• FoneMonkey => MonkeyTalk, GorillaLogic
• KIF, Square
• NativeDriver, http://code.google.com/p/nativedriver/
19
fredag den 23. marts 12
![Page 40: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/40.jpg)
References• https://github.com/calabash
• https://github.com/calabash/calabash-ios
• https://github.com/calabash/calabash-ios/wiki
• https://github.com/calabash/calabash-ios-server
• https://github.com/calabash/calabash-android
• http://blog.lesspainful.com/
• https://www.lesspainful.com/
20
fredag den 23. marts 12
![Page 41: Calabash, an open-source automated testing technology for native mobile, by Karl Krukow](https://reader036.fdocuments.us/reader036/viewer/2022081400/554f597eb4c905b9508b52c6/html5/thumbnails/41.jpg)
Making app testing less painful...Please contact us with any questions:
[email protected]@lesspainful.com - iOS
[email protected] - Android
http://www.lesspainful.com
Questions?
fredag den 23. marts 12