Ensuring quality in cloud and mobile applications
description
Transcript of Ensuring quality in cloud and mobile applications
1 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Ensuring Quality in Cloud and Mobile Applications
Gerard Meszaros [email protected]
2 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Product & I.T.
I.T.
Embedded Telecom
My Background
Gerard Meszaros [email protected]
• Software developer
• Development manager
• Project Manager
• Software architect
• OOA/OOD Mentor
• Requirements (Use Case) Mentor
• XP/TDD Mentor
• Agile PM Mentor
• Test Automation Consultant & Trainer
• Lean/Agile Coach/Consultant
80’s ----- 90’s ----- 00’s
3 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Agenda • What’s Special About Cloud and Mobile Apps? • What is Quality? • A Quality Strategy • Testing Cloud-Based Apps • Testing Mobile Apps • No DOH! • Ensuring Design for Testability • Tools, Frameworks and Patterns
4 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Current Challenges
The Good Old Days • Captive Audience
– Have no real choice
• Stable requirements – Can have a single release
• Long lead times – Can have long manual test
cycle
• Relatively simple environment – Just a few inputs (user, DB) – Mostly synchronous
Current Reality
• Users Have Real Choice – Many alternatives
• Rapid change – Highly iterative; multi releases
• Rapid change – No time for extensive manual
testing
• Highly Complex Environment – Many sources of inputs – Sync and Async
5 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
DB Server Desktop
Other System
Other System
User Interface
Business Logic
OS Services
Other System
Typical Desktop Application • UI Logic and
Business Logic run on desktop
• Server hosts the database.
Database
6 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Desktop Browser
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Typical Cloud-Based Application • UI Logic runs in
browser • Business Logic is split
between Server and Browser
• Both use browser services
– Widgets, Cookies, SSL,DOM rendering
• Business logic may also interact with other systems.
– E.g. IP-based location • Browser-based BL
may receive data from server: – synchronous
(request-reply) – asynchronous
(event-based)
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
Dynamic scripting languages • won’t report compile errors until run.
7 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Device Browser
Sensors GPS Acc
Gyro
OthrApps Contacts Msg’ing Gyro Batt
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Typical Web-Based Mobile Application • UI Logic runs in
browser • Business Logic is split
between Server and Browser
• Both use browser services
– Widgets, Cookies, SSL,DOM rendering
• Business logic may also interact with other systems.
– E.g. IP-based location • Browser-based BL
may receive data from server: – synchronous
(request-reply) – asynchronous
(event-based)
Business Logic
Browser Services
Platform Services
(A.K.A. Software-as-a-Service or SaaS)
8 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Device
Server
Other System
UI Logic
Business Logic Database
Container Services
Other System
Typical Mobile Application • UI Logic runs in
device • Business Logic is split
between Device and Servers
• Both use Platform services
– Config, Prefs, Widgets,SSL,
• Business logic may also interact with other systems.
– E.g. IP-based location • Browser-based BL
may receive data from server: – synchronous
(request-reply) – asynchronous
(event-based)
Business Logic
Platform Services
Sensors GPS Acc
Gyro
OthrApps Contacts Msg’ing Gyro
Config Prefs
Time Prefs
Batt
9 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Outputs / Controlables • Screen (LCD) • LED indicators • Vibration motor • Camera • Wifi, Bluetooth, 3G,
NFC Radios (on/off)
• Other Applications • Add event to
Calendar • Add/edit/read contact
10 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Other static inputs Physical Configuration • Screen size • Screen density • Roles of Physical
buttons • Camera(s)
capabilities
User Preferences • Language / locale • Keyboard prefs • Application prefs
11 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Sources of Events Sensors • Proximity • Accelerometer • GPS • Signal Strength (Wifi,
3g, BT, NFC) • Compass
Services • Send message (e-mail,
SMS, MMS, IAM) • Voice interpretation • Output Audio • IP Location • Wifi Location • GPS Location • Get/Show Advertising • In-App purchasing
12 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Events & Reponses • Low Battery • Lost Data
Connection • Change in data
capabilities • Geofence crossed
• Valid Responses • Data returned sync • Data returned async • Error responses • Low resources • Can’t communicate
13 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Example – Screen Layout • Large number of available screen sizes • Need to verify app uses screen space well • Four different orientations • For each screen size and orientation:
– need to verify how various screen elements are sized and placed
• May be possible to do manually for IOS – iPhone 1, iPhone 4, iPhone 5, iPad1/2, iPad3, iPod Touch
• Not possible to do well manually for Android – Automation is essential; best done at component level
14 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Example – Screen Layout • What about changes to orientation of the
device? • That’s 4P2 or 12 transitions we need to test
North ->South North -> East North -> West
• And we have to run this test at various states of the app
• So, 12 tests in 10’s or 100’s of states • On 10’s to 100’s of device configurations
South -> North South -> East South -> West
East -> North East -> West East -> South
West -> North West -> East West -> South
15 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Agenda • What’s Special About Cloud and Mobile Apps? • What is Quality? • A Quality Strategy • Testing Cloud-Based Apps • Testing Mobile Apps • No DOH! • Ensuring Design for Testability • Tools, Frameworks and Patterns
16 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Google Quality Checklists Core App Quality Quality Criteria • Design and Interaction • Functionality • Performance and Stability • Google Play Testing • Setting Up a Test
Environment • Test Procedures You Should Also Read • Tablet App Quality Checklist • Improving App Quality
Tablet App Quality 1. Test for Core App Quality 2. Optimize your layouts 3. Use the extra screen area 4. Use assets designed for
tablets 5. Adjust fonts and touch targets 6. Adjust HomeScreen widgets 7. Offer the app's full feature set 8. Don’t require hardware
features 9. Declare tablet screen support 10. Follow best practices for
publishing in Google Play http://developer.android.com/distribute/googleplay/quality/core.html http://developer.android.com/distribute/googleplay/quality/tablet.html
17 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
What Exactly is Quality?
Building the Right Thing • Fitness-for-purpose • Understand the users • And their usage env’ts
Practices: • UxD • Usability testing • Lean Startup stuff
Building the Thing Right • Good architecture • Well implemented in
code • Well tested at all levels
Practices: • Automated Unit Testing • Test-Driven
Development • ATDD • Exploratory Testing
18 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Traditional Test Cycle
Development Verification & Acceptance
Software
Requirements Requirements
19 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Agenda • What’s Special About Cloud and Mobile Apps? • What is Quality? • A Quality Strategy • Testing Cloud-Based Apps • Testing Mobile Apps • No DOH! • Ensuring Design for Testability • Tools, Frameworks and Patterns
20 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
How to Achieve Quality?
21 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
What Should We Test? • Product X consists of components A, B, and C • Components A and B are well tested but have
never been used together. • Component C is not well tested
• We need to Component-test C, • Interaction test A vs. B • Interaction test A vs C
22 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
A Quality Strategy • Use automated tests at unit and component
level – to maximize confidence in quality of components.
• Use system-level tests focusing on the logic not already tested – mostly integration tests – automated if possible, manual otherwise
• Use Exploratory testing to find out what else could possibly go wrong. – Let testers follow their hunches to find problem areas
23 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
A word about Terminology • Testing vs. Checking
– Checking is verifying that a behaviour occurs as expected
– Testing is seeing what happens when I do something – Automated Testing is therefor an oxymoron; it should be
automated checking. • Direct vs Indirect Inputs (and outputs)
SUT methodA( )
methodB( ) Client
Client
Client Objects (Users)
DOC
DOC
DOC
Depended-on Components
Client
Indirect Output
Input
Output
Direct Input
Indirect Input
Direct Output
System Under Test
24 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Test Automation Pyramid
Unit Tests
Component Tests
System Tests
Pyramid originally proposed by Mike Cohn
Exploratory Tests • Tools to support effective
exploratory testing • A small number of tests
for the entire application & workflow – Ensure application(s) support
users’ requirements • Medium number of
functional tests for major components – Verify integration of units
• Large numbers of very small unit tests – Ensures integrity of code
25 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Agenda • What’s Special About Cloud and Mobile Apps? • What is Quality? • A Quality Strategy • Testing Cloud-Based Apps • Testing Mobile Apps • No DOH! • Ensuring Design for Testability • Tools, Frameworks and Patterns
26 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Cloud-Based Application • Natural interface
between Browser & Server facilitates testing – Server testing – Browser testing with
Server stubbed out • Component tests for
Browser Business Logic bypassing UI – Requires API on BL
• Also need unit tests for Business Logic code
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
27 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Cloud-Based Application • Natural interface
between Browser & Server facilitates testing – Server testing – Browser testing with
Server stubbed out • Component tests for
Browser Business Logic bypassing UI – Requires API on BL
• Also need unit tests for Business Logic code
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
Server Test
28 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Cloud-Based Application • Natural interface
between Browser & Server facilitates testing – Server testing – Browser testing with
Server stubbed out • Component tests for
Browser Business Logic bypassing UI – Requires API on BL
• Also need unit tests for Business Logic code
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
Whole Client Test
Server Test Stub
29 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Cloud-Based Application • Natural interface
between Browser & Server facilitates testing – Server testing – Browser testing with
Server stubbed out • Component tests for
Browser Business Logic bypassing UI – Requires API on BL
• Also need unit tests for Business Logic code
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
Unit Test
Test Stub Unit
Unit Test
Test Stub Unit
Unit Test
Test Stub Unit
Unit Test
Test Stub Unit
30 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Cloud-Based Application • Natural interface
between Browser & Server facilitates testing – Server testing – Browser testing with
Server stubbed out • Component tests for
Browser Business Logic bypassing UI – Requires API on BL
• Also need unit tests for Business Logic code
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
Business Logic Test
Server Test Stub
Platform Services Test Stub
31 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Cloud-Based Application • Natural interface
between Browser & Server facilitates testing – Server testing – Browser testing with
Server stubbed out • Component tests for
Browser Business Logic bypassing UI – Requires API on BL
• Also need unit tests for Business Logic code
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
Server Logic Test
DB Test Stub
Platform Services Test Stub
32 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Browser
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Cloud-Based Application • Natural interface
between Browser & Server facilitates testing – Server testing – Browser testing with
Server stubbed out • Component tests for
Browser Business Logic bypassing UI – Requires API on BL
• Also need unit tests for Business Logic code
Business Logic
Browser Services
OS Services
(A.K.A. Software-as-a-Service or SaaS)
Whole System
Test
33 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Special Considerations • Different Browsers and Browser-Versions
– Eg. Rendering behaviour, widget behavior, etc. • Different Host-OSes
– For OS-based services such as file choosers, etc. • Different languages & locales • Browser resizing
34 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Device
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Browser-Based Mobile Application Similar to testing
Cloud-based apps but:
• Different browsers & Screen sizes
• Need to test with events from relevant sensors – E.g. Orientations
and changes
Business Logic
Platform Services
Sensors GPS Acc
Gyro
OthrApps Contacts Msg’ing Gyro
Config Prefs
Time Prefs
Batt
35 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Device
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Browser-Based Mobile Application Similar to testing
Cloud-based apps but:
• Different browsers & Screen sizes
• Need to test with events from relevant sensors – E.g. Orientations
and changes
Business Logic
Platform Services
Sensors GPS Acc
Gyro
OthrApps Contacts Msg’ing Gyro
Config Prefs
Time Prefs
Batt
Business Logic Test
Server Test Stub
Platform Services Test Stub
36 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Device
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Native Mobile Application • To test device BL,
need to: – Provide API
bypassing UI – Trap outgoing
reqests to sensors and services
– Inject responses from same
• Need ability to: – Stub out platform
services, sensors and other apps
• Need unit tests for all code
Business Logic
Platform Services
Sensors GPS Acc
Gyro
OthrApps Contacts Msg’ing Gyro
Config Prefs
Time Prefs
Batt
37 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Device
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Native Mobile Application • To test device BL,
need to: – Provide API
bypassing UI – Trap outgoing
reqests to sensors and services
– Inject responses from same
• Need ability to: – Stub out platform
services, sensors and other apps
• Need unit tests for all code
Business Logic
Platform Services
Sensors GPS Acc
Gyro
OthrApps Contacts Msg’ing Gyro
Config Prefs
Time Prefs
Batt
Business Logic Test
Server Test Stub
Platform Services Test Stub
38 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Server Device
Other System
UI Logic
Business Logic Database
Container Services
Other System
Testing Native Mobile Application • To test device BL,
need to: – Provide API
bypassing UI – Trap outgoing
reqests to sensors and services
– Inject responses from same
• Need ability to: – Stub out platform
services, sensors and other apps
• Need unit tests for all code
• Characterization Tests to detect changes in dependencies
Business Logic
Platform Services
Sensors GPS Acc
Gyro
OthrApps Contacts Msg’ing Gyro
Config Prefs
Time Prefs
Batt
Dependency Characterization
Tests
39 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Agenda • What’s Special About Cloud and Mobile
Apps? • What is Quality? • A Quality Strategy • Testing Cloud-Based Apps • Testing Mobile Apps • No DOH! • Ensuring Design for Testability • Tools, Frameworks and Patterns
40 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
DOH! • DOH! = Debugging on Hardware • Should be avoided because • Slow, tedious, non-deterministic • Complete code coverage is impossible
because we cannot cause on-demand: – Hardware errors – “Impossible” return codes or exceptions – All but most common events from sensors
You have to ask yourself the following question: Do I feel lucky?
41 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
James Grenning on DOH! • Reasons why DOH is not feasible:
– There is no hardware, it’s being concurrently developed. – The hardware is expensive, so team members have to
share it. – DOH is slow, with tools that may be less capable than
the development system native tools. – The hardware has bugs, as if DOH! isn’t slow enough;
buggy hardware makes it extra difficult. – Building code for the target is slow. – Downloading to the target is slow. – The embedded processor is probably slow too.
42 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Test in IDE (Automated)
Test in Emulator
(Automated)
Test on Hardware (Automated)
Test in Acceptance
(manual)
How No DOH? • How do we avoid Debugging on Hardware?
Unit Tests Component Tests
System Tests
Which tests:
When: Day 1 As soon as Available Weekly
43 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Test in IDE (Automated)
Test in Emulator
(Automated)
Test on Hardware (Automated)
Test in Acceptance
(manual)
No DOH How • How do we avoid Debugging on Hardware?
Frequency: Every few minutes Daily Weekly+
Each code change Per component test Per user story
44 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Agenda • What’s Special About Cloud and Mobile
Apps? • What is Quality? • A Quality Strategy • Testing Cloud-Based Apps • Testing Mobile Apps • No DOH! • Ensuring Design for Testability • Tools, Frameworks and Patterns
45 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
How Do We Ensure Testability? “Design for Testability”, then code • Write tests last • And hope it’s easy
Or • Code for Testability • Via Test-Driven Development (TDD)
46 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
TDD Rhythm
Test Code • First Test • Second Test • Third Test • Fourth Test
Product Code • Hard-coded method • Introduce variable • Introduce conditional • Surround with a loop
Just Like Double-Entry Booking: • An entry on the test side for each entry on
the Prod side
47 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Test - After Architecture
System Under Test
• Must test through User Interface
Should we Notify? Process
Transaction
Configure Notification Threshold
Notification Rules
Transaction Interface
Configuration User
Interface
Notification Log
Workflow Test
Do Notification.
48 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
System Under Test
Test-Driven Architecture • Need to provide API’s to invoke functionality
directly
• And ways to stub out dependencies
Should we Notify?
Do Notification.
Process Transaction
Configure Notification Threshold
Notification Rules
Notification Log
Transaction Interface
Configuration User
Interface Workflow
Test Test Stub
49 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Test-Driven Architecture • “What kind of tests do I want to be able to automate?” • “Which component would be responsible for that part
of the behavior?”
Should we Notify?
Do Notification.
Process Transaction
Configure Notification Threshold
Notification Rules
Notification Log
Transaction Interface
Configuration User
Interface
Configuration TX Test
50 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Test-Driven Architecture • “What other components would that component
depend on?” • “How can I break that dependency when component
testing this component?”
• With the right architecture, automating these tests is trivial
Should we Notify?
Do Notification.
Process Transaction
Configure Notification Threshold
Notification Rules
Notification Log
Transaction Interface
Configuration User
Interface Notification Rule Test
Notification Method Test
Notification Rules Fake
51 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Unit Tests
Component Tests
System Tests
Workflow
Transactions
Behavior Specification at Right Level • Specify broad scope at minimum detail
– E.g. Use least detail when specifying workflow • Specify most detailed req’ts at narrowest scope
– E.g. Don’t use workflow when specifying business rules
Make examples / tests easy to
understand and easy to write
Broad Narrow Scope
Det
ail
High
Lo
w
Business Rules
Too vague
Too much detail Unmaintainable
Unit Tests
52 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Agenda • What’s Special About Cloud and Mobile
Apps? • What is Quality? • A Quality Strategy • Testing Cloud-Based Apps • Testing Mobile Apps • No DOH! • Ensuring Design for Testability • Tools, Frameworks and Patterns
53 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Tools, Frameworks & Patterns • Tools come & go
– Different ecosystems (Android, IOS, BBOS, WP, etc.) – New, better tools appear regularly
• Strategies persist – Need to define strategy first
• Then choose tools to support the strategy • And Patterns for testable designs
Avoids “Impedance Mismatch” between tools and strategy
54 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Framework Issues • Many frameworks not designed for testability • May need to introduce level of indirection to
API (via Proxy or Adapter) • May need to extract Plain Ordinary Testable
Object (POTO) leaving a Humble Object that subclasses framework base class
55 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Extract POTO - Before
Framework Plug-in
ActionBase
MyAction
doAction()
MyActionTest
testDoAction()
tests Rules
ActionContext
+rules
getRules()
56 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Framework Plug-in
Extract POTO - After
Runs in device; Tests “wrapping”
(Optional)
Runs in IDE; Tests “business logic”
MyActionHOTest
testDoAction()
ActionBase
MyActionHO
doAction()
tests
MyActionPoto
doAction(rules)
MyActionPotoTest
testDoAction() tests
Rules
ActionContext
+rules
Humble Object
Testable Object
Rules
Fake Rules
Simple design change; Huge testability improvement!
getRules()
57 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Conclusions • Mobile and Cloud-Based Applications come
with special challenges – While business logic tends to be simpler, constraints
and variations make design and testing more challenging
• Testing-in Quality is not an option – There are too many potentially interacting dimensions – Many are not controllable from outside the black box
• Need to avoid Debugging on Hardware – Test as much as possible “off device” – Only deploy code you know works correctly
58 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Conclusions • Design-for-Testability is essential for success
– Otherwise, tests will be too expensive to execute and/or automate
• Beware tools/frameworks that make DfT difficult – May need to decouple from framework to ensure
testability – But extra effort is compensated for by better quality and
productivity
59 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
What Can You Do? • Product Owner:
– Break work into manageble chunks – Provide examples/tests for each chunk to drive
development – Insist that team automates the tests
• Tester/BA: – Provide tests to developers Before they start dev – Insist on seeing the results of the automate tests before
you start testing – Focus manual testing on areas not covered by
automated tests
60 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
What Can You Do? Developer: • Ask for tests from PO/Testers before agreeing to build • Discuss the kinds of tests that they will provide • Include the work to automate the tests in any
estimates you provide • Implement the automation before you develop the
code, not after! • Use the automated tests to drive development • Don’t recreate provided tests at unit test level; find
ways of reusing business rule tests as unit tests • Think/Do Design-for-Testability • If you find yourself in an interactive debugger, ask
yourself: “What unit tests would tell me the same information and prevent this from happening again?”
61 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
References • James Grenning “Don’t Let Embedded Tool Chain
Slow You Down” – http://www.renaissancesoftware.net/blog/archives/41
• Michael Bolton “Testing vs. Checking” – http://www.developsense.com/blog/2009/08/testing-vs-checking/
• Dion Johnson “Cyber-dueling Over ‘Check’ vs. ‘Test’ and Other Semantics” – http://www.automatedtestinginstitute.com/home/index.php?
option=com_content&task=view&id=1373 • Gerard Meszaros “Humble Object”
– http://xunitpatterns.com/Humble%20Object.html
• Homer Simpson “Doh!” – http://en.wikipedia.org/wiki/D%27oh!
62 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Resources • http://www.mobileapptesting.com/ • For Success, Build Record/Playback into Your
Application - StarEast 2008 – http://builtinrecordandplayback.xunitpatterns.com
– http://mobile.testAutomationPatterns.com
63 Copyright 2012 Gerard Meszaros InfoQ Hangzhou 2012
Thank You! Gerard Meszaros
[email protected] http://www.xunitpatterns.com
Slides:
http://infoQ2012hz.testautomationpatterns.com Call me when you: • Want to transition to Agile or Lean • Want to do Agile or Lean better • Want to teach developers how to test • Need help with test automation strategy • Want to improve your test automation
Jolt Productivity Award winner - Technical Books
Coming Soon: