How to Use Social Media to Navigate the Startup Scene and Land a Lob
Acceptance testing in the land of the startup
-
Upload
joseph-wilk -
Category
Documents
-
view
488 -
download
0
Transcript of Acceptance testing in the land of the startup
![Page 1: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/1.jpg)
In the land of the startup
Joseph Wilk
Acceptance Testing
![Page 2: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/2.jpg)
What kind of land is this?
![Page 3: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/3.jpg)
What kind of land is this?
![Page 4: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/4.jpg)
Do startups need acceptance tests?
Measure
Learn
Build
![Page 5: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/5.jpg)
Context is king
![Page 6: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/6.jpg)
![Page 7: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/7.jpg)
![Page 8: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/8.jpg)
Toolset
Rails, MongoDB, Mysql, RabbitMQ, Memcache
Cucumber, Rspec
![Page 9: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/9.jpg)
Cucumber
• Behaviour Driven Development framework
• Plaintext
• Promotes communication
Given /^I have trained monkeys$/ do @test_pilots << Monkeys.newend
Feature: Filling Cucumbers with rocket fuel
Scenario: Rocket fuel Given I have trained monkeys When I click the launch button Then the monkeys should not die
![Page 10: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/10.jpg)
Why Acceptance?
• Developer centric tool
• Behaviour Driven Development
• Conversations
![Page 11: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/11.jpg)
Where acceptance tests are born
ProductOwner
Developer
QAUser
interaction
![Page 12: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/12.jpg)
Where acceptance tests are born
ProductOwner
Developer
QA
Just In TimeTalking
User interaction
![Page 13: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/13.jpg)
![Page 14: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/14.jpg)
![Page 15: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/15.jpg)
Non-technical people writing
Gherkin?
![Page 16: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/16.jpg)
![Page 17: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/17.jpg)
Feature: Ajax pagination for upcoming events In order to reduce friction finding concerts I want to go to As a logged in user I want an Ajax button which loads more concerts
Scenario: Events 60 days in the futures Given ... When ... Then ... Scenario: No events outside of the next 60 days Given ... When ... Then ... Scenario: No events Given ... When ... Then ...
![Page 18: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/18.jpg)
Are we done yet?
“A story isn't done until it is being used by real users in production and has been validated to be a useful part of a product.”
Joshua Kerievsky
![Page 19: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/19.jpg)
LearningsGood and bad
![Page 20: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/20.jpg)
Asynchronous Testing is messy
Scenario: Upload image ... Given all the messages have been processed ...
![Page 21: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/21.jpg)
Feature Overload
254 Feature files
1257 Scenarios/Examples
10807 Steps
![Page 22: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/22.jpg)
Slicing Features
Tagging
Filenames
$ cucumber --tags @media
Subsystems
$ cucumber visitor_*
$ cucumber features/admin/*
Profiles $ cucumber --profile admin
@media, @publicFeature: Visitor views artist’s media
@feed, @adminFeature: Logged in user views views their activity feed
![Page 23: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/23.jpg)
Acceptance testsUnit tests
vs
![Page 24: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/24.jpg)
The Test Build
254 Features, 1257 scenarios, 10807 steps
![Page 25: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/25.jpg)
The Test Build
254 Features, 1257 scenarios, 10807 steps
1 build server ~ 4 hours
![Page 26: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/26.jpg)
The Test Build
254 Features, 1257 scenarios, 10807 steps
1 build server ~ 4 hours
19 EC2 build servers ~ 13:40 minutes
![Page 27: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/27.jpg)
The Test Build
254 Features, 1257 scenarios, 10807 steps
$2068.99/$3000
1 build server ~ 4 hours
19 EC2 build servers ~ 13:40 minutes
![Page 28: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/28.jpg)
Divide and Concuquer
![Page 29: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/29.jpg)
Divide and Concuquer
![Page 30: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/30.jpg)
Divide and Concuquer
![Page 31: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/31.jpg)
Divide and Concuquer
![Page 32: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/32.jpg)
Divide and Concuquer
![Page 33: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/33.jpg)
Divide and Concuquer
Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
![Page 34: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/34.jpg)
Divide and conquer
Rails
Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
![Page 35: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/35.jpg)
Divide and conquer
RelatedArtistsService
Rails
Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
![Page 36: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/36.jpg)
Divide and conquer
related(artist)
RelatedArtistsService
Rails
Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
![Page 37: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/37.jpg)
Divide and conquer
related(artist)
RelatedArtistsService
HTMLRails
Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
![Page 38: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/38.jpg)
Divide and conquer
related(artist)
RelatedArtistsService
HTMLRails
Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
![Page 39: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/39.jpg)
Divide and conquer
related(artist)
RelatedArtistsService
HTMLRails
Scenario: Related artists Given an artist “SYGC” And “M.Bison” is related to “SYGC” When I visit the artist Then I should see within Similar Artists a link to “M.Bison”
unit testunit test
![Page 40: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/40.jpg)
![Page 41: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/41.jpg)
![Page 42: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/42.jpg)
![Page 43: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/43.jpg)
![Page 44: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/44.jpg)
Tests that never fail
![Page 45: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/45.jpg)
Tests that never fail
![Page 46: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/46.jpg)
Tests that never fail
Tests which regularly fail
![Page 47: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/47.jpg)
Daily Build
Tests that never fail
Tests which regularly fail
![Page 48: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/48.jpg)
Daily Build Nightly Build
Tests that never fail
Tests which regularly fail
![Page 49: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/49.jpg)
Daily Build Nightly Build
Tests that never fail
Tests which regularly fail
$cucumber --tags ~@nightly $cucumber --tags @nightly
![Page 50: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/50.jpg)
Flickering Confidence
![Page 51: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/51.jpg)
Flickering Confidence
FAIL
![Page 52: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/52.jpg)
Flickering Confidence
FAIL
FAIL
![Page 53: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/53.jpg)
Flickering Confidence
FAIL
PASS
FAIL
![Page 54: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/54.jpg)
Flickering Confidence
FAIL
PASS
FAIL
Main Flicker@flicker
![Page 55: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/55.jpg)
PairwiseScenario: Testing Cucumber against different environments Given I have a rails app <Rails version> And I’m using Ruby <Ruby version> And I am using the <Test Framework> And I am using the driver <Driver> Then Cucumber should install and play nicely Examples: | Rails version | Ruby version | test framework | Driver | | 3.0 | 1.8.7 | Rspec | Webrat | | 3.0 | 1.8.7 | Testunit | Capybara | | 3.0 | 1.8.7 | Rspec | Capybara | | 3.0 | 1.8.7 | Testunit | Webrat | | 3.0 | 1.9 | Rspec | Webrat | | 3.0 | 1.9 | Testunit | Webrat | | 2.3.1 | 1.9 | Rspec | Webrat | ....
![Page 56: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/56.jpg)
| Rails version | Ruby version | test framework | Driver | | 2.3.1 | 1.8.7 | Rspec | Webrat | | 2.3.1 | 1.8.7 | Rspec | Capybara | | 2.3.1 | 1.8.7 | Testunit | Webrat | | 2.3.1 | 1.8.7 | Testunit | Capybara | | 2.3.1 | 1.9 | Rspec | Capybara | | 2.3.1 | 1.9 | Rspec | Webrat | | 2.3.1 | 1.9 | Testunit | Capybara | | 2.3.1 | 1.9 | Testunit | Webrat |
....
“most faults are caused by interactions of at most two factors”
![Page 57: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/57.jpg)
Run Just Enough Tests
# * Test files must be stored in test/# * Test files names must start with test_# * Test class names must start with Test# * Implementation files must be stored in lib/# * Implementation files must match up with a # test file named# test_.*implementation.rb
@analyzer = Rcov::CodeCoverageAnalyzer.new
Cucover
Autotest
![Page 58: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/58.jpg)
![Page 59: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/59.jpg)
The Art of War Plaintext
, , @"===, ,_____cctI "?AAAAAAAAAAAAAAAA,,,,,,,,,,,,,,,,,,,,,;LLLLLLLLLL ~",,, 1""""""""""""###OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO '"EEEEE, !'"***"~~~~~~"OOOIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII ,EEEEE)>"'''???????"WWW!MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM "E.,)+="WWW~~~~~~#"OOO1OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO ,~:#")LLL!"+++???????"$$$1==========##/ &LLLLLLLLL;;;;;;;;;;;;,,,/ 1#LLLLLLLLLLLLLLLLLLLLLL! ,!###LLLLLL"'EEEE,'"LLLLL! !######LLL" "EEE" "LLLL" !#########L! "EEJ. "LL! !##########1 "JJ*,l" !############"! ,l" 1##########" 1"~~,~~" !##########" !###########! !###########1 !############! 1############# !"#############" !##############! 1##########"' A magnum, By calendron 1#####"' """""
![Page 60: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/60.jpg)
Don’t force structure
Feature: Title In order to <value> As a <role> I want <feature>
![Page 61: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/61.jpg)
Avoid Noise! Given I am able to loginGiven I am able to login
![Page 62: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/62.jpg)
Avoid Inconsistency
Given I loginGiven I authenticateGiven I gain access
![Page 63: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/63.jpg)
BalanceAbstraction
Given I go to the login pageAnd I fill in "username" with "cuke"And I fill in "password" with "cuker"And I click "login"
Given I'm logged in
![Page 64: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/64.jpg)
Use Language Building blocks
Given /I’m logged in/ do User.create!(:user => 'josephwilk', :password => "pass") Given 'I fill in "password" with "josephwilk"' Given 'I fill in "password" with "pass"' Given 'I click "login"'end
![Page 65: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/65.jpg)
Continuous Integration(WIP)Work in progress Done
list movies
add movies
edit movies
delete movies
Pending PassFail
$ cucumber --wip --tags @WIP features/
![Page 66: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/66.jpg)
Limiting Tags in FlowWork in progress Done
add movies
add movies
add movies
add movies
add movies
add movies
add movies
editmovies
add movies
add movies
add movies
deletemovies
$ cucumber --tags @WIP:3 features/
![Page 67: Acceptance testing in the land of the startup](https://reader035.fdocuments.us/reader035/viewer/2022070317/5562e77bd8b42ad26c8b4d78/html5/thumbnails/67.jpg)
Further reading
• http://cukes.info
• http://wiki.github.com/aslakhellesoy/cucumber
• http://www.cuke4ninja.com/
• http://blog.josephwilk.net