Ruby meetup 7_years_in_testing
-
Upload
digital-natives -
Category
Technology
-
view
243 -
download
3
description
Transcript of Ruby meetup 7_years_in_testing
![Page 1: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/1.jpg)
7 Years In TestingTamás Tompa (Digital Natives)
Budapest Ruby Meetup 2013
11/09/2013
Thursday, September 12, 13
![Page 2: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/2.jpg)
§ 7 years in ruby, rails§ 7 years in webapp testing§ 5 years in agile development§ 5 years in automated testing
Thursday, September 12, 13
![Page 3: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/3.jpg)
Looking for testing best practices
Thursday, September 12, 13
![Page 4: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/4.jpg)
Short version
INTEGRATEAUTOMATE
Thursday, September 12, 13
![Page 5: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/5.jpg)
Short version
INTEGRATEAUTOMATE(everything)
Thursday, September 12, 13
![Page 6: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/6.jpg)
Human VS Machine§ Use humans for thinking§ Use machines for executing
§ Humans can forgive, forget things§ Machines are strict, consequent
Thursday, September 12, 13
![Page 7: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/7.jpg)
Theory
Thursday, September 12, 13
![Page 8: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/8.jpg)
Development Workflow
Resolved Deployed
Testing / Working
Testing / Not
working
In progress Stable
for feature based development
Thursday, September 12, 13
![Page 9: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/9.jpg)
Automating steps
Resolved Deployed
Testing / Working
Testing / Not
working
In progress Stable
Manual Automatic Manual Automatic
Thursday, September 12, 13
![Page 10: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/10.jpg)
Automated Testing§ rspec§ turnip§ cucumber§ capybara§ selenium -> phantomjs§ factory girl§ jenkins / cloud CI
TOOLS
Thursday, September 12, 13
![Page 11: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/11.jpg)
Working, bug free software is priceless...
Thursday, September 12, 13
![Page 12: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/12.jpg)
Working, bug free software is more expensive...
Thursday, September 12, 13
![Page 13: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/13.jpg)
... but only if you don’t know how to test
and maintain your code
Thursday, September 12, 13
![Page 14: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/14.jpg)
Write tests for those parts of the code which are...
§ the most often used§ changing frequently§ changing data and working with
financial data§ more likely to fail
Thursday, September 12, 13
![Page 15: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/15.jpg)
Instrumentation§ Instrumentation means that the code gets enhanced with
flags, and when the execution passes a certain flag, it gets set. When a test code coverage measurement is made then the tool:§ instruments the code - places these flags§ runs the test cases - flags get set§ and finally prints out the result - how many flags have
been set
§ During usage coverage measurement, test execution is replaced by customer interaction.
Thursday, September 12, 13
![Page 16: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/16.jpg)
Most often used code§ Instrument the code base and deliver
it to the testers / customer§ Turn on coverage measurement in your
(staging) application§ Deploy it§ Do manual testing of critical functions§ Summarize and analyze metrics§ Write tests for highly covered classes
Thursday, September 12, 13
![Page 17: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/17.jpg)
Frequently changing code§ Use version control system logs
§ Gitinspector§ StatSVN
§ Find out which files change often!
Thursday, September 12, 13
![Page 18: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/18.jpg)
Working with data§ Make code reviews§ Re-prioritize classes and methods
§ Add more priority when data is critical§ Move to top when working with money
Thursday, September 12, 13
![Page 19: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/19.jpg)
Failing, problematic code§ Find and analyze bugs in your Issue
Tracker System§ Write test before fixing the issue§ Review CI server logs
§ collect often failing tests§ Refactor unstable code
§ refactor your tests also
Thursday, September 12, 13
![Page 20: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/20.jpg)
Git branching§ Work only with feature branches§ Test the branches separately§ Use Pull Requests to integrate
feature branches to a development branch
§ Do more (manual) testing on development branch
§ Merge easily to stable, than finally to production
Thursday, September 12, 13
![Page 21: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/21.jpg)
Methods§ TDD - it’s up to you :)§ BDD - based on TDD + domain
specific acceptance criterias§ CDD - for prototypes, spikes
Thursday, September 12, 13
![Page 22: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/22.jpg)
In practice...
Thursday, September 12, 13
![Page 23: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/23.jpg)
Manual Testing§ Plan your test cases
§ at Sprint Planning Meetings§ Allocate a good Tester to the team
§ If you can’t, test all User Story after completion, within team, but between team members
§ Try to measure coverage during manual testing
Thursday, September 12, 13
![Page 24: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/24.jpg)
Continuous Integration§ Jenkins server§ Cloud alternatives: travis-ci, circleci, drone.io§ Parallel job execution (queues / executors)§ Main branches VS Feature branches§ Instant builds VS Daily builds§ Conflicting databases and shared resources§ CI rake task in the project§ Re-run logic for Cucumber tests§ Long running jobs
Thursday, September 12, 13
![Page 25: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/25.jpg)
CI - Ruby, Rails§ Jenkins / RVM Plugin
§ RVM-managed environment:ruby-1.9.3-p392@PROJECTNAME
§ Jenkins / Ruby Metrics Plugin§ Rcov coverage report§ Other stats
§ Bundler integration
Thursday, September 12, 13
![Page 26: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/26.jpg)
Thursday, September 12, 13
![Page 27: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/27.jpg)
CI - Feature branches§ Jenkins / Git Plugin
§ Branches to build: ^(?!.*master|stable|development|origin\/HEAD).*$
§ Jenkins / Version Number Plugin§ Version Number Format String:
#${BUILD_NUMBER}-${GIT_BRANCH}
Thursday, September 12, 13
![Page 28: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/28.jpg)
CI - Build types§ Instant builds
§ Higher priority§ Immediately after commit§ Build trigger: Poll SCM
§ Daily builds§ Once a day§ Long running tests§ Build trigger: Build periodically
Thursday, September 12, 13
![Page 29: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/29.jpg)
CI - Rake
task :commit => :clean do Rake::Task['db:migrate'].invoke Rake::Task['db:test:load'].invoke Rake::Task['db:test:purge'].invoke Rake::Task['ci:build:rspec_internal'].invoke Rake::Task['ci:build:cucumber_internal'].invokeend
rake ci:build:commit
> Example: https://gist.github.com/tompata/6513021
Thursday, September 12, 13
![Page 30: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/30.jpg)
CI - Rerun§ Re-run only previously failing tests§ Re-try the randomly failing tests§ Built-in support in Cucumber§ Easy integration with Jenkins§ Save resources and time
> Example: https://gist.github.com/tompata/6513060
Thursday, September 12, 13
![Page 31: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/31.jpg)
CI - Integration§ Bitbucket§ Redmine§ Notifiers (HipChat, Email, Desktop)§ Deployment (Capistrano)§ Monitoring (Airbrake, New Relic)
Thursday, September 12, 13
![Page 32: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/32.jpg)
Thursday, September 12, 13
![Page 33: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/33.jpg)
Thursday, September 12, 13
![Page 34: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/34.jpg)
Thursday, September 12, 13
![Page 35: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/35.jpg)
Thursday, September 12, 13
![Page 36: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/36.jpg)
Thursday, September 12, 13
![Page 37: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/37.jpg)
Browser tests§ integration tests
§ cucumber / capybara§ responsive design
§ phantomjs / screen sizes§ cross browser (design)
§ mtest (npm) - multiplatform§ saucelabs / cloud
§ 162 device/os/browser platforms
Thursday, September 12, 13
![Page 38: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/38.jpg)
Responsive design§ PhantomJS§ Screen resolutions (window_size)§ Cucumber profiles
> Example: https://gist.github.com/tompata/6512905
Thursday, September 12, 13
![Page 39: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/39.jpg)
Stress tests§ penetration / load testing before
releases§ use separated, dedicated
environments (staging test)§ use virtual instances (ec2)§ monitor instances (new relic)§ use online tools (loadimpact,
loader.io) or local (jmeter)
Thursday, September 12, 13
![Page 40: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/40.jpg)
Coverage, Quality§ rcov§ simple cov§ the magical 80%
§ code climate - code analysis, quality
Thursday, September 12, 13
![Page 41: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/41.jpg)
INTEGRATEAUTOMATE
Thursday, September 12, 13
![Page 42: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/42.jpg)
Resources§ http://zsoltfabok.com/blog/2011/05/narrow-down-what-to-test/
§ http://nvie.com/posts/a-successful-git-branching-model/
§ http://jenkins-ci.org/
§ https://wiki.jenkins-ci.org/display/JENKINS/Ruby+Metrics+Plugin
§ https://saucelabs.com/§ https://github.com/digitalnatives/
§ https://github.com/digitalnatives/mtest
§ https://gist.github.com/tompata/6513021
§ https://gist.github.com/tompata/6513060
§ https://gist.github.com/tompata/6512905
§ https://codeclimate.com/
§ http://loadimpact.com/
Thursday, September 12, 13
![Page 43: Ruby meetup 7_years_in_testing](https://reader031.fdocuments.us/reader031/viewer/2022020306/554bd717b4c905ac708b5022/html5/thumbnails/43.jpg)
We’re hiring!http://digitalnatives.hu/jobs
Thursday, September 12, 13