Post on 29-Jun-2015
Introduction to Automated Testing
by Lars Thorup, ZeaLakeMay 26th 2011
Who is Lars Thorup?
• Software developer
• Coach: Teaching TDD and automated testing
• Advisor: Assesses software projects and companies
• Founder and CEO of BestBrains and ZeaLake
Why are we here today?
• How do we do automated testing?– Write test programs– Run the tests automatically
• Why should we write tests?– Enjoy more efficient and predictable course of development– Find and fix bugs fast– Prevent bugs from reappearing– Improve the design of our software
Different kinds of automated tests
• Unit tests– Tests individual pieces of code and the interaction between code blocks
• System tests– Tests the entire system against the requirements
• Performance tests– Tests non functional requirements
Unit tests or system tests?
• Unit tests are efficient– Fast to run (a few seconds)– Robust and predictable– Easy to write– Is written together with the code it is testing
• System tests are thorough– Tests all layers together– Most efficient way to create a set of tests for existing code
Can we automate performance tests?
• Performance tests are brittle– Tip: create performance trend curves instead
So, how do we actually do this?
• IsNumeric– C#– Ruby
• TransferFunds– C++
How do we run the tests automatically?
• From our programming environment (IDE)– Command line: make test– Right click | Run Tests
• On every commit– Setup a build server
• Hudson / Jenkins• TeamCity
– Let the build server run all tests– Get build notifications– Keep the build green
• Fixing a broken build has priority over any other development task
How can tests help improve our design?
• The software design needs to evolve over time• A refactoring modifies the design without changing behavior• Tests ensure that behavior is not accidentally changed
• Without tests, refactoring is scary– and with no refactoring, the design decays over time
• With tests, we have the courage to refactor– so we continually keep our design healthy
What is good design?
• One element of good design is loose dependencies– Use interfaces (for static languages)– Inject dependencies
• Avoid this
• Do this instead
Are we wasting valuable developer time writing tests?
• No• The time spent writing tests is not taken from the time spent
coding– It is taken from the time otherwise spent on manual testing and debugging
• The cost of a bug keeps increasing until we fix it
• Find bugs fast– Avoid losing customer confidence– Free QA to do exploratory testing
so they find the hard-to-find bugs– Spend less time trying to figure out
what is causing the bug and how to fix it
• Avoid spending time testing again
How do we get started?
• When we have a lot of existing code without tests– Create a set of system tests to get a safety net
• When we are writing new code– Write unit tests in conjunction with the new code
• Set up a standard test environment for our specific application– Test data
• Automate the creation of standard testdata in a local database– External dependencies
• Write stubs to use in the tests
What does a real-world project look like?
• wizerize.com– Web application: C# and JavaScript– 3 years of production– 2-4 developers
• 40% test code, 60% production code (in lines of code)• 71% code coverage of unit tests• 614 unit tests – run in 1 minute• 54 system tests – run in 20 minutes• No functional errors seen by end users in production (yet)
Where can I read more?
• http://googletesting.blogspot.com/• http://testdrivendeveloper.com/• http://codesheriff.blogspot.com/
Which tools do we use?
Environment ToolC# NUnitJavaScript qUnitHTML-based UI WatiN, SeleniumC++ CppUnit, googletestPython unittestRuby Test::UnitC check, cunitJava JUnit... ...