The Future Is Now: Writing Automated Tests

To Grow Your Code

Isaac Murchie, Ecosystem and Integrations Developer Sauce Labs

Why test?

http://www.history.navy.mil/photos/images/h96000/h96566kc.htm

Types of tests• Unit tests - testing individual software

components or modules. Tests module API, not internals.

• Integration tests - testing of integrated modules to verify combined functionality.

• Functional / GUI tests - testing of functionality to make sure it works as required (also System, End-to-End, etc.).

• Load, Stress, Performance, Usability, Acceptance tests, etc.

Unit tests• White-box tests of individual units. • Isolate each part of the program and show that the

individual parts are correct. • Should test functions isolated from their interaction with

other parts and from I/O. • Substitute interacting parts with method stubs, mock

objects, and fakes. • Separation of interface from implementation. • Benefits

• Find problems early. • Allow for safer refactoring. • Simplify integration. • Can drive the design of more modular, reusable

software.

def add(a, b): return a + b!def test_add(): assert add(4, 5) == 9

Simple assertions

Assertions on errorsdef divide_by_zero(a): a / 0!def test_zero_division(): with pytest.raises(ZeroDivisionError) as einfo: divide_by_zero(3)! assert 'exceptions.ZeroDivisionError' in str(einfo.type) assert 'by zero' in str(einfo.value)

import datetimefrom django.utils import timezone!from polls.models import Poll!!class TestPollMethod(object): def test_was_published_recently_with_future_poll(self): """ was_published_recently() should return False for polls whose pub_date is in the future """ future_poll = Poll(pub_date=timezone.now() + datetime.timedelta(days=30)) assert future_poll.was_published_recently() == False

Integration tests• Individual modules are combined and tested as a

group. • Verifies functional, performance, and reliability

requirements placed on larger units of the software. • Tests communication between modules. • May still involve mocking out certain services.

import datetimefrom django.core.urlresolvers import reversefrom django.utils import timezone!from polls.models import Poll!@pytest.mark.django_dbclass TestPollView(object): def test_index_view_with_a_past_poll(self, client): """ Polls with a pub_date in the past should be displayed on the index page. """ create_poll(question="Past poll.", days=-30) response = client.get(reverse('polls:index'))! assert 'Past poll.' == response.context['latest_poll_list'][0].question

Functional tests• Black-box tests of a portion of the software. • Tests what the program does, not how it does it. • Verifies a program against its specification and

requirements. • Slow. • Brittle.

import unittestfrom selenium import webdriver!class TestPollSelenium(unittest.TestCase): def setUp(self): self.driver = webdriver.Remote( desired_capabilities = { 'platform': 'MAC', 'browserName': 'chrome', 'version': '38' }, command_executor = 'http://localhost:4444/wd/hub' )! def tearDown(self): self.driver.quit()! def test_poll_index(self): self.driver.get('http://localhost:8000/polls/') assert 'Poll Index' in self.driver.title! el = self.driver.find_element_by_link_text('What\'s up?') assert el != None

Originally from Succeeding with Agile, Mike Cohn 2009.

Test early and test often.

Which is to say,

Regression testing"As a consequence of the introduction of new bugs, program

maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire bank of test cases previously run against the system,

to ensure that it has not been damaged in an obscure way. In practice, such regression testing must indeed approximate this

theoretical ideal, and it is very costly."

The Mythical Man-Month, Frederick P. Brooks, Jr., 1974, 122.

• Can be at any level of testing. • Test for regressions, re-emergence of old faults, as

well the emergence of new ones. • Most easily done through use of Continuous

Integration systems.

Continuous Integration• Merging working copies of code in short intervals. • Coupled with automated running of automated tests. • Use of dedicated build environment.

def add(a, b): return a + b!def test_add(): assert add(4, 5) == 9

Simple assertions

Assertions on errorsdef divide_by_zero(a): a / 0!def test_zero_division(): with pytest.raises(ZeroDivisionError) as einfo: divide_by_zero(3)! assert 'exceptions.ZeroDivisionError' in str(einfo.type) assert 'by zero' in str(einfo.value)

Testing Django appsimport datetimefrom django.utils import timezone!from polls.models import Poll!!class TestPollMethod(object): def test_was_published_recently_with_future_poll(self): """ was_published_recently() should return False for polls whose pub_date is in the future """ future_poll = Poll(pub_date=timezone.now() + datetime.timedelta(days=30)) assert future_poll.was_published_recently() == False

import datetimefrom django.core.urlresolvers import reversefrom django.utils import timezone!from polls.models import Poll!@pytest.mark.django_dbclass TestPollView(object): def test_index_view_with_a_past_poll(self, client): """ Polls with a pub_date in the past should be displayed on the index page. """ create_poll(question="Past poll.", days=-30) response = client.get(reverse('polls:index'))! assert 'Past poll.' == response.context['latest_poll_list'][0].question

import unittestfrom selenium import webdriver!class TestPollSelenium(unittest.TestCase): def setUp(self): self.driver = webdriver.Remote( desired_capabilities = { 'platform': 'MAC', 'browserName': 'chrome', 'version': '38' }, command_executor = 'http://localhost:4444/wd/hub' )! def tearDown(self): self.driver.quit()! def test_poll_index(self): self.driver.get('http://localhost:8000/polls/') assert 'Poll Index' in self.driver.title! el = self.driver.find_element_by_link_text('What\'s up?') assert el != None

isaac@saucelabs.com https://github.com/imurchie