Bad Tests, Good Tests

Tomek Kaczanowski

http://twitter.com/#!/devops_borat

Tomek Kaczanowski

• Developer

• Team lead

• Blogger

• http://kaczanowscy.pl/tomek

• Book author

• http://practicalunittesting.com

• Working at CodeWise (Krakow, Poland)

• ...we are hiring, wanna join us?

JUnit version coming soon!

Why bother with tests?

• System works as expected

• Changes do not hurt

• Documentation

Tests help to achieve quality

Not sure when I saw this picture – probably in GOOS?

What happens if we do it wrong?

• Angry clients

• Depressed developers

http://www.joshcanhelp.com

When I started out with unit tests, I was enthralled with the promise of ease and security that they would bring to my projects. In practice, however, the theory of sustainable software through unit tests started to break down. This difficulty continued to build up, until I finally threw my head back in anger and declared that "Unit Tests have become more trouble than they are worth."

Llewellyn Falco and Michael Kennedy, Develop Mentor August 09

http://chrispiascik.com/daily-drawings/express-yourself/

write the right test

write the right test

write this test right

Before we begin

• All of the examples are real but were:

• obfuscated

• to protect the innocents :)

• truncated

• imagine much more complex domain objects

• Asking questions is allowed

• ...but being smarter than me is not ;)

We don't need no stinkin' asserts! public void testAddChunks() {

System.out.println("*************************************");

System.out.println("testAddChunks() ... ");

ChunkMap cm = new ChunkMap(3);

cm.addChunk(new Chunk("chunk"));

List testList = cm.getChunks("chunk",null);

if (testList.isEmpty())

fail("there should be at least one list!");

Chunk chunk = cm.getActualChunk("chunk",null);

if (chunk.getElements().isEmpty())

fail("there should be at least one element!");

if (cm.getFinalChunkNr() != 1)

fail("there should be at least one chunk!");

// iterate actual chunk

for (Iterator it = chunk.getElements().iterator();

it.hasNext();) {

Element element = (Element) it.next();

System.out.println("Element: " + element);

}

showChunks(cm);

System.out.println("testAddChunks() OK ");

}

Courtesy of @bocytko

Success is not an option...

/**

* Method testFailure.

*/

public void testFailure() {

try {

Message message = new Message(null,true);

fail();

} catch(Exception ex) {

ExceptionHandler.log(ExceptionLevel.ANY,ex);

fail();

}

}

Courtesy of @bocytko

What has happened? Well, it failed...

public void testSimple() {

IData data = null;

IFormat format = null;

LinkedList<String> attr = new LinkedList<String>();

attr.add("A");

attr.add("B");

try {

format = new SimpleFormat("A");

data.setAmount(Amount.TEN);

data.setAttributes(attr);

IResult result = format.execute();

System.out.println(result.size());

Iterator iter = result.iterator();

while (iter.hasNext()) {

IResult r = (IResult) iter.next();

System.out.println(r.getMessage());

...

}

catch (Exception e) {

fail();

}

} Courtesy of @bocytko

What has happened? Well, it failed...

public void testSimple() {

IData data = null;

IFormat format = null;

LinkedList<String> attr = new LinkedList<String>();

attr.add("A");

attr.add("B");

try {

format = new SimpleFormat("A");

data.setAmount(Amount.TEN);

data.setAttributes(attr);

IResult result = format.execute();

System.out.println(result.size());

Iterator iter = result.iterator();

while (iter.hasNext()) {

IResult r = (IResult) iter.next();

System.out.println(r.getMessage());

...

}

catch (Exception e) {

fail();

}

}

data is null - ready or not, NPE is coming!

Courtesy of @bocytko

No smoke without tests class SystemAdminSmokeTest extends GroovyTestCase {

void testSmoke() {

def ds = new org.h2.jdbcx.JdbcDataSource(

URL: 'jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle',

user: 'sa', password: '')

def jpaProperties = new Properties()

jpaProperties.setProperty(

'hibernate.cache.use_second_level_cache', 'false')

jpaProperties.setProperty(

'hibernate.cache.use_query_cache', 'false')

def emf = new LocalContainerEntityManagerFactoryBean(

dataSource: ds, persistenceUnitName: 'my-domain',

jpaVendorAdapter: new HibernateJpaVendorAdapter(

database: Database.H2, showSql: true,

generateDdl: true), jpaProperties: jpaProperties)

…some more code below

}

No smoke without tests class SystemAdminSmokeTest extends GroovyTestCase {

void testSmoke() {

// do not remove below code

// def ds = new org.h2.jdbcx.JdbcDataSource(

// URL: 'jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle',

// user: 'sa', password: '')

//

// def jpaProperties = new Properties()

// jpaProperties.setProperty(

// 'hibernate.cache.use_second_level_cache', 'false')

// jpaProperties.setProperty(

// 'hibernate.cache.use_query_cache', 'false')

//

// def emf = new LocalContainerEntityManagerFactoryBean(

// dataSource: ds, persistenceUnitName: 'my-domain',

// jpaVendorAdapter: new HibernateJpaVendorAdapter(

// database: Database.H2, showSql: true,

// generateDdl: true), jpaProperties: jpaProperties)

…some more code below, all commented out :(

}

Let's follow the leader!

@Test

public class ExampleTest {

public void testExample() {

assertTrue(true);

}

}

Uh-oh, I feel lonely...

@Test

public class ExampleTest {

public void testExample() {

assertTrue(true);

}

}

Conclusions

• Automation!

• Running

• Verification

• Do not live with broken window

• And remember there is no one else to fix them but

you!

• It is a full time job!

• You should be informed why your test failed

• Master your tools

• …at least learn the basics!

Use of the real objects obscures the test

@Test

public void shouldGetTrafficTrend() {

//given

TrafficTrendProvider trafficTrendProvider

= mock(TrafficTrendProvider.class);

Report report = new Report(null, "", 1, 2, 3,

BigDecimal.ONE, BigDecimal.ONE, 1);

TrafficTrend trafficTrend = new TrafficTrend(report, report,

new Date(), new Date(), new Date(), new Date());

given(trafficTrendProvider.getTrafficTrend())

.willReturn(trafficTrend);

TrafficService service

= new TrafficService(trafficTrendProvider);

//when

TrafficTrend result = service.getTrafficTrend();

//then

assertThat(result).isEqualTo(trafficTrend);

}

Use of the real objects obscures the test

@Test

public void shouldGetTrafficTrend() {

//given

TrafficTrendProvider trafficTrendProvider

= mock(TrafficTrendProvider.class);

TrafficTrend trafficTrend = mock(TrafficTrend.class);

given(trafficTrendProvider.getTrafficTrend())

.willReturn(trafficTrend);

TrafficService service

= new TrafficService(trafficTrendProvider);

//when

TrafficTrend result = service.getTrafficTrend();

//then

assertThat(result).isEqualTo(trafficTrend);

}

Mock'em All!

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = mock(ModelAndView.class);

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

verify(modelAndView).addObject("timezone", "timezone X");

}

Mock'em All!

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = mock(ModelAndView.class);

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

verify(modelAndView).addObject("timezone", "timezone X");

}

ModelAndView from SpringMVC – a mere container for data, without any behaviour

Use the front door

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = new ModelAndView();

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

assertThat(modelAndView).contains("timezone", "timezone X");

}

a pseudocode but that is what we mean

Mock'em All!

Public class Util {

public String getUrl(User user, String timestamp) {

String name = user.getFullName();

String url = baseUrl

+"name="+URLEncoder.encode(name, "UTF-8")

+"&timestamp="+timestamp;

return url;

}

public String getUrl(User user) {

Date date = new Date();

Long time = date.getTime()/1000; //convert ms to seconds

String timestamp = time.toString();

return getUrl(user, timestamp);

}

}

Developer wants to check whether timestamp is added to the URL when this method is used

Mock'em All!

Public class Util {

public String getUrl(User user, String timestamp) {

...

}

public String getUrl(User user) {

...

}

}

@Test

public void shouldUseTimestampMethod() {

//given

Util util = new Util();

Util spyUtil = spy(util);

//when

spyUtil.getUrl(user);

//then

verify(spyUtil).getUrl(eq(user), anyString());

}

Use the front door

@Test

public void shouldAddTimestampToGeneratedUrl() {

//given

TimeProvider timeProvider = mock(TimeProvider.class);

Util util = new Util(timeProvider);

when(timeProvider.getTime()).thenReturn("12345");

util.set(timeProvider);

//when

String url = util.getUrl(user);

//then

assertThat(url).contains("timestamp=12345");

}

Dependency injection will save us

Single Responsibility Principle

A test should have one and only one reason to fail.

Testing two things at once

@DataProvider

public Object[][] data() {

return new Object[][] { {"48", true}, {"+48", true},

{"++48", true}, {"+48503", true}, {"+4", false},

{"++4", false}, {"", false},

{null, false}, {" ", false}, };

}

@Test(dataProvider = "data")

public void testQueryVerification(String query, boolean expected) {

assertEquals(expected, FieldVerifier.isValidQuery(query));

}

Testing two things at once

@DataProvider

public Object[][] data() {

return new Object[][] { {"48", true}, {"+48", true},

{"++48", true}, {"+48503", true}, {"+4", false},

{"++4", false}, {"", false},

{null, false}, {" ", false}, };

}

@Test(dataProvider = "data")

public void testQueryVerification(String query, boolean expected) {

assertEquals(expected, FieldVerifier.isValidQuery(query));

}

testQueryVerification1() {

assertEquals(true, FieldVerifier.isValidQuery(„48”));

}

testQueryVerification2() {

assertEquals(true, FieldVerifier.isValidQuery(„+48”));

}

testQueryVerification3() {

assertEquals(true, FieldVerifier.isValidQuery(„++48”));

}

testQueryVerification4() {

assertEquals(true, FieldVerifier.isValidQuery(„+48503”));

}

...

Concentrate on one feature

@DataProvider

public Object[][] validQueries() {

return new Object[][] { {"48"}, {"48123"},

{"+48"}, {"++48"}, {"+48503"}};

}

@Test(dataProvider = "validQueries")

public void shouldRecognizeValidQueries(String validQuery) {

assertTrue(FieldVerifier.isValidQuery(validQuery));

}

@DataProvider

public Object[][] invalidQueries() {

return new Object[][] {

{"+4"}, {"++4"}, {""}, {null}, {" "} };

}

@Test(dataProvider = "invalidQueries")

public void shouldRejectInvalidQueries(String invalidQuery) {

assertFalse(FieldVerifier.isValidQuery(invalidQuery));

}

“And”

@Test

public void shouldReturnRedirectViewAndSendEmail() {

//given

given(bindingResult.hasErrors()).willReturn(false);

given(userData.toEntity()).willReturn(user);

given(userService.saveNewUser(eq(userData.toEntity())))

.willReturn(user);

//when

ModelAndView userRegisterResult = userRegisterController

.registerUser(userData, bindingResult, request);

//then

assertThat(userRegisterResult.getViewName())

.isEqualTo("redirect:/signin");

verify(mailSender).sendRegistrationInfo(user);

}

One feature at a time

@Test

public void shouldRedirectToSigninPageWhenRegistrationSuceeded () {

...

}

@Test

public void shouldNotifyAboutNewUserRegistration() {

...

}

Hint: forget about methods

Readability is the king

Who the heck is “user_2” ?

@DataProvider

public static Object[][] usersPermissions() {

return new Object[][]{

{"user_1", Permission.READ},

{"user_1", Permission.WRITE},

{"user_1", Permission.REMOVE},

{"user_2", Permission.WRITE},

{"user_2", Permission.READ},

{"user_3", Permission.READ}

};

}

Ah, logged user can read and write...

@DataProvider

public static Object[][] usersPermissions() {

return new Object[][]{

{ADMIN, Permission.READ},

{ADMIN, Permission.WRITE},

{ADMIN, Permission.REMOVE},

{LOGGED, Permission.WRITE},

{LOGGED, Permission.READ},

{GUEST, Permission.READ}

};

}

domain1, domain2, domain3, ...

domain1, domain2, domain3, ...

domain1, domain2, domain3, ...

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

private static final boolean RESPONSE_IS_A_FILE = true;

private static final boolean NO_SSL = false;

server = new MockServer(responseMap, RESPONSE_IS_A_FILE,

new URL(SERVER_ROOT).getPort(), NO_SSL);

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

server = createFileNonSSLMockServer(responseMap);

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

server = new MockServerBuilder()

.withResponse(responseMap)

.withResponseType(FILE)

.withUrl(SERVER_ROOT)

.withoutSsl().create();

server = MockServerBuilder

.createFileNoSSLServer(responseMap, SERVER_ROOT);

Naming is really important

Test methods names are important

• When test fails

• Relation to focused tests

Test methods names are important

@Test

public void testOperation() {

configureRequest("/validate")

rc = new RequestContext(parser, request)

assert rc.getConnector() == null

assert rc.getOperation().equals("validate")

}

“should” is better than “test”

• shouldRejectInvalidRequests()

• shouldSaveNewUserToDatabase()

• constructorShouldFailWithNegativePrice()

• shouldReturnOnlyUsersWithGivenName()

• testOperation()

• testQuery()

• testConstructor()

• testFindUsersWithFilter()

“should” is better than “test”

• Starting test method names

with “should” steers you in

the right direction.

• “test” prefix makes your test

method a limitless bag

where you throw everything

worth testing

http://www.greenerideal.com/

http://jochopra.blogspot.com/

Test methods names are important

@Test public void testQuery(){ when(q.getResultList()).thenReturn(null); assertNull(dao.findByQuery(Transaction.class, q, false)); assertNull(dao.findByQuery(Operator.class, q, false)); assertNull(dao.findByQuery(null, null, false)); List result = new LinkedList(); when(q.getResultList()).thenReturn(result); assertEquals(dao.findByQuery(Transaction.class, q, false), result); assertEquals(dao.findByQuery(Operator.class, q, false), result); assertEquals(dao.findByQuery(null, null, false), null); when(q.getSingleResult()).thenReturn(null); assertEquals(dao.findByQuery(Transaction.class, q, true).size(), 0); assertEquals(dao.findByQuery(Operator.class, q, true).size(), 0); assertEquals(dao.findByQuery(null, null, true), null); when(q.getSingleResult()).thenReturn(t); assertSame(dao.findByQuery(Transaction.class, q, true).get(0), t); when(q.getSingleResult()).thenReturn(o); assertSame(dao.findByQuery(Operator.class, q, true).get(0), o); when(q.getSingleResult()).thenReturn(null); assertSame(dao.findByQuery(null, null, true), null); }

Test methods names are important

@Test public void shouldReturnNullListWhenDaoReturnsNull { when(q.getResultList()).thenReturn(null); assertNull(dao.findByQuery(Transaction.class, q, false)); assertNull(dao.findByQuery(Operator.class, q, false)); assertNull(dao.findByQuery(null, null, false)); } public void shouldReturnEmptyListWhenDaoReturnsIt { List result = new LinkedList(); when(q.getResultList()).thenReturn(result); assertEquals(dao.findByQuery(Transaction.class, q, false), result); assertEquals(dao.findByQuery(Operator.class, q, false), result); assertEquals(dao.findByQuery(null, null, false), null); } public void shouldReturnNullSingleResultWhenDaoReturnsNull { when(q.getSingleResult()).thenReturn(null); assertEquals(dao.findByQuery(Transaction.class, q, true).size(), 0); assertEquals(dao.findByQuery(Operator.class, q, true).size(), 0); assertEquals(dao.findByQuery(null, null, true), null); } public void shouldReturnSingleResultReturnedByDao { when(q.getSingleResult()).thenReturn(t); assertSame(dao.findByQuery(Transaction.class, q, true).get(0), t); when(q.getSingleResult()).thenReturn(o); assertSame(dao.findByQuery(Operator.class, q, true).get(0), o); when(q.getSingleResult()).thenReturn(null); assertSame(dao.findByQuery(null, null, true), null); }

Assertion part is freaking huge public void shouldPreDeployApplication() {

// given

Artifact artifact = mock(Artifact.class);

when(artifact.getFileName()).thenReturn("war-artifact-2.0.war");

ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH);

Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);

String destDir = new File(".").getCanonicalPath() + SLASH + "target" + SLASH;

new File(destDir).mkdirs();

// when

tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

//then

JSch jsch = new JSch();

jsch.addIdentity(KEY_FILE);

Session session = jsch.getSession(USER, ADDRESS, 22);

session.setConfig("StrictHostKeyChecking", "no");

session.connect();

Channel channel = session.openChannel("sftp");

session.setServerAliveInterval(92000);

channel.connect();

ChannelSftp sftpChannel = (ChannelSftp) channel;

sftpChannel.get(TEMP_PATH + SLASH + artifact.getFileName(), destDir);

sftpChannel.exit();

session.disconnect();

File downloadedFile = new File(destDir, artifact.getFileName());

assertThat(downloadedFile).exists().hasSize(WAR_FILE_LENGTH);

}

Just say it

public void shouldPreDeployApplication() {

// given

Artifact artifact = mock(Artifact.class);

when(artifact.getFileName())

.thenReturn(ARTIFACT_FILE_NAME);

ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER,

KEY_FILE, TOMCAT_PATH, TEMP_PATH);

Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);

// when

tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

// then

SSHServerAssert.assertThat(ARTIFACT_FILE_NAME)

.existsOnServer(tomcat).hasSize(WAR_FILE_LENGTH);

}

Just say it

public void shouldPreDeployApplication() {

// given

Artifact artifact = mock(Artifact.class);

when(artifact.getFileName())

.thenReturn(ARTIFACT_FILE_NAME);

ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER,

KEY_FILE, TOMCAT_PATH, TEMP_PATH);

Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);

// when

tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

// then

assertThatFileIsOnServer(ARTIFACT_FILE_NAME,

Tomcat, WAR_FILE_LENGTH);

}

WHY NOT USE WHY NOT CREATE

A PRIVATE ASSERTION METHOD?

Asserting using private methods

@Test

public void testChargeInRetryingState() throws Exception {

// given

TxDTO request = createTxDTO(RequestType.CHARGE);

AndroidTransaction androidTransaction = ...

request.setTransaction(androidTransaction);

// when

final TxDTO txDTO = processor.processRequest(request);

// then

List<AndroidTransactionStep> steps

= new ArrayList<>(androidTransaction.getSteps());

AndroidTransactionStep lastStep = steps.get(steps.size() - 1);

assertEquals(lastStep.getTransactionState(), CHARGED_PENDING);

assertEquals(txDTO.getResultCode(), CHARGED);

}

Asserting using private methods

@Test

public void testChargeInRetryingState() throws Exception {

// given

TxDTO request = createTxDTO(RequestType.CHARGE);

AndroidTransaction androidTransaction = ...

request.setTransaction(androidTransaction);

// when

final TxDTO txDTO = processor.processRequest(request);

// then

List<AndroidTransactionStep> steps

= new ArrayList<>(androidTransaction.getSteps());

AndroidTransactionStep lastStep = steps.get(steps.size() - 1);

assertEquals(lastStep.getTransactionState(), CHARGED_PENDING);

assertEquals(txDTO.getResultCode(), CHARGED);

}

WHY NOT CREATE A PRIVATE ASSERTION METHOD?

Asserting using private methods

@Test

public void testChargeInRetryingState() throws Exception {

// given

TxDTO request = createTxDTO(RequestType.CHARGE);

AndroidTransaction androidTransaction = ...

// when

final TxDTO txDTO = processor.processRequest(request);

// then

assertState(request, androidTransaction,

CHARGED, CHARGE_PENDING, AS_ANDROID_TX_STATE,

ClientMessage.SUCCESS, ResultCode.SUCCESS);

}

Matchers vs. private methods assertState(TxDTO txDTO, AndroidTransaction androidTransaction,

AndroidTransactionState expectedAndroidState,

AndroidTransactionState expectedPreviousAndroidState,

ExtendedState expectedState,

String expectedClientStatus, ResultCode expectedRequestResultCode) {

final List<AndroidTransactionStep> steps

= new ArrayList<>(androidTransaction.getTransactionSteps());

final boolean checkPreviousStep = expectedAndroidState != null;

assertTrue(steps.size() >= (checkPreviousStep ? 3 : 2));

if (checkPreviousStep) {

AndroidTransactionStep lastStep = steps.get(steps.size() - 2);

assertEquals(lastStep.getTransactionState(),

expectedPreviousAndroidState);

}

final AndroidTransactionStep lastStep = steps.get(steps.size() - 1);

assertEquals(lastStep.getTransactionState(), expectedAndroidState);

assertEquals(lastStep.getMessage(), expectedClientStatus);

assertEquals(txDTO.getResultCode(), expectedRequestResultCode);

assertEquals(androidTransaction.getState(), expectedAndroidState);

assertEquals(androidTransaction.getExtendedState(), expectedState);

if (expectedClientStatus == null) {

verifyZeroInteractions(client);

}

}

Matchers vs. private methods

@Test

public void testChargeInRetryingState() throws Exception {

// given

TxDTO request = createTxDTO(CHARGE);

AndroidTransaction androidTransaction = ...

// when

final TxDTO txDTO = processor.processRequest(request);

// then

assertThat(androidTransaction).hasState(CHARGED)

.hasMessage(ClientMessage.SUCCESS)

.hasPreviousState(CHARGE_PENDING)

.hasExtendedState(null);

assertEquals(txDTO.getResultCode(), ResultCode.SUCCESS);

}

Asserting implementation details

public void invalidTxShouldBeCanceled() {

...

String fileContent =

FileUtils.getContentOfFile("response.csv");

assertTrue(fileContent.contains(

"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));

}

Asserting implementation details

public void invalidTxShouldBeCanceled() {

...

String fileContent =

FileUtils.getContentOfFile("response.csv");

assertTrue(fileContent.contains(

"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));

}

public void invalidTxShouldBeCanceled() {

...

String fileContent =

FileUtils.getContentOfFile("response.csv");

TxDTOAssert.assertThat(fileContent)

.hasTransaction("123cancel").withResultCode(SUCCESS);

}

Know your tools

• Unit testing framework

• Use of temporary file rule

• Listeners

• Concurrency

• @Before/@After

• Parametrized tests

• Test dependencies

• Additional libraries

• Hamcrest, FEST, Mockito,

catch-exception, awaitility,

JunitParams, tempus-fugit, …

• Build tool

• Parallel execution

• CI

• IDE

• Templates

• Shortcuts

Expected exceptions

@Test(expected=IndexOutOfBoundsException.class)

public void shouldThrowExceptionGettingElementOutsideTheList() {

MyList<Integer> list = new MyList<Integer>();

list.add(0);

list.add(1);

list.get(2);

}

Expected exceptions

@Test(expected=IndexOutOfBoundsException.class)

public void shouldThrowExceptionGettingElementOutsideTheList() {

MyList<Integer> list = new MyList<Integer>();

list.add(0);

list.add(1);

list.get(2);

}

@Test

public void shouldThrowExceptionGettingtElementOutsideTheList() {

MyList<Integer> list = new MyList<Integer>();

list.add(0);

list.add(1);

catchException(list).get(2);

assertThat(caughtException())

.isExactlyInstanceOf(IndexOutOfBoundsException.class);

}

http://code.google.com/p/catch-exception/

Expected exceptions (with catch-exception)

@Test

public void shouldThrowException() throws SmsException {

catchException(gutExtractor)

.extractGut(„invalid gut”);

then(caughtException())

.isInstanceOf(SmsException.class)

.hasMessage("Invalid gut")

.hasNoCause();

}

http://code.google.com/p/catch-exception/

Awaitility

@Test

public void updatesCustomerStatus() throws Exception {

// Publish an asynchronous event:

publishEvent(updateCustomerStatusEvent);

// Awaitility lets you wait until

// the asynchronous operation completes:

await().atMost(5, SECONDS)

.until(costumerStatusIsUpdated());

...

}

http://code.google.com/p/awaitility/

What do you really want to test?

@Test

public void shouldAddAUser() {

User user = new User();

userService.save(user);

assertEquals(dao.getNbOfUsers(), 1);

}

You wanted to see that the number increased

@Test

public void shouldAddAUser() {

int nb = dao.getNbOfUsers();

User user = new User();

userService.save(user);

assertEquals(dao.getNbOfUsers(), nb + 1);

}

Because: 1) This is closer to what you wanted to test 2) There is no assumption about the database “users” table being empty

The dream of stronger, random-powered tests

public void myTest() {

SomeObject obj = new SomeObject(

randomName(), randomValue(), ....);

// testing of obj here

}

Does it make your test stronger?

The dream of stronger, random-powered tests

public void myTest() {

SomeObject obj = new SomeObject(

randomName(), randomValue(), ....);

// testing of obj here

}

Does it make your test stronger?

...or does it only bring confusion?

Test failed

Expected

SomeObject(„a”, „b”, ....)

but got

SomeObject(„*&O*$NdlF”, „#idSLNF”, ....)

Random done wrong

public void myTest() {

SomeObject obj = new SomeObject(

a, b, c, productCode());

// testing of obj here

}

private String productCode(){

String[] codes = {"Code A", "Code B",

"Code C", "Code D"};

int index = rand.nextInt(codes.length);

return codes[index];

}

Ceremony

@Test

public void shouldBeAdministrator() {

//given

User user = new Administrator();

//when

boolean administrator = user.isAdministrator();

boolean advertiser = user.isAdvertiser();

boolean domainer = user.isDomainer();

//then

assertThat(administrator).isTrue();

assertThat(advertiser).isFalse();

assertThat(domainer).isFalse();

}

Ceremony

@Test

public void shouldBeAdministrator() {

User user = new Administrator();

assertThat(user.isAdministrator()).isTrue();

assertThat(user.isAdvertiser()).isFalse();

assertThat(user.isDomainer()).isFalse();

}

Asking for troubles...

LoggingPropertyConfigurator configurator = mock(...);

BaseServletContextListener baseServletContextListener =

= new BaseServletContextListener(configurator)

@Test public void shouldLoadConfigProperties() {

baseServletContextListener.contextInitialized();

verify(configurator).configure(any(Properties.class));

}

@Test(expected = LoggingInitialisationException.class)

public void shouldThrowExceptionIfCantLoadConfiguration() {

System.setProperty("logConfig", "nonExistingFile");

baseServletContextListener.contextInitialized();

}

Should load some default config

Should load this specific file

Asking for troubles...

LoggingPropertyConfigurator configurator = mock(...);

BaseServletContextListener baseServletContextListener =

= new BaseServletContextListener(configurator)

@Test public void shouldLoadConfigProperties() {

baseServletContextListener.contextInitialized();

verify(configurator).configure(any(Properties.class));

}

@Test(expected = LoggingInitialisationException.class)

public void shouldThrowExceptionIfCantLoadConfiguration() {

System.setProperty("logConfig", "nonExistingFile");

baseServletContextListener.contextInitialized();

}

@Before

public void cleanSystemProperties() {

...

}

Test-last? No!

• makes people not write tests at all

• makes people do only happy path testing

• tests reflect the implementation

For six or eight hours spread over the next few weeks I struggled to get the first test written and running. Writing tests for Eclipse plug-ins is not trivial, so it’s not surprising I had some trouble. [...] In six or eight hours of solid programming time, I can still make significant progress. If I’d just written some stuff and verified it by hand, I would probably have the final answer to whether my idea is actually worth money by now. Instead, all I have is a complicated test that doesn’t work, a pile of frustration, eight fewer hours in my life, and the motivation to write another essay.

Kent Beck, Just Ship, Baby

Always TDD?

There is so much more to discuss…

• Integration / end-to-end tests which are not parametrized (so they all try to set up jetty on port 8080),

• Tests which should be really unit, but use Spring context to create objects,

• Tests with a lot of dependencies between them (a nightmare to maintain!),

• Tests which are overspecified and will fail whenever you touch the production code,

• Tests with monstrous objects-creation code,

• Tests which run slow,

• Tests which try to cover the deficiencies of production code and end up being a total mess,

• Tests which verify methods instead of verifying responsibilities of a class,

• Happy path tests,

• etc., etc.

Treat tests as the first class citizens

• do it everyday or forget about it

• use the right tool for the job

• and learn to use it!

• do not live with broken windows

• respect KISS, SRP, DRY (?)

• write good code, and you will also write

good tests

• or rather write good tests and you

will get good code for free

• code review your tests

• do more than happy path testing

• do not make the reader learn the API,

make it obvious

• bad names lead to bad tests

• make tests readable using matchers,

builders and good names

• test behaviour not methods

• be pragmatic about the tests you write

• TDD always?

• what is the best way to test it?

unit/integration/end-to-end ?

• automate!

• always concentrate on what is worth

testing

• ask yourself questions like: 'is it

really important that X should send

message Y to Z?'

• use the front door – state testing before

interaction testing (mocks)

You can learn more about writing

high quality tests by reading my

book – „Practical Unit Testing”.

You can also participate in

writing of my new (free!) e-

book devoted to bad and good

tests.

Thank you!