2013 DevFest Vienna - Bad Tests, Good Tests

88
Bad Tests, Good Tests Tomek Kaczanowski

description

my talk from 2013 DevFest Vienna about how to write really good tests. I hope you enjoy it!

Transcript of 2013 DevFest Vienna - Bad Tests, Good Tests

Page 1: 2013 DevFest Vienna - Bad Tests, Good Tests

Bad Tests, Good Tests

Tomek Kaczanowski

Page 2: 2013 DevFest Vienna - Bad Tests, Good Tests

http://twitter.com/#!/devops_borat

Page 3: 2013 DevFest Vienna - Bad Tests, Good Tests

Tomek Kaczanowski

• Developer• Team lead• Blogger

• http://kaczanowscy.pl/tomek

• Book author• http://practicalunittesting.com

• Trainer• Interested? Contact me!

[email protected]

• Trainings in Vienna? Sure!

Page 4: 2013 DevFest Vienna - Bad Tests, Good Tests

Before we begin

• All of the examples are real but were:• obfuscated

• to protect the innocents :)

• truncated

• imagine much more complex domain objects

• Asking questions is allowed• ...but being smarter than me is not ;)

Page 5: 2013 DevFest Vienna - Bad Tests, Good Tests

Who writes tests?

Page 6: 2013 DevFest Vienna - Bad Tests, Good Tests

What about the ROI?

Page 7: 2013 DevFest Vienna - Bad Tests, Good Tests

http://ripper1331.deviantart.com

Page 8: 2013 DevFest Vienna - Bad Tests, Good Tests

http://ripper1331.deviantart.com

Page 9: 2013 DevFest Vienna - Bad Tests, Good Tests

http://ripper1331.deviantart.com

http://twitter.com/#!/devops_borat

Survey is show junior devopsare still believein Tooth Fairy, Santa Claus anddocumentation

Page 10: 2013 DevFest Vienna - Bad Tests, Good Tests

http://ripper1331.deviantart.com

http://twitter.com/#!/devops_borat

Survey is show junior devopsare still believein Tooth Fairy, Santa Claus anddocumentation

If you think good designis expensive,you should look at the cost of bad design.

Dr Ralph Speth, CEO Jaguar

Page 11: 2013 DevFest Vienna - Bad Tests, Good Tests

Tests help to achieve quality

Not sure when I saw this picture – probably in GOOS?

Page 12: 2013 DevFest Vienna - Bad Tests, Good Tests

What happens if we do it wrong?

• Angry clients• Depressed developers

http://www.joshcanhelp.com

Page 13: 2013 DevFest Vienna - Bad Tests, Good Tests

When I started out with unit tests, I was enthralled with the promise of ease and security that they would bring to my projects. In practice, however, the theory of sustainable software through unit tests started to break down. This difficulty continued to build up, until I finally threw my head back in anger and declared that "Unit Tests have become more trouble than they are worth."

Llewellyn Falco and Michael Kennedy, Develop Mentor August 09

Page 14: 2013 DevFest Vienna - Bad Tests, Good Tests

http://chrispiascik.com/daily-drawings/express-yourself/

Page 15: 2013 DevFest Vienna - Bad Tests, Good Tests

write the right test

Page 16: 2013 DevFest Vienna - Bad Tests, Good Tests

write the right test

write this test right

Page 17: 2013 DevFest Vienna - Bad Tests, Good Tests

Ancient times

Page 18: 2013 DevFest Vienna - Bad Tests, Good Tests

public void testAddChunks() {System.out.println("*************************************");System.out.println("testAddChunks() ... ");ChunkMap cm = new ChunkMap(3);cm.addChunk(new Chunk("chunk"));

List testList = cm.getChunks("chunk",null);

if (testList.isEmpty())fail("there should be at least one list!");

Chunk chunk = cm.getActualChunk("chunk",null);if (chunk.getElements().isEmpty())

fail("there should be at least one element!");if (cm.getFinalChunkNr() != 1)

fail("there should be at least one chunk!");// iterate actual chunkfor (Iterator it = chunk.getElements().iterator();

it.hasNext();) {Element element = (Element) it.next();System.out.println("Element: " + element);

}showChunks(cm);System.out.println("testAddChunks() OK ");

} Courtesy of @bocytko

Page 19: 2013 DevFest Vienna - Bad Tests, Good Tests

public void testSimple() {IData data = null;IFormat format = null;LinkedList<String> attr = new LinkedList<String>();attr.add("A");attr.add("B");

try {format = new SimpleFormat("A");data.setAmount(Amount.TEN);data.setAttributes(attr);IResult result = format.execute();System.out.println(result.size());Iterator iter = result.iterator();while (iter.hasNext()) {

IResult r = (IResult) iter.next(); System.out.println(r.getMessage());

...}catch (Exception e) {

fail();}

}Courtesy of @bocytko

Page 20: 2013 DevFest Vienna - Bad Tests, Good Tests

What has happened? Well, it failed...public void testSimple() {

IData data = null;IFormat format = null;LinkedList<String> attr = new LinkedList<String>();attr.add("A");attr.add("B");

try {format = new SimpleFormat("A");data.setAmount(Amount.TEN);data.setAttributes(attr);IResult result = format.execute();System.out.println(result.size());Iterator iter = result.iterator();while (iter.hasNext()) {

IResult r = (IResult) iter.next(); System.out.println(r.getMessage());

...}

catch (Exception e) {fail();

}}

data is null - ready or not, NPE is coming!

Courtesy of @bocytko

Page 21: 2013 DevFest Vienna - Bad Tests, Good Tests

Success is not an option...

/** * Method testFailure. */public void testFailure() { try { Message message = new Message(null,true); fail(); } catch(Exception ex) { ExceptionHandler.log(ExceptionLevel.ANY,ex); fail(); }}

Courtesy of @bocytko

Page 22: 2013 DevFest Vienna - Bad Tests, Good Tests

No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {

void testSmoke() { def ds = new org.h2.jdbcx.JdbcDataSource( URL: 'jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle', user: 'sa', password: '') def jpaProperties = new Properties() jpaProperties.setProperty( 'hibernate.cache.use_second_level_cache', 'false') jpaProperties.setProperty( 'hibernate.cache.use_query_cache', 'false') def emf = new LocalContainerEntityManagerFactoryBean( dataSource: ds, persistenceUnitName: 'my-domain', jpaVendorAdapter: new HibernateJpaVendorAdapter( database: Database.H2, showSql: true, generateDdl: true), jpaProperties: jpaProperties)

…some more code below}

Page 23: 2013 DevFest Vienna - Bad Tests, Good Tests

No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {

void testSmoke() {// do not remove below code// def ds = new org.h2.jdbcx.JdbcDataSource(// URL: 'jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle',// user: 'sa', password: '')//// def jpaProperties = new Properties()// jpaProperties.setProperty(// 'hibernate.cache.use_second_level_cache', 'false')// jpaProperties.setProperty(// 'hibernate.cache.use_query_cache', 'false')//// def emf = new LocalContainerEntityManagerFactoryBean(// dataSource: ds, persistenceUnitName: 'my-domain',// jpaVendorAdapter: new HibernateJpaVendorAdapter(// database: Database.H2, showSql: true,// generateDdl: true), jpaProperties: jpaProperties)

…some more code below, all commented out :(}

Page 24: 2013 DevFest Vienna - Bad Tests, Good Tests

Let's follow the leader!

@Testpublic class ExampleTest {

public void testExample() {assertTrue(true);

}}

Page 25: 2013 DevFest Vienna - Bad Tests, Good Tests

Uh-oh, I feel lonely...

@Testpublic class ExampleTest {

public void testExample() {assertTrue(true);

}}

Page 26: 2013 DevFest Vienna - Bad Tests, Good Tests

Conclusions

• Automation!• Running• Manual verification is evil

• Do not live with broken window• If you don't fix things no one else will!• It is a full time job!

• You should be informed why your test failed• Master your tools

• …at least learn the basics!

Page 27: 2013 DevFest Vienna - Bad Tests, Good Tests

Mocks & Co.

Page 28: 2013 DevFest Vienna - Bad Tests, Good Tests

@Testpublic void shouldGetTrafficTrend() { //given TrafficTrendProvider trafficTrendProvider

= mock(TrafficTrendProvider.class); Report report = new Report(null, "", 1, 2, 3,

BigDecimal.ONE, BigDecimal.ONE, 1); TrafficTrend trafficTrend = new TrafficTrend(report, report,

new Date(), new Date(), new Date(), new Date()); given(trafficTrendProvider.getTrafficTrend())

.willReturn(trafficTrend); TrafficService service

= new TrafficService(trafficTrendProvider);

//when TrafficTrend result = service.getTrafficTrend();

//then assertThat(result).isEqualTo(trafficTrend);}

Page 29: 2013 DevFest Vienna - Bad Tests, Good Tests

Use of the real objects obscures the test

@Testpublic void shouldGetTrafficTrend() { //given TrafficTrendProvider trafficTrendProvider

= mock(TrafficTrendProvider.class); TrafficTrend trafficTrend = mock(TrafficTrend.class); given(trafficTrendProvider.getTrafficTrend())

.willReturn(trafficTrend);

TrafficService service = new TrafficService(trafficTrendProvider);

//when TrafficTrend result = service.getTrafficTrend();

//then assertThat(result).isEqualTo(trafficTrend);}

Page 30: 2013 DevFest Vienna - Bad Tests, Good Tests

Mock'em All!

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = mock(ModelAndView.class);

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

verify(modelAndView).addObject("timezone", "timezone X");

}

Page 31: 2013 DevFest Vienna - Bad Tests, Good Tests

Mock'em All!

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = mock(ModelAndView.class);

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

verify(modelAndView).addObject("timezone", "timezone X");

}

ModelAndView from SpringMVC – a mere container for data, without any behaviour

Page 32: 2013 DevFest Vienna - Bad Tests, Good Tests

Do not test interactions if not needed

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = new ModelAndView();

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

assertThat(modelAndView).contains("timezone", "timezone X");

}

a pseudocode but that is what we mean

Page 33: 2013 DevFest Vienna - Bad Tests, Good Tests

public class Util {

public String getUrl(User user, String timestamp) {String name = user.getFullName();String url = baseUrl

+"name="+URLEncoder.encode(name, "UTF-8")+"&timestamp="+timestamp;

return url; }

public String getUrl(User user) {Date date = new Date();Long time = date.getTime()/1000; //convert ms to secondsString timestamp = time.toString();return getUrl(user, timestamp);

}}

Developer wants to checkwhether timestamp is added to the URL when this method is used

Page 34: 2013 DevFest Vienna - Bad Tests, Good Tests

public class Util {

public String getUrl(User user, String timestamp) {...

}

public String getUrl(User user) {...

}}

Developer wants to checkwhether timestamp is added to the URL when this method is used

Page 35: 2013 DevFest Vienna - Bad Tests, Good Tests

public class Util {

public String getUrl(User user, String timestamp) {...

}

public String getUrl(User user) {...

}}

@Testpublic void shouldUseTimestampMethod() {

//givenUtil util = new Util();Util spyUtil = Mockito.spy(util);

//whenspyUtil.getUrl(user);

//thenverify(spyUtil).getUrl(eq(user), anyString());

}

Developer wants to checkwhether timestamp is added to the URL when this method is used

Bad design → bad tests

Page 36: 2013 DevFest Vienna - Bad Tests, Good Tests

Dependency Injection will save us

@Test

public void shouldGenerateURLWithTimestamp() {

//given

TimeProvider timeProvider = mock(TimeProvider.class);

Util util = new Util(timeProvider);

given(timeProvider.getTime()).willReturn("12345");

util.set(timeProvider);

//when

String url = util.getUrl(user);

//then

assertThat(url).contains("timestamp=12345");

}

Page 37: 2013 DevFest Vienna - Bad Tests, Good Tests

Single Responsibility Principle

A test should have one and only one reason to fail.

Page 38: 2013 DevFest Vienna - Bad Tests, Good Tests

Testing two things at once@DataProviderpublic Object[][] data() {

return new Object[][] { {"48", true}, {"+48", true}, {"++48", true}, {"+48503", true}, {"+4", false},{"++4", false}, {"", false},{null, false}, {" ", false}, };

}

@Test(dataProvider = "data")public void testQueryVerification(String query, boolean expected) {

assertEquals(expected, FieldVerifier.isValidQuery(query));}

Page 39: 2013 DevFest Vienna - Bad Tests, Good Tests

Testing two things at once@DataProviderpublic Object[][] data() {

return new Object[][] { {"48", true}, {"+48", true}, {"++48", true}, {"+48503", true}, {"+4", false},{"++4", false}, {"", false},{null, false}, {" ", false}, };

}

@Test(dataProvider = "data")public void testQueryVerification(String query, boolean expected) {

assertEquals(expected, FieldVerifier.isValidQuery(query));}

testQueryVerification1() {assertEquals(true, FieldVerifier.isValidQuery(„48”));

}testQueryVerification2() {

assertEquals(true, FieldVerifier.isValidQuery(„+48”));}testQueryVerification3() {

assertEquals(true, FieldVerifier.isValidQuery(„++48”));}testQueryVerification4() {

assertEquals(true, FieldVerifier.isValidQuery(„+48503”));}...

Page 40: 2013 DevFest Vienna - Bad Tests, Good Tests

Testing two things at once@DataProviderpublic Object[][] data() {

return new Object[][] { {"48", true}, {"+48", true}, {"++48", true}, {"+48503", true}, {"+4", false},{"++4", false}, {"", false},{null, false}, {" ", false}, };

}

@Test(dataProvider = "data")public void testQueryVerification(String query, boolean expected) {

assertEquals(expected, FieldVerifier.isValidQuery(query));}

Page 41: 2013 DevFest Vienna - Bad Tests, Good Tests

Concentrate on one feature@DataProviderpublic Object[][] validQueries() {

return new Object[][] { {"48"}, {"48123"}, {"+48"}, {"++48"}, {"+48503"}};

}

@Test(dataProvider = "validQueries")public void shouldRecognizeValidQueries(String validQuery) {

assertTrue(FieldVerifier.isValidQuery(validQuery));}

@DataProviderpublic Object[][] invalidQueries() { return new Object[][] {

{"+4"}, {"++4"}, {""}, {null}, {" "} };}

@Test(dataProvider = "invalidQueries")public void shouldRejectInvalidQueries(String invalidQuery) {

assertFalse(FieldVerifier.isValidQuery(invalidQuery));}

Page 42: 2013 DevFest Vienna - Bad Tests, Good Tests

“And”

@Test

public void shouldReturnRedirectViewAndSendEmail() {

//given

given(bindingResult.hasErrors()).willReturn(false);

given(userData.toEntity()).willReturn(user);

given(userService.saveNewUser(eq(userData.toEntity())))

.willReturn(user);

//when

ModelAndView userRegisterResult = userRegisterController

.registerUser(userData, bindingResult, request);

//then

assertThat(userRegisterResult.getViewName())

.isEqualTo("redirect:/signin");

verify(mailSender).sendRegistrationInfo(user);

}

Page 43: 2013 DevFest Vienna - Bad Tests, Good Tests

One feature at a time

@Test

public void shouldRedirectToSigninPageWhenRegistrationSuceeded () {

...

}

@Test

public void shouldNotifyAboutNewUserRegistration() {

...

}

Hint: forget about methods

Page 44: 2013 DevFest Vienna - Bad Tests, Good Tests

Readability is the king

variables

Page 45: 2013 DevFest Vienna - Bad Tests, Good Tests

@DataProvider

public static Object[][] usersPermissions() {

return new Object[][]{

{"user_1", Permission.READ},

{"user_1", Permission.WRITE},

{"user_1", Permission.REMOVE},

{"user_2", Permission.WRITE},

{"user_2", Permission.READ},

{"user_3", Permission.READ}

};

}

Page 46: 2013 DevFest Vienna - Bad Tests, Good Tests

Who the heck is “user_2” ?

@DataProvider

public static Object[][] usersPermissions() {

return new Object[][]{

{"user_1", Permission.READ},

{"user_1", Permission.WRITE},

{"user_1", Permission.REMOVE},

{"user_2", Permission.WRITE},

{"user_2", Permission.READ},

{"user_3", Permission.READ}

};

}

Who the heck is “user_2”?!

Page 47: 2013 DevFest Vienna - Bad Tests, Good Tests

Ah, logged user can read and write...

@DataProvider

public static Object[][] usersPermissions() {

return new Object[][]{

{ADMIN, Permission.READ},

{ADMIN, Permission.WRITE},

{ADMIN, Permission.REMOVE},

{LOGGED, Permission.WRITE},

{LOGGED, Permission.READ},

{GUEST, Permission.READ}

};

}

Page 48: 2013 DevFest Vienna - Bad Tests, Good Tests

domain1, domain2, domain3, ...

Page 49: 2013 DevFest Vienna - Bad Tests, Good Tests

domain1, domain2, domain3, ...

Page 50: 2013 DevFest Vienna - Bad Tests, Good Tests

domain1, domain2, domain3, ...

Page 51: 2013 DevFest Vienna - Bad Tests, Good Tests

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

Page 52: 2013 DevFest Vienna - Bad Tests, Good Tests

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

private static final boolean RESPONSE_IS_A_FILE = true;

private static final boolean NO_SSL = false;

server = new MockServer(responseMap, RESPONSE_IS_A_FILE,

new URL(SERVER_ROOT).getPort(), NO_SSL);

Page 53: 2013 DevFest Vienna - Bad Tests, Good Tests

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

server = createFileNonSSLMockServer(responseMap);

Page 54: 2013 DevFest Vienna - Bad Tests, Good Tests

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);

server = new MockServerBuilder()

.withResponse(responseMap)

.withResponseType(FILE)

.withUrl(SERVER_ROOT)

.withoutSsl().create();

server = MockServerBuilder

.createFileNoSSLServer(responseMap, SERVER_ROOT);

Page 55: 2013 DevFest Vienna - Bad Tests, Good Tests

Readability is the king

test method names

Page 56: 2013 DevFest Vienna - Bad Tests, Good Tests

Test methods names are important

• When test fails• Relation to focused tests

Page 57: 2013 DevFest Vienna - Bad Tests, Good Tests
Page 58: 2013 DevFest Vienna - Bad Tests, Good Tests

What is it all about?

@Test

public void testOperation() {

configureRequest("/validate")

rc = new RequestContext(parser, request)

assert rc.getConnector() == null

assert rc.getOperation().equals("validate")

}

Page 59: 2013 DevFest Vienna - Bad Tests, Good Tests

“should” is better than “test”

• shouldRejectInvalidRequests()• shouldSaveNewUserToDatabase()• constructorShouldFailWithNegativePrice()• shouldReturnOnlyUsersWithGivenName()

• testOperation()• testQuery()• testConstructor()• testFindUsersWithFilter()

Page 60: 2013 DevFest Vienna - Bad Tests, Good Tests

“should” is better than “test”

• Starting test method names with “should” steers you in the right direction.

• “test” prefix makes your test method a limitless bag where you throw everything worth testing

http://www.greenerideal.com/

http://jochopra.blogspot.com/

Page 61: 2013 DevFest Vienna - Bad Tests, Good Tests

Test methods names are important@Testpublic void testQuery(){ when(q.getResultList()).thenReturn(null); assertNull(dao.findByQuery(Transaction.class, q, false)); assertNull(dao.findByQuery(Operator.class, q, false)); assertNull(dao.findByQuery(null, null, false));

List result = new LinkedList(); when(q.getResultList()).thenReturn(result); assertEquals(dao.findByQuery(Transaction.class, q, false), result); assertEquals(dao.findByQuery(Operator.class, q, false), result); assertEquals(dao.findByQuery(null, null, false), null);

when(q.getSingleResult()).thenReturn(null); assertEquals(dao.findByQuery(Transaction.class, q, true).size(), 0); assertEquals(dao.findByQuery(Operator.class, q, true).size(), 0); assertEquals(dao.findByQuery(null, null, true), null);

when(q.getSingleResult()).thenReturn(t); assertSame(dao.findByQuery(Transaction.class, q, true).get(0), t); when(q.getSingleResult()).thenReturn(o); assertSame(dao.findByQuery(Operator.class, q, true).get(0), o); when(q.getSingleResult()).thenReturn(null); assertSame(dao.findByQuery(null, null, true), null);}

Page 62: 2013 DevFest Vienna - Bad Tests, Good Tests

Test methods names are important@Testpublic void shouldReturnNullListWhenDaoReturnsNull { when(q.getResultList()).thenReturn(null); assertNull(dao.findByQuery(Transaction.class, q, false)); assertNull(dao.findByQuery(Operator.class, q, false)); assertNull(dao.findByQuery(null, null, false));}public void shouldReturnEmptyListWhenDaoReturnsIt { List result = new LinkedList(); when(q.getResultList()).thenReturn(result); assertEquals(dao.findByQuery(Transaction.class, q, false), result); assertEquals(dao.findByQuery(Operator.class, q, false), result); assertEquals(dao.findByQuery(null, null, false), null);}public void shouldReturnNullSingleResultWhenDaoReturnsNull { when(q.getSingleResult()).thenReturn(null); assertEquals(dao.findByQuery(Transaction.class, q, true).size(), 0); assertEquals(dao.findByQuery(Operator.class, q, true).size(), 0); assertEquals(dao.findByQuery(null, null, true), null);}public void shouldReturnSingleResultReturnedByDao { when(q.getSingleResult()).thenReturn(t); assertSame(dao.findByQuery(Transaction.class, q, true).get(0), t); when(q.getSingleResult()).thenReturn(o); assertSame(dao.findByQuery(Operator.class, q, true).get(0), o); when(q.getSingleResult()).thenReturn(null); assertSame(dao.findByQuery(null, null, true), null);}

Page 63: 2013 DevFest Vienna - Bad Tests, Good Tests

Assertions

Page 64: 2013 DevFest Vienna - Bad Tests, Good Tests

public void shouldPreDeployApplication() {// givenArtifact artifact = mock(Artifact.class);when(artifact.getFileName()).thenReturn("war-artifact-2.0.war");ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH);Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);String destDir = new File(".").getCanonicalPath() + SLASH + "target" + SLASH;new File(destDir).mkdirs();

// whentomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

//thenJSch jsch = new JSch();jsch.addIdentity(KEY_FILE);Session session = jsch.getSession(USER, ADDRESS, 22);session.setConfig("StrictHostKeyChecking", "no");session.connect();

Channel channel = session.openChannel("sftp");session.setServerAliveInterval(92000);channel.connect();ChannelSftp sftpChannel = (ChannelSftp) channel;

sftpChannel.get(TEMP_PATH + SLASH + artifact.getFileName(), destDir);sftpChannel.exit();

session.disconnect();

File downloadedFile = new File(destDir, artifact.getFileName());

assertThat(downloadedFile).exists().hasSize(WAR_FILE_LENGTH);}

Page 65: 2013 DevFest Vienna - Bad Tests, Good Tests

Just say it

public void shouldPreDeployApplication() {// givenArtifact artifact = mock(Artifact.class);when(artifact.getFileName())

.thenReturn(ARTIFACT_FILE_NAME);ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH);

Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);

// whentomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

// thenSSHServerAssert.assertThat(ARTIFACT_FILE_NAME)

.existsOnServer(tomcat).hasSize(WAR_FILE_LENGTH);}

Page 66: 2013 DevFest Vienna - Bad Tests, Good Tests

Just say it

public void shouldPreDeployApplication() {// givenArtifact artifact = mock(Artifact.class);when(artifact.getFileName())

.thenReturn(ARTIFACT_FILE_NAME);ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH);

Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);

// whentomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

// thenassertThatFileIsOnServer(ARTIFACT_FILE_NAME,

Tomcat, WAR_FILE_LENGTH);}

WHY NOT USEWHY NOT CREATE

A PRIVATE ASSERTION METHOD?

Page 67: 2013 DevFest Vienna - Bad Tests, Good Tests

Assertions repeated in many tests

@Testpublic void testChargeInRetryingState() throws Exception { // given

TxDTO request = createTxDTO(RequestType.CHARGE);AndroidTransaction androidTransaction = ...

request.setTransaction(androidTransaction);

// when final TxDTO txDTO = processor.processRequest(request);

// thenList<AndroidTransactionStep> steps

= new ArrayList<>(androidTransaction.getSteps());AndroidTransactionStep lastStep = steps.get(steps.size() - 1);

assertEquals(lastStep.getTransactionState(), CHARGED_PENDING);assertEquals(txDTO.getResultCode(), CHARGED);

}

Page 68: 2013 DevFest Vienna - Bad Tests, Good Tests

Assertions repeated in many tests

@Testpublic void testChargeInRetryingState() throws Exception { // given

TxDTO request = createTxDTO(RequestType.CHARGE);AndroidTransaction androidTransaction = ...

request.setTransaction(androidTransaction);

// when final TxDTO txDTO = processor.processRequest(request);

// thenList<AndroidTransactionStep> steps

= new ArrayList<>(androidTransaction.getSteps());AndroidTransactionStep lastStep = steps.get(steps.size() - 1);

assertEquals(lastStep.getTransactionState(), CHARGED_PENDING);assertEquals(txDTO.getResultCode(), CHARGED);

}

WHY NOT CREATEA PRIVATE ASSERTION METHOD?

Page 69: 2013 DevFest Vienna - Bad Tests, Good Tests

Asserting using private methods

@Testpublic void testChargeInRetryingState() throws Exception { // given

TxDTO request = createTxDTO(RequestType.CHARGE);AndroidTransaction androidTransaction = ...

// when final TxDTO txDTO = processor.processRequest(request);

// then assertState(request, androidTransaction,

CHARGED, CHARGE_PENDING, AS_ANDROID_TX_STATE, ClientMessage.SUCCESS, ResultCode.SUCCESS);}

Page 70: 2013 DevFest Vienna - Bad Tests, Good Tests

assertState(TxDTO txDTO, AndroidTransaction androidTransaction,AndroidTransactionState expectedAndroidState,

AndroidTransactionState expectedPreviousAndroidState, ExtendedState expectedState, String expectedClientStatus, ResultCode expectedRequestResultCode) { final List<AndroidTransactionStep> steps

= new ArrayList<>(androidTransaction.getTransactionSteps()); final boolean checkPreviousStep = expectedAndroidState != null; assertTrue(steps.size() >= (checkPreviousStep ? 3 : 2));

if (checkPreviousStep) { AndroidTransactionStep lastStep = steps.get(steps.size() - 2); assertEquals(lastStep.getTransactionState(),

expectedPreviousAndroidState); }

final AndroidTransactionStep lastStep = steps.get(steps.size() - 1); assertEquals(lastStep.getTransactionState(), expectedAndroidState); assertEquals(lastStep.getMessage(), expectedClientStatus);

assertEquals(txDTO.getResultCode(), expectedRequestResultCode); assertEquals(androidTransaction.getState(), expectedAndroidState); assertEquals(androidTransaction.getExtendedState(), expectedState);

if (expectedClientStatus == null) { verifyZeroInteractions(client); }}

Page 71: 2013 DevFest Vienna - Bad Tests, Good Tests

Custom assertions to the rescue

@Test

public void testChargeInRetryingState() throws Exception {

// given

TxDTO request = createTxDTO(CHARGE);

AndroidTransaction androidTransaction = ...

// when

final TxDTO txDTO = processor.processRequest(request);

// then

assertThat(androidTransaction).hasState(CHARGED)

.hasMessage(ClientMessage.SUCCESS)

.hasPreviousState(CHARGE_PENDING)

.hasExtendedState(null);

assertEquals(txDTO.getResultCode(), ResultCode.SUCCESS);

}

Page 72: 2013 DevFest Vienna - Bad Tests, Good Tests

Asserting implementation details

public void invalidTxShouldBeCanceled() {

...

String fileContent =

FileUtils.getContentOfFile("response.csv");

assertTrue(fileContent.contains(

"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));

}

Page 73: 2013 DevFest Vienna - Bad Tests, Good Tests

Asserting implementation details

public void invalidTxShouldBeCanceled() {

...

String fileContent =

FileUtils.getContentOfFile("response.csv");

assertTrue(fileContent.contains(

"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));

}

public void invalidTxShouldBeCanceled() {

...

String fileContent =

FileUtils.getContentOfFile("response.csv");

TxDTOAssert.assertThat(fileContent)

.hasTransaction("123cancel").withResultCode(SUCCESS);

}

Page 74: 2013 DevFest Vienna - Bad Tests, Good Tests

Know your tools

• Unit testing framework• Use of temporary file rule

• Listeners

• Concurrency

• @Before/@After

• Parametrized tests

• Test dependencies

• Additional libraries

• Hamcrest, FEST, Mockito, catch-exception, awaitility, JUnitParams, tempus-fugit, …

• Build tool• Parallel execution

• CI

• IDE• Templates

• Shortcuts

Page 75: 2013 DevFest Vienna - Bad Tests, Good Tests

Expected exceptions

@Test(expected=IndexOutOfBoundsException.class)

public void shouldThrowExceptionGettingElementOutsideTheList() {

MyList<Integer> list = new MyList<Integer>();

list.add(0);

list.add(1);

list.get(2);

}

Page 76: 2013 DevFest Vienna - Bad Tests, Good Tests

Expected exceptions

@Test(expected=IndexOutOfBoundsException.class)

public void shouldThrowExceptionGettingElementOutsideTheList() {

MyList<Integer> list = new MyList<Integer>();

list.add(0);

list.add(1);

list.get(2);

}

@Test

public void shouldThrowExceptionGettingtElementOutsideTheList() {

MyList<Integer> list = new MyList<Integer>();

list.add(0);

list.add(1);

catchException(list).get(2);

assertThat(caughtException())

.isExactlyInstanceOf(IndexOutOfBoundsException.class);

}

http://code.google.com/p/catch-exception/

Page 77: 2013 DevFest Vienna - Bad Tests, Good Tests

Awaitility

@Testpublic void updatesCustomerStatus() throws Exception {

// Publish an asynchronous event: publishEvent(updateCustomerStatusEvent);

// Awaitility lets you wait until // the asynchronous operation completes:

await().atMost(5, SECONDS).until(costumerStatusIsUpdated());

...}

http://code.google.com/p/awaitility/

Page 78: 2013 DevFest Vienna - Bad Tests, Good Tests

What do you really want to test?

@Test

public void shouldAddAUser() {

User user = new User();

userService.save(user);

assertEquals(dao.getNbOfUsers(), 1);

}

Page 79: 2013 DevFest Vienna - Bad Tests, Good Tests

You wanted to see that the number increased

@Test

public void shouldAddAUser() {

int nb = dao.getNbOfUsers();

User user = new User();

userService.save(user);

assertEquals(dao.getNbOfUsers(), nb + 1);

}

Because:1) This is closer to what you wanted to test2) There is no assumption about the database “users” table being empty

Page 80: 2013 DevFest Vienna - Bad Tests, Good Tests

Asking for troubles...

LoggingPropertyConfigurator configurator = mock(...);

BaseServletContextListener baseServletContextListener =

= new BaseServletContextListener(configurator)

@Test public void shouldLoadConfigProperties() {

baseServletContextListener.contextInitialized();

verify(configurator).configure(any(Properties.class));

}

@Test(expected = LoggingInitialisationException.class)

public void shouldThrowExceptionIfCantLoadConfiguration() {

System.setProperty("logConfig", "nonExistingFile");

baseServletContextListener.contextInitialized();

}

Should load some default config

Should load this specific file

Page 81: 2013 DevFest Vienna - Bad Tests, Good Tests

Asking for troubles...

LoggingPropertyConfigurator configurator = mock(...);BaseServletContextListener baseServletContextListener =

= new BaseServletContextListener(configurator)

@Test public void shouldLoadConfigProperties() {baseServletContextListener.contextInitialized();verify(configurator).configure(any(Properties.class));

}

@Test(expected = LoggingInitialisationException.class)public void shouldThrowExceptionIfCantLoadConfiguration() {

System.setProperty("logConfig", "nonExistingFile");baseServletContextListener.contextInitialized();

}

@Beforepublic void cleanSystemProperties() {

...}

Page 82: 2013 DevFest Vienna - Bad Tests, Good Tests

Ceremony

@Testpublic void shouldBeAdministrator() { //given User user = new Administrator();

//when boolean administrator = user.isAdministrator(); boolean advertiser = user.isAdvertiser(); boolean domainer = user.isDomainer();

//then assertThat(administrator).isTrue(); assertThat(advertiser).isFalse(); assertThat(domainer).isFalse();}

Page 83: 2013 DevFest Vienna - Bad Tests, Good Tests

Ceremony

@Testpublic void shouldBeAdministrator() { User user = new Administrator();

assertThat(user.isAdministrator()).isTrue(); assertThat(user.isAdvertiser()).isFalse(); assertThat(user.isDomainer()).isFalse();}

Page 84: 2013 DevFest Vienna - Bad Tests, Good Tests

private static final int PER_PAGE = 10;

@Testpublic void shouldGiveOffsetZeroWhenOnZeroPage() { Pager pager = new Pager(PER_PAGE);

assertThat(pager.getOffset()).isEqualTo(0);}

@Testpublic void shouldIncreaseOffsetWhenGoingToPageOne() { Pager pager = new Pager(PER_PAGE);

pager.goToNextPage();

assertThat(pager.getOffset()).isEqualTo(PER_PAGE);}

Page 85: 2013 DevFest Vienna - Bad Tests, Good Tests

private static final int PER_PAGE = 10;

@Testpublic void shouldGiveOffsetZeroWhenOnZeroPage() { Pager pager = new Pager(PER_PAGE);

assertThat(pager.getOffset()).isEqualTo(0);}

@Testpublic void shouldIncreaseOffsetWhenGoingToPageOne() { Pager pager = new Pager(PER_PAGE);

pager.goToNextPage();

assertThat(pager.getOffset()).isEqualTo(PER_PAGE);}

public void goToNextPage() { this.offset = +perPage;}

Page 86: 2013 DevFest Vienna - Bad Tests, Good Tests

Test-last?

• makes people not write tests at all• makes people do only happy path testing• tests reflect the implementation

Page 87: 2013 DevFest Vienna - Bad Tests, Good Tests

Treat tests as the first class citizens

• do it everyday or forget about it

• use the right tool for the job

• and learn to use it!

• do not live with broken windows

• respect KISS, SRP, DRY (?)

• write good code, and you will also write good tests

• or rather write good tests and you will get good code for free

• code review your tests

• do more than happy path testing

• do not make the reader learn the API, make it obvious

• bad names lead to bad tests

• make tests readable using matchers, builders and good names

• test behaviour not methods

• be pragmatic about the tests you write

• TDD always?

• what is the best way to test it? unit/integration/end-to-end ?

• automate!

• always concentrate on what is worth testing

• ask yourself questions like: 'is it really important that X should send message Y to Z?'

• use the front door – state testing before interaction testing (mocks)

Page 88: 2013 DevFest Vienna - Bad Tests, Good Tests

You can learn more about writing high quality tests by reading my book – „Practical Unit Testing”.

You can also participate in writing of my new (free!) e-book devoted to bad and good tests.

Thank you!