Automated Test Results: tester vs manager

Prepared for IT Meet UP

Design about automation

Confidential 2

Selection criteria:

• Input parameters

• Cost

• Integration

• Clarity

• Support

• Metrics

• UI

Frameworks

Confidential 3

Frameworks

Confidential 4

Open Source

CI integrable

Supportable

Java+XML

Open Source

CI integrable

Supportable

Java+XML

Open Source

CI integrable

Supportable

Ruby/Java

Open Source

CI integrable

Supportable

Java+XML

• Live project visibility

• Live customer visibility

• Statistics/history

• Switch manual capacity to automation

• Best practice

Where is automation value?

Confidential 5

Selection Criteria:

Up-to-date

Integration

Transferring

Startup time

Dependencies

Debug mode

Universality

Others

Reports

Confidential 6

Up-to-date: 2013

Integration: via Jenkins HTMLReporter plugin

Transferring: easy

Startup time: close to 1h

Dependencies: extra download not from maven

Debug mode: local only

Universality: for any TestNG/JUnit tests

Others: none

ReportNG / HTMLReporter

Confidential 7

ReportNG / HTMLReporter

Confidential 8

ReportNG / HTMLReporter

Confidential 9

Up-to-date: 2015

Integration: via Allure plugins

Transferring: easy with appropriate test structure

Startup time: close to 1h, additional steps needed

Dependencies: extra dependencies (log4j, etc.)

Debug mode: none

Universality: yes – with multiple adapters

Others: issues with proxy

Allure

Confidential 10

Allure

Confidential 11

Up-to-date: 2015

Integration: via Jenkins plugin

Transferring: hard

Startup time: 2-3 days

Dependencies: thucydides dependencies only

Debug mode: local only

Universality: for Junit/JBehave tests

Others: BDD-oriented, all-in-one

Thucydides

Confidential 12

Thucydides

Confidential 13

Up-to-date: 2015

Integration: via Jenkins plugins

Transferring: easy with appropriate test structure

Startup time: more than 1 week

Dependencies: no extra dependencies

Debug mode: local only

Universality: yes – with multiple adapters

Others: have reporting issues

BDD: Cucumber/JBehave

Confidential 14

BDD: Cucumber/JBehave

Confidential 15

• Q&A

Summary

Confidential 16