Testing fundamentals in a changing world

Derk-Jan de GroodSIGIST London September 2016

1

ShiftsReleasing x-times a day reduces the need for formal acceptance

Shift to Automation and Bug hunting

Shift to Business tests

Towards value and systems thinking

Shift to Non-functional tests

From Project to Operations

Methodic testing is declining

From: BUFDTo: JIT

Aim of this session

5

What can we learn

from our projects

about the

Fundamentals of

testing ?

How do we

position our

profession and

work

Understand how our profession is evolvingInsight in how we

need to develop

ourselves

6

Beyond the T-shape (Tutorial/webinar with Jan Jaap Cannegieter)

Beyond the T-shape (Tutorial/webinar with Jan Jaap Cannegieter)

8

9

Our End Goal

10

CI/CD Assumptions

11

Teams Collaborate

Integration is Continue

Tests are Automated

Deployment is hands-off process

No Automation Backlog

Clear Acceptance Criteria

Feedback loop to improve Testing

Frequent Product Launch

12

A test report still has value

YES !NO !

Releasing x-times a day reduces the need for formal acceptance

13

The value of my work….

The testers mind

14

UT, ST, SIT, UAT

Big Data

Test techniquesDocumentation

New Technologies

MetricsIoT

Defects

Risks

StakeholdersProcess Stayin’ in Demand

15

It ain’t done until its accepted

16

Witness report: 2 outcomes

Advice to accept solution• Professional testing• Good process• Good test results

Advice to not accept solution• No professional testing• No Good process• No good test results

17

Witness proces

18

Agile techniques

In the real worldStakeholders need status info

19

20

It ain’t done until it’s tested

21

Testing a Central Topic Has the train arrived at the station?

PROJECT

22

OPSFrom: ProjectTo: Operations

23

It ain’t done until it’s tested

It ain’t done until it’s integrated

24

Another Case Study

25

Architecture• What are the

business processes?

• What are the components?

• What are the interfaces?

Acceptance criteria• What is the

Minimal Viable Product?

• What integrations are needed to make it work?

Requirements traceability• When are we

complete?• How do test

results add up to acceptance?

26

Missing

What should a car minimally do?

27

Planned Integration Tests

28

Integration tests

Release Date

Effects

29

Focus shifts from technique to Value solution

Split between

technical tests

and business

tests

30

Reduction of depencies

Early Feedback

Early Error Detection

Shift to Automation

31

Working software over Documentation

32

From Silo to Team effort

Methodic testing is declining

How much documentation?

33

Type of Test

Throw away

Regression

Compliance

Smoke & Monitoring

From: BUFDTo: JIT

TBYDWTFIP

34

Shift to Non-functional tests

Annotation: Asking the stakeholders for The Bugs You Don’t Want To Find In Production will define the

test you need to do before the code goes live. No doubt stakeholders will mention non functional items

like Security, UX and Performance

From componentto feature thinking

35

Ensuring Integration (rough sketch)

36

Organization

Component

System

Service

Continuously(in the sprint)

Occasionally(e.g. prior to a release)

CI/C

D

MBT

Service

Virtualization

Stubbing

e2e

ST

Test

Automation

UT

Integration

sprint

Manual RT

Interface

testing

Towards Value and System thinking

Annotation: Increasing the system (e.g from Units tot Systems) results in less frequent integration, because it becomes harder to test the integration. This has impact on the time-to-market.

PROFiTABLE?

37

Shift to Business tests

Annotation: Business test verify the business results of the

change or new service. Is the reduction in costs, the expected

revenue earned?

38

Agile Test Stategy

39

What needs to be tested

Steering committee Dashboard/ Roadmapping

Auditing the test work

Coaching the testing team members

Organizing tests that do not fit the sprint

We still need a plan to know

were completeThere is a need feedback and

traceability(comfort & adaptivety)

Testing needs be of quality to

justify itself

Testing needs to add value to

hold up an release

Test Report

40

Does it work as a

whole?

Hygienic

tests done?

Should I accept?

Tell the testing story

Do you know what is tested in the various teams?Do you know how well they did it?Do you have proof?How do you rate the quality?What concerns do you have (combine the bugs and translate to busines impact)?

41

42

Please, will you do some testing for me

WRAP-UP

43

ShiftsReleasing x-times a day reduces the need for formal acceptance

Shift to Automation and Bug hunting

Shift to Business tests

Towards value and systems thinking

Shift to Non-functional tests

From Project to Operations

Methodic testing is declining From: BUFD

To: JIT

Effects

45

Widespread Integrated in the process

Technical and faster

46

Integrated with development Integrated with

Business

Efficient and Complete

Traceable

Automatic

Is perceived to have value

47

Derk-Jan

ValoriColtbaan 4a3439 NG NIEUWEGEINThe Netherlands

• derkjandegrood@valori.nl• +31(0)651807878• www.valori.nl• @DerkJanDeGrood• http://djdegrood.wordpress.com

Derk-Jan

48

Success !