The Changing Roles of Testers

@pau

l_ge

rrard

Paul Gerrard paul@gerrardconsulting.com

gerrardconsulting.com

Programme Chair 2014

Helping clients transform their testing through

INNOVATION, COACHING and LEADERSHIP

Our CLIENTS

– Want to be agile rather than follow Agile dogma

– Have a pragmatic approach and are focused on delivery

– Want a solution that fits, not a badly fitting suit.

Typical tester job spec?

• Skills & Requirements – Thorough C# knowledge including Linq

– Good .Net experience including charting and winforms applications.

– Experience building and maintaining automated test systems

– Ability to interact with the developers to anticipate future needs

– Ability to influence to ensure good test procedures are developed and implemented

• Desirable Skills – Some knowledge of Java for mobile testing

– An interest in photography.

Agenda

• Pressure to Change

• Three Innovations

• Where is this Leading?

• A New Model for Testing

• Consequences

• Why Shift-Left is More Important than Agile

What's happening?

• Stampede to mobile computing

• Big Data

• Internet of (Every)Thing, Pervasive Computing

• Continuous Delivery, DevOps

• (Test) Analytics, Data-Driven Development

• Shift-left, embedded testers, no test team

• Did you really think Agile was the last word?

Three Innovations That

Are Driving Change

Being agile

Shift Left

Analytics

#1

Agile doesn’t work

but being agile might

Agile is no longer innovative.

Challenging Agile and being agile

might be

Is continuous delivery for you?

Agile doesn’t work?

• It works for some

• But many organisations have difficulties

• Mostly a cultural problem

– Requirements take months to elicit and agree

– Complex stakeholder relationships

– Stakeholders with ‘all the knowledge’ can’t be spared to be the on-site customer/product owner

– Lack of confidence => big system/acceptance tests

• We are Agile but ...

Pragmatism rules (again)

• Pragmatic companies use the things that work

and discard those that don’t

• Let's be honest: Most are doing some form of

hybrid approach:

– Chaotic interfaces between Agile teams and

structured businesses

– TDD, BDD, ATDD, SBE today…

– Continuous Delivery, DevOps… tomorrow?

#2

Shift Left

Developers in test?

Testers in development?

Testing (testers?) everywhere

Testing BDD/Test-Driven) Trusted

Requirements + Stories

Shift left (redistributed testing)

Functional Requirements

Build (+little testing) but lots of rework

Testing (lots of checking + bugs)

Trusted Requirements +

Stories Testing

SAVED TIME

BDD/Test-Driven

Who does the testing now? BAs, Developers or Testers? Stepping stone to continuous delivery...

#3

Test Analytics

Analytics from production

Analytics from test

An opportunity for testing

From traditional delivery…

… to Continuous Delivery

Deployment into

Production is not the

Same as ‘Going Live’

Release to production and test

environments on-demand

Go Live weekly/monthly/quarterly…

Analysis Insight Decision Development and Testing

Production Testing

Introducing Test Analytics

“Introducing Test Analytics”, Paul Gerrard:

http://gerrardconsulting.com/index.php?q=node/630

Julian Harty, "Symbiotic Relationships between Testing and SMAC":

http://blog.bettersoftwaretesting.com/2013/07/symbiotic-relationships-between-

testing-and-social-mobile-analytics-cloud-computing/

“The capture, integration and analysis of test and

production monitoring data to inform business and

software development decision-making”.

Data silos for test analytics (an illustration)

Stakeholder Requirements Assurance (Production)

Application

Monitoring

Production

Environment

Monitoring

Stakeholders

Business Goals

and measures

Stakeholder

Involvement/

Engagement

Risk

Requirements

Stories/Feature

Descriptions

Glossary of terms

and Term Usage

Processes

Process Paths

(Workflows)

Manual Tests

Generated Test

Code (unit,

integration,

system level)

Application

Instrumentation

Automated Test

Execution History

Test Status

Feature Code

Change History

Application

Process Flows

User Accesses/

Activities

Feature Calls,

Response times

Feature use cases

(data usage)

Application

Alerts/Failures

Database Accesses

Database Content

Production

failures

System assets

Resource Usage

Log

Performance data

System Events

System Alerts/

Failures/Incidents

Outages

Where is this leading?

If I knew that, I wouldn't be here...

Three development patterns

Structured

Agile Continuous

Autonomous

But there aren't three

patterns

There are many

From high process to DIY process

• In the past, high process was seen to be the

technical solution to the software problem

• But technical solutions cannot solve human

problems

• Agile promised flexibility, but a lot of Agile is

really 'high process-in-the-small'

• True agility means a team can adapt or select

its process on-the-fly.

The old ways won't

work in the future

We need a New Model of Testing

(free from logistics)

A New Model for

Testing

Don't throw stones. Yet.

I'm working on a refined version (with others).

Want to help/review?

Forget Logistics (for the time being)

Document or not?

Automated or manual?

Agile v waterfall?

Planned v exploratory?

This technology v that technology?

ALL Testing is

Exploratory

We explore sources of knowledge ...

... to build test models ...

... that inform our testing.

All Testing is Based on

Models

Examples of models

Models are innate, essential, human

Judgement, exploring and testing

Testing (the system)

Our model(s) are adequate

Our model(s) are not adequate

Exploring (sources)

Judgement

Creates test models

Uses test models

We explore sources of knowledge to build test models that inform our testing

BTW – Do Developers explore the same way? I think so.

Exploration process

Exploration Definitions specs/stories

People (& you)

Sources

Require-ments

Test Models

Enquiring

Challenging

Sources: People, documents, experience, system under test

Modelling

Test Models: Can be documented

or mental models

Predicting

System under test

Testing process

Testing System

Under Test

Refining

Informing Applying

Interpreting

Test Models

Revise the System

More exploring Reporting

New Model Testing

29 page paper: http://dev.sp.qa/download/newModel

Some Consequences

Do developers and

testers think the same

way?

Similar processes, not thinking the same

Easier to embed testers in development

Developer testing will improve

Covers functional and

non-functional testing?

Ignoring logistics means we can have

the same foundational process for

ALL testing

(I think)

Relation to

TDD and BDD?

TDD is not testing

BDD is modelling using stories

Testing v Checking?

I submit that a Check is "a test that

can be applied by a human or tool"

But only humans can interpret

Test automation from a

different perspective

Automation uses different test models

Is that why automation efforts fail so often?

Developer Testing

The model works (for me)

Capabilities

Enquiring, Modelling, Predicting, Challenging

Informing, Applying, Interpreting, Refining

Reporting and Logging

• Analysis, enquiry and elicitation

• Modelling

• Creation of custom models, using

heuristics, guesses, brainstorming,

ideation, creative thinking

• Custom test design techniques

• Comparison of models, value, advantages,

disadvantages, compromises

• Identification, validation and use of

oracles

• Predicate logic and proof

• Hypothesis and inference

• Socratic method

• Rapid Review and Inspection techniques

• Test case design

• Test models and the meaning of coverage

• Testing as controlled experiment

• Observation, Note taking, recording

A very different skillset • Basic data analysis and statistics

• Decision-making with incomplete data

• Computer forensics

• Fault tree analysis

• Failure diagnosis

• Bug advocacy, triage processes and

negotiation

• Meaningful software and test metrics

• Visual presentation of data

• Reporting and presentation skills

• Understanding stakeholders

• Test analytics

• Risk management, risk-based testing and

decision-making

• Critical Thinking

• Interpersonal skills

• Dealing with uncertainty/fallibility

Testing Career Development (speculative)

Foundations

Technical

Management

Strategic

Test Strategy Project Intelligence

Test Assurance

Exploration Forensics Interpretation

Scripting/ Programming

Test Automation Technical (Excel, SQL, OS utils etc)

Stakeholder management

Analytics & visualisation

Managing uncertainty

Critical Thinking

ISTQB etc...

Supplier Management

Test Process Management

Methodology

Why Shift-Left is More

Important than Agile

Developers write code;

testers test because

developers can’t

No they don’t

Goal: End Manual

Feature Checking

Let the tools do it and

free people to do testing please!

Testing doesn’t just

apply to existing

software, at the end

We test systems, not software.

We test early, we test often but it

might not be called testing

Testers Need to Learn

How to Code?

Some, perhaps many will have to

Testing is about

measuring achievement,

not quality

Align systems delivery with project

goals and risks and you’ve got a seat

on the board

Testers don’t own

testing anymore

Be a TestMaster

"Assurance through the lifecycle"

New Model Testing

changes what testers &

developers need to

learn

Roles of developers/testers become blurred

Perhaps they are not different disciplines after all?

Why have a tester certification?

The phase after

development is

REWORK, not Testing

Testing must align with development,

not compete with it or rescue it

Agile was for some

Shift-left is for ALL

That is why it's more important.

The Changing Roles of Testers

@pau

l_ge

rrard

Paul Gerrard paul@gerrardconsulting.com

gerrardconsulting.com

Programme Chair 2014