P R E S E N T A T I O N

International Conference On Software Testing Analysis & Review

October 29-November 2, 2001 San Jose, CA USA

Thursday, November 1, 2001 10:30 AM

THEORY AND PRACTICE IN TEST PROCESS IMPROVEMENT AT

BARCLAYS BANK

Kath Harrison Barclays Bank Pic

and

Martin Pol POLTEQ IT Services B.V.

T5

Presentation Bio

Theory and Practice in Test Process Improvement

Kath HarrisonMartin Pol

Objectives

To highlight practical learning points from developing

and implementing a Test Process Improvement project

in a large IT development organisation

• What will this presentation tell me ?

• Who is this presentation for ?

• What this presentation is not

Agenda

• Introduction to TPI

• Process of change

– Theory

– Barclays Experience

• Lessons learnt

TQM

SPI

TPI

Positioning and Scope of TPI

Test Process Improvement ?

Testing is uncontrollable

Testing costs too much

Testing takes too much time

Testing doesn’t help, defects left!

“Continuous improvement of the quality and the

efficiency of the testing process, related to the

output of the total software process”

Complaints

Test Process Improvement!

• Quality Timeliness CoverageInsight Control

• Costs Cheaper

• Lead Time Faster

“Continuous improvement of the quality and the

efficiency of the testing process, related to the

output of the total software process”

Improvement Life-cycle in General

• Goal and scope

• Current situation

• Desired situation

• Implementing changes

Required:

reference model

Key areas

Levels Test MaturityMatrix

Improvement suggestionsCheckpoints

The TPI® Model

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Process of Change

Change Project: Requirements

Knowledge of:

– Structured testing

– Test processes

– Organisation

– Management of change

– The TPI model

A Team!

Time

Sponsors

Social skills

Agenda

• Introduction to TPI

• Process of change

– Theory

– Barclays Experience

• Lessons learnt

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Process of Change

Awareness

• Necessity of change

• Commitment on

– Problem

– Approach

– Goals

Continuous activity !

Target, Scope, Approach

Target

– Time

– Money

– Quality

Scope

– One or more test levels

– Project / organisation

– Key areas / Levels

Approach

– Change process

– Organisation

– Costs of change

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Barclays Process of Change

Barclays

• UK based financial services organisation with operations in 60 countries

• Incorporates Barclaycard - largest credit card group in Europe

• Profits for year to Dec 2000 of £3,456 million

• Facts

– UK Branches - 2129

– Overseas Branches - 624

– ATM’s - 3,800

– Current Accounts - 8.3 million

– Telephone Banking customers 1.2 million

– Internet Banking customers - 3 million

Barclays IT Development Organisation

General

• Development staff of approximately 1,800

• About 200 of these work in testing

• Split across 2 major sites and some minor ones

• Delivers around 500 projects each year

• IT Development budget of £100 million

Testing

• Centralised testing capability

• Testing Services currently delivers 50% of all testing (Approx. 130 staff)

Background Testing

• A familiar story - improve speed to market, cost, quality of products

• Some high profile issues that could be exploited

• Created a vision of where we wanted to go - the horizon

TPI Approach

• TPI is first and foremost a Business Change project (Transition project)

• Scope was carefully managed

• Management and stakeholder buy-in secured

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Process of Change

Assessment

“If you don't know where you are,

a map won't help” (Watts Humphrey)

• Preparation

• Collecting information

• Analysis (using the TPI model)

• Reporting

Improvement actions

• In synchronisation with targets!

• Levels of TPI model

• Other criteria:

– Fast, visible results

– Low costs

– Easiest actions first

– Acceptance level

– Best cost / profit ratio

– Decrease highest risk

– …..

Plan

How to steer the change process into the right

direction?

What ?

Who ?

When ?

How ?

How much ?

.. … ? Test specific activities and

Change specific activities

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Barclays Process of Change

TPI Approach

• Approach and planning thorough and linked to vision

• Used TPI model as a vehicle to take us where we were going

• Transition project recognised most of the things that would impact the project

Areas of focus

• Lifecycle and techniques

– Lifecycle model

– Strategy technique

– Specification techniques

– Defect management

– Reporting

• Tools

• Environments

• Management of change

Management of change

• Project management

• Communication

• Metrics

• Training

• Pilots

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Process of Change

Implementation

• Execute the plan

• Measure (self-assessments)

• Consolidate

Communicate !

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Barclays Process of Change

Areas for Focus

Managementof Change Lifecycle

& Techniques

Tools

Environments

More Managementof Change More Lifecycle

& Techniques

More Tools

More Environments

• Detail takes over

• Vision is not static

Below the Water - Life Cycle & Techniques

• Fit with development lifecycle tool set

• Legacy test products

Below the Water - Tools

• Historically problematic

• High negative impact on testing

Below the Water - Environments

• Communication

• Metrics

• Support

• Project demands

• Culture

Below the Water - Management of Change

• Process driven

• Date driven

• Project focus

• Resistance

Below the Water - Culture in Barclays

• SPI

• Process V tools

• Integration V complexity

• Resistance

Below the Water - Culture in Development

• BAU V transition

• 3rd Party suppliers

• Programme

• Resistance

Below the Water - Culture in Testing Services

• Skills V ability

• Comfort zone

• Resistance

Below the Water - Culture of Testing Professionals

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Process of Change

Evaluation

• Success ?

• Targets met ?

• Next step

– Assessment

– Adjust improvement actions

– New targets or areas of consideration

– Next improvement process

– Stop

performevaluation

formulateplan

implementimprovement

actions

commitment

executeassessment

defineimprovement

actions

objectives scope

approach

Barclays Process of Change

• Testing (Testing Services) on the map

• Master Test Plan: integral risk and test control

• First steps in automation

• Improved morale

• Roles, career path and training available

• Baseline established

• Gaining experience

• Reduced costs ?

• Reduced timescales ?

Achieved Benefits

Agenda

• Introduction to TPI

• Process of change

– Theory

– Barclays Experience

• Lessons learnt

Lessons learnt

• Implementation requires competent testing and project management professionals, and sound leadership

• Accept that resistance is an integral part of any business change project

• Don’t over engineer the solution - plan for incremental change over time

• Pay attention to all 4 cornerstones of testing

• Tools can only support the lifecycle and techniques

Stay focussed on Testing!

Thank you !

Martin Pol Martin Pol has more than 25 years experience in the information business. Since 1983 he's completely dedicated to Testing. He's gained experience by managing testing processes and implementing structured testing in many organizations in different branches. With exceptional insight and experience in practical testing issues, Martin is a regarded presenter at conferences and training sessions throughout Europe, the Americas and Asia. As Manager R&D of IQUIP Informatica B.V. he was responsible for the creation of the structured testing approach TMap(r) and the Test Process Improvement model, TPI(r). Martin Pol is the co-author of three (Dutch) books on TMap(r) and TPI(r) which have been translated in English and German. A Japanese version is due in 2001. Under his management TMap(r) became a de facto testing standard. Now the approach is used in more than 400 companies Europe and the USA. The international publication of the TPI(r) book in English and German caused world wide application by many key-players in the IT-industry, e.g. NOKIA, Deutsche Telekom, AXA, Dell, Compaq, Intel, Novell, ABN-AMRO bank, Barclays bank, etc. Since the model is complementary to Software Process Improvement models such as the CMM, TPI(r) is frequently used within Software Engineering Process Groups. Martin Pol was Programme Chair of the EuroSTAR conference in 1996 and 1997 and is chairman of the Dutch Special Interest Group in Software Testing, TestNet. He recently received the "European Testing Excellence Award" for his contribution to the field of testing across Europe. Through his company Polteq IT Services B.V. he's offering international test consultancy services now. TMap(r) and TPI(r) are registered trademarks of IQUIP Informatica B.V. in the Netherlands. Martin Pol was the former manager R&D of IQUIP Software Control and responsible for the development of both the TMap(r) approach and the TPI(r) model.