TMF – July 2010

Getting the Best Results out of Test Support Functions

Susan WindsorTel: +44 (0) 7974 808604

susan@gerrardconsulting.com

Web: gerrardconsulting.com

Slide 1Assurance with Intelligence

Where did this topic come from?

• After 15 + years of “Susan, help us find out what’s wrong with our testing!”– e.g. we spend a shed load of money on testing

and still have major faults in production!

• Can guarantee issues include:– Test environments– Test data– Configuration management

• Testing is dependent upon these – but they are the poor relations of testing

Assurance with Intelligence Slide 2

Why focus on Test Support?

• On average 60% of major defects in production (as discovered in 5 years of causal analysis) are due to environment or data issues

• Investment in testing on process improvement may only address 40% of your problems

• Do your own causal analysis to plan your investment priority

Assurance with Intelligence Slide 3

What do we focus on?

• We expend the majority of our time on direct test activities such as:– Planning preparation and execution cycles– What techniques to use?– Structured and/or exploratory?– What test tools to use, if any?– Have we got the right people with the right

skills in place to support testing?

• All very important of course, but......

Assurance with Intelligence Slide 4

What about......

• Environments

• Test Data

• Configuration Management

• Test Tools Support

Assurance with Intelligence Slide 5

Environments• Always a problem! Aren’t they?• Testers typically:

– Define environment requirements late– Talk in non-technical terms– Don’t ensure they are understood– Moan when they don’t get what they want when they

want

• Environment builders typically:– Also support production so this is 2nd priority– Don’t ask questions to clarify the requirements– Assume that once the kit is in place, the environment

is ready

Assurance with Intelligence Slide 6

Am I being harsh?

Test Data

• Why don’t we have a strategy for test data rather than assume the individual test phases will sort it out for themselves?

• Can we built it automatically?• How representative does it need to

be for each test phase?• Do we have any privacy or security

issues?• Your experience?Assurance with Intelligence Slide 7

Configuration Management

• Typically a weakness for most organisations

• So, how do we know what we’ve received when we start testing

• How much time is lost testing against incomplete releases? Do we track that?

• How do we control our own test assets

• Your experience?Assurance with Intelligence Slide 8

Test Tools Support

• Within the project or organisation wide?

• How much time is lost “getting up to speed” with standards, templates, the tool itself?

• Do we capture lost time?• Can you have a standard approach

anyway?• What’s in your plan for

training/awareness? Assurance with Intelligence Slide 9

In Summary

• Lets plan for the resources required to undertake Test Support functions, and where they will be located

• Lets pro-actively build relationships with environment providers now – so that when it really matters we already know how to communicate

• Lets capture lost time waiting for test support to be delivered

• Lets do causal analysis to identify (and evidence) our problem areas for investment

Assurance with Intelligence Slide 10