Low budget tooling- Excel-lent?

Matthias Daigl

03.08.2009

Version 1.5

Agenda

1. Real Life

2. Myths about tools

3. Example: Defect management

4. Example: Test specification and test management

5. Example: Test case design and test case generation

6. Conclusion

Real life constraints

No time.

No staff.

No money.

Common Statements – True or False?

Commercial tools are expensive.

Freeware tools are unreliable.

In the long run, homemade tools are more expensive than commercial tools.

Spreadsheets are bad databases.

Good testers need good tools.

It is better to buy a tool than to build a tool.

Example: Defect tracking (Excel)

ID Label Date TesterDeveloperSeverityPriority Status Description History1 Wrong Label on Icon27.07.09 AB JH 4-Typo 4-Low 2-Open Label should be "CarConfigurator", is 2-Open (28.07.09 - JH)2 Model without price 27.07.09 AB CL 2-Restrict2-High 1-New Price for model "Rolo" is 0€, should be 12.800,-1-New (27.07.09 - AB)

7 Wrong discount 27.07.09 MD CL 2-Restrict2-High 4-InProcessDiscount is granted only at 4 items, should be at 3 items

4-InProcess (28.07.09 - CL)2-Open (28.07.09 - CL)1-New (27.07.09 - MD)

8 Config dialog 28.07.09 MD PK 3-Workaround3-Medium1-NewThe buttons in the configuration dialog are at wrong positions (cf to gui guidelines)

1-New (28.07.09 - MD)

15 App Crashs after Start28.07.09 DR JH 1-Crash2-High 5-Test The application crashes immediately after start 5-Test (28.07.09 - JH)

What you can do Store your data (insecure)

Work concurrently (limited)

Filter data

What you can‘t do Manage large projects

Real concurrent work

Have a role based workflow

Automatically send messages

Keep some historical values

Reporting…

Conveniently track your history

Get a clear view on the data if you are using detailed attributes

Example: Testspec & Testmanagement (Excel)

Requirements document: Functional Specification myCarTest object version: 1.0

Test specification date: 01.04.2009Path: P:\projects\imbus\sample\new\docs

Reviewer: John Doe

FrameworkRequired tools: Word, Excel

Test environment: StandardSW environment: StandardHW environment: Standard PC

Referenced documents: cf. Req. DocumentRemarks: -

Status# test cases created

# requirementsAnteil (in %)

Status wk 19 wk 20 wk 21 wk 22 wk 23 wk 24 wk 25 wk 26open 0 0 0 0 0 0 0 0

ok 0 0 0 0 0 0 0 0nok 0 0 0 0 0 0 0 0

blocked 0 0 0 0 0 0 0 0executed (total) 0 0 0 0 0 0 0 0planned (total) 1 3 4 5 6 8 9 10

percentage 0 0 0 0

Statusopen

oknok

blockedtotal

0

test cases total 1st reg.test A10

System Testing Specification

Test case creation progress

0

2nd reg.test B 3rd reg.test C

Test execution progress

00

wk 140

000

00000

00

0

status of test cases in execution

0000

20

wk 15104

250 166,66666676

10wk 16 wk 17

107

142,8571429

0

2

4

6

8

10

12

wk 19 wk 20 wk 21 wk 22 wk 23 wk 24 wk 25 wk 26

testcase

count

planned (total)

executed (total)

0

2

4

6

8

10

12

wk 14 wk 15 wk 16 wk 17

test cases

# test cases created

# requirements

Testcase # Type Priority Requirement Creator Created Tester Test date Status Bug # System Precondition

xxxx Reg.test ?high/

medium/low Requirement

Name Date NameDate

ok/nok/

blocked/open

#1 2

TUW_0010 high check basic details md 08.04.2009 md open CarConfig …

Organisation Contents

What you can do Keep your data…

Get comprehensive reports (test progress…)

What you can‘t do Span reports across test objects

Work in large teams

Manage large projects

Versioning (or forget reporting)

Only limited workflow…

Only limited support for logging of multiple test cycles

A Professional Test Management Tool: TestBench

Version control for test cases and test execution dataConcurrent, distributed workControl large projects with a large number of people and test casesProvide statistical data across test cyclesInterfaces with other tools readily availableWorkflow, role model…

Example: Test design (CEG)

Support advanced test techniques

Generate test cases/test data

Conclusion: Why we prefer professional tools

Office based tools are only advisable for small projects or small teams (in general - there are exceptions)

Functionality of homemade tools is often limited due to limited development resourcesFunctionality of homemade tools is often limited as it represents the skill level/maturity level of the organisationProfessional tools provide more comprehensive featuresProfessional tools provide support (not usually for free tools)

Development effort for professional tools is shared by all the customers – so they are (should be?) cheaper in the long run (don‘t forget maintenance…)Higher productivityFrom (many) documents to (one) databaseKnow-how is for all projectsEasy to use

Conclusion: Why we might still use homemade tools

If we can‘t get a "real" tool:Because there is no budget

It is not yet available (interim solution)

It might be easier to get the budget for people to build a tool, than to get the budget to buy a tool (regardless of the final costs – don‘t forget maintenance…)

If we see that staff can not handle professional tools *today*

If we need time to figure out our real needs

If we need features which are not available in commercial products

Commercial tools might not „fit“ our people

If a quick solution is needed

Conclusion: Risks & Opportunities

Risks in creating advanced tools based on office products:

Eventually, higher costs than commercial tools

Working with office based tools is generally less efficient

Own solution might block progress

It’s likely to get a high complexity in the interaction of many simple documents missing overview, mistakes, lacking precision

Opportunities:

A solution is available at short hand

The solution will fit our environment

Customising is possible and cheap

We still might buy a professional tool later

Be not afraid of going slowly; be afraid only of standing still.

Chinese Proverb

imbus AGKleinseebacher Str. 991096 MöhrendorfDEUTSCHLANDTel. +49 9131 7518-0Fax +49 9131 7518-50

Weitere Standorte:

imbus AGUnter der Linde 1680939 MünchenDEUTSCHLANDTel. +49 89 3219909-0Fax +49 89 3219909-50

imbus Rhein-Main GmbHKirschgartenstr. 1565719 HofheimDEUTSCHLANDTel. +49 6192 92192-0Fax +49 6192 92192-50

imbus Rheinland GmbHVolksgartenstr. 3650677 KölnDEUTSCHLANDTel. +49 221 998788-0Fax +49 221 998788-50

info@imbus.de

www.imbus.de © 2009 imbus AG