ISO/IEC 29119:ISO/IEC 29119:Software Testing Software Testing StandardStandardTest Automation WorkshopTest Automation WorkshopAugust 2007August 2007

Ms. Tafline Murnane

K. J. Ross & Associates Pty. Ltd.

PO Box 131, West Burleigh, 4219

Ph: 07 5522 5131 Fax: 07 5522 5232

taflinem@kjross.com.au

http://www.kjross.com.au

2

OverviewOverview

• Motivation• Goals & contributions• Overview of standard• Test processes• Test activity descriptions - example• Current progress• Summary

3

MotivationMotivation

• Existing standards do not cover all aspects of STLC– Techniques in BS 7925-2 only cover unit testing– BS 7925-1 testing vocabulary written for BS 7925-2– Missing higher level methods such as Use Case Testing and

non-functional testing approaches such as & Performance Testing, Security Testing, etc.

– Risk-based testing and test strategy development not covered – Static Testing not covered

• Potential conflict in definitions, processes & procedures• Practitioners may not know which standard to follow

4

Goals & ContributionsGoals & Contributions

• “The purpose is to produce an integrated set of international standards to cover the software testing process throughout the development and maintenance of a software product or system.” W. Suryn, 2007

• ISO/IEC JTC1/SC7 – working group 26– Dr. Stuart Reid, representing British Computer Society (UK) – Ms. Ursula Parker, representing IEEE (USA)– Ms. Tafline Murnane (Australia)– Mr. Alphonse Philippe (France)– Dr. Alastair Walker (South Africa)

• Contributions/reviews from industry & academia are invited

5

Overview of StandardOverview of Standard

Part 1Concepts & Vocabulary

Part 2Strategy & Process

Part 3Documentation

Part 4Techniques

ISO 25051:6

ISO 25051:7

IEEE 1008

BSI 7925-2

IEEE 829:2007

BSI 7925-1

Normative (Generic)

Appendices

e.g. Unit….

e.g. Integration…

e.g. System…

e.g. Acceptance…

e.g. Maintenance..

Normative (Generic)

Appendices Level

Technique 1 e.g. technique 1 Unit

e.g. technique 1 Integration

e.g. technique 1 System

e.g. technique 1 Acceptance

e.g. technique 1 Maintenance

Technique 2 … …

6

Part 1Part 1Concepts & VocabularyConcepts & Vocabulary

• Topics included:– Software testing concepts– Introduction to software testing– Relationship between testing, development & maintenance– Implications of lifecycle models– Types of testing– Testing vocabulary– … etc

7

Part 2Part 2Testing ProcessTesting Process

• Topics included:– Test management process

• Test strategy• Test process monitoring• Test project completion

– Test process• Test planning• Test design• Test execution• Anomaly reporting• Test completion

– Reporting• Status• Project

– Test environment support– Appendices:

• Examples of these processes, at each level of testing & for various lifecycle models

• Mappings to existing standards

8

Part 3Part 3DocumentationDocumentation

• Topics included:– Test management documentation

• Test strategy• Test project completion report

– Test documentation • Test plan• Test specification • Test results • Anomaly reports• Test level completion report

– Interim status reporting• Interim test status report• Interim test level status report

– Test environment report – Appendices – examples of each report, at each level of testing

9

Part 4Part 4TechniquesTechniques

• Topics included:– Test case design techniques

• Static testing techniques– Inspections & walkthroughs, etc.

• Dynamic testing techniques– Black-box, white-box, etc

• Non-functional testing techniques– Security, performance, etc.

– Test measurement techniques– Appendices

• Examples of each technique, at each level of testing

• Test technique effectiveness

• Mappings to existing standards

10

Part 2 – Testing ProcessesPart 2 – Testing Processes

Test ProjectCompletion Report

[Test completion criterion not met]

Test Level Completion

Report

Test Planning

(TP1)

Test Execution

(TP3)

Anomaly Reporting

(TP4)

Test Plan

TestSpec

Test Results [Issue

noticed]

[No issues noticed]

Anomaly Report

[Test completioncriterion met]

Design & Build Test(s)(TP2)

Write/Update& Initiate Test

Strategy(TMP1)

Monitor/Update Test Process

(TMP2)

Test Completion

(TMP3)

Test Process

Test Management Process

Test Strategy

Interim Test Status Reports

Test Strategy Updates

Project Management Plan,Regulatory Standards,System Requirements,

Specifications & Documentation

Test Item,Test Item

Specifications,Regulatory Standards

Interim Test Level Status Report

Test Level Completion

(TP5)

11

Test Activity Descriptions Test Activity Descriptions – Example – Example

• Each testing activity will be described using a standard template, for example:

Identifier: TP2Title: Test Design & BuildingPurpose: To design , document and resource test cases that will be executed during a

particular test activity. Entry Criteria: Test Plan for the testing activity has been writtenInputs: Test Plan, Test Item(s), Test Item Specification(s) Roles: Test AnalystTasks:

1. Review Test Plan2. Become familiar with test item(s)3. Review test item specification(s)4. Design and document test cases5. Allocate test cases to tester(s)

Outputs: Test SpecificationOutcomes:

1. Test case(s) has been documented; and2. Test case(s) has been allocated to tester(s) or required testing staff have been requested.

Exit Criteria: test case(s) has been documented.

12

Current ProgressCurrent Progress

• Developing requirements specification • Initial drafts of parts 1 to 3 in progress

– Review at first interim meeting in October 2007 (Montrèal)• Part 1: Concepts & Vocabulary (Alphonse)

• Part 2: Testing Process (Stuart & Tafline)

• Part 3: Documentation (Ursula)

– Liaise with other working groups on related standards

• Next plenary May 2008 (Berlin)– Respond to feedback on Parts 1 to 3– Continue review of Parts 1 to 3– Possibly review Part 4: Test Techniques (Stuart & Tafline)

13

SummarySummary

• International standard will provide practitioners with guidelines for testing that cover all aspects of STLC

– Provide consistent set of definitions, processes, procedures & techniques for software testing

• Will be adopted by IEEE, ISO & BSI• Currently has representation from five nations & will be

reviewed by software testing specialists world-wide– Many companies will be contributing/reviewing, including KJRA

14

Questions?Questions?

For more information…

• Email taflinem@kjross.com.au to be added to email list for information on progress of the working group & requests for contributions and/or reviews

• Visit http://www.jtc1-sc7.org/ for access to all documents produced by working group 26 and other working groups