Sogeti’s updated Test Management approach · Sogeti’s updated Test Management approach Geert...
Transcript of Sogeti’s updated Test Management approach · Sogeti’s updated Test Management approach Geert...
TMap® Next
Sogeti’s updated Test Management approach
Geert Vanhove
People you know. Knowledge you need2
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Agenda
Sogeti
Situation of TMap® Next
Business driven test management
Structured test process
Complete toolbox
Adaptivity
Conclusion
Questions
People you know. Knowledge you need3
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Sweden
Norway
Denmark
Germany
Switzerland
Spain
Belgium/Luxemburg
Netherlands
Ireland*
France
UK**
Geographical presence Sogeti
◦ Revenues 2008 €1.6 billion, profit >10%
◦ A work force of over 20.000 staff spread over 200 locations
India
USA
* Insight Test Services
- acquired 04/2008
- 80 employees
** Vizuri
- acquired 10/2008
- 90 employees
People you know. Knowledge you need4
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Our services
4
Application Management
High Tech Engineering
Infrastructure Management
Software Control Testing
People you know. Knowledge you need5
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
A company specialized on business driven testing based on internationally acknowledged methods TMap® and TPI®
Markets the unique Managed Testing Services model with a fit-for-purpose match of on shore, near shore and off shore capabilities
Sogeti Group employs 2.500 career test specialists organized in dedicated entities spread over 14 countries including our full-flex off shore test center in India
On shore in 12 European countries
On shore throughout the U.S.
Sogeti testing facts
People you know. Knowledge you need6
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Sogeti testing service portfolio
Test Management
— BDTM
— Risk based testing strategy
Test Automation / Tool Support
— Compuware suite
— HP Mercury suite
— IBM Rational suite
— Borland (Performance testing as a service)
— Other specialist tool providersTest Engineering
— Unit testing
— Integration testing
— System testing
— Acceptance testing
— Maintenance testing
Training
— Test Management
— Test Engineering
— TPI
— Test Automation
— TMap Certification (Foundation, Advanced)
— ISTQBSpecialized Testing
— Agile testing
— SOA testing
— End-to-end testing
— SAP testing
— Data warehouse & BI testing
— Migration testing
— Performance testing
— Security testing
— Usability testing
— Infrastructure testing
Consulting & Quality Assurance
— Assessments (e.g. TPI)
— Audits (e.g. CMMI)
— Quick scans
— Implementation management
— Operational QA
High Tech Testing
— System testing (e.g. aircraft control)
— General physics testing (e.g. mechanical)
Managed Testing Services
People you know. Knowledge you need7
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Agenda
Sogeti
Situation of TMap® Next
Business driven test management
Structured test process
Complete toolbox
Adaptivity
Conclusion
Questions
People you know. Knowledge you need8
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Many different Many different
disciplinesdisciplines
Conflicting Conflicting
interestsinterests
UnpredictabilityUnpredictability
Complex Complex
managementmanagement
Time pressureTime pressure
Maximum value Maximum value
from softwarefrom software
Take control on Take control on
risksrisks
Make better Make better
decisionsdecisions
Manage risksManage risks
Software testing
People you know. Knowledge you need9
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Delivers insight into, and advice on, any risks in
respect of the quality of the tested system
Testing is on the critical path of the total
development as briefly as possible
Finds defects at an early stage, prevents
defects
Test products are reusable
Test process is understandable and manageable
For any situation, client or system
development approach:
Deliver
"more for less, faster and better"
with TMap® Next
Software testing
People you know. Knowledge you need10
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Sogeti and testing
Inventor of TMap®(Next), TPI®, TAKT®
International standards
Various alternatives
People you know. Knowledge you need11
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test Management Approach (TMap®Next)
Key elements
◦ Business driven test
management
◦ Full description of the test process
◦ Complete tool set
◦ Adaptive
Business driven test management Structured test
process
Complete toolbox
Adaptive
Flexible
Method
Testing is a processthat provides insight into, and advice on, quality and the related risks
People you know. Knowledge you need12
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
TMap® Next - the book and the web site
Checklists
Templates
Tools
White papers
Etc.
www.tmap.net
People you know. Knowledge you need13
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
TMap® Next – index of book
1 Introduction
2 Framework and importance of testing
3 The essentials of TMAP®
4 Introduction to the processes
5 Master test plan, managing the total test process
6 Acceptance and system tests
7 Development tests
8 Supporting processes
9 Product risk analysis
10 Quality characteristics and test types
11 Estimation techniques
12 Defect management
13 Metrics
14 Test design techniques
15 Evaluation techniques
16 Test roles
GENERAL
PROCESSES
COMPONENTS
People you know. Knowledge you need14
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
TMap® related services
TMap®
People you know. Knowledge you need15
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Agenda
Sogeti
Situation of TMap® Next
Business driven test management
Structured test process
Complete toolbox
Adaptivity
Conclusion
Questions
People you know. Knowledge you need16
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Business driven test management
Business case:
• Result
• Risks
• Time
• Costs
Find a good balance between the 4 BDTM aspects
People you know. Knowledge you need17
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Business driven test management
Demonstrate
Infor
m
Appoint
BDTM
Project status(Time/Costs)
Risks
Result
People you know. Knowledge you need18
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test execution
Creating test cases
Test basis
5
Allocating test techniques
Business driven test management
4
Result, risks,time and costs
6
3
Determining light/thorough testing
Critical success factors
Change proposalsRequirements
Business processes… 2
Determining risk class
CLIENTCLIENT
1
Assignment and test goals
People you know. Knowledge you need19
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Risks and test coverage
risks
testcoverage
Plan:- strategy-budget- planning
risks
testcoverage
Execution:- other risks- less test coverage
risks
testcoverage
Adjusted approach:- strategy-budget- planning
With client
People you know. Knowledge you need20
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
• Damage• Frequency of use
System
administrator
Data
center
Accountancy
CM & CC
Technical
design
Project
TEST
QADBA
Business
Functional
design
• Chance of defects
management
Developer
Physical
management
Business Driven Test Management
People you know. Knowledge you need21
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
BDTM – Product risk analysis (example)
Outcome of product risk analysis
• Combining characteristics/object parts/risk classes
Performance
B-> Online
BUser-friendliness
Functionality
High chance of failure, used in vital processes 1 and 2A-> System 1
Medium chance of failure, used only to limited extent in vital process2
B-> System 2
If system 1 and 2 function correctly, the risk of integration problemsis low
C-> Total
C-> Batch
Risk class
MotivationCharacteristic
People you know. Knowledge you need22
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
BDTM – High level test strategy (example)
�
�
Evaluation
�
�
DT
I
�
��
���
ST
���
�
���
UAT
��
���
PAT
Performance
B-> Online
BUser-friendliness
Functionality
A-> System 1
B-> System 2
C-> Total
C-> Batch
Risk class
Characteristic
Product risk analysis Test strategy
People you know. Knowledge you need23
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
BDTM – Detailed test strategy (example)
ToolPerformance testTotal system (B / �)
Performance - online
Exploratory testingRegression testSystem 2
(B / �)
Functionality
DCT, SYN, SEMFunctional testSystem 2
(A / ��)
Functionality
Test techniqueTest typeObject partCharacteristic
High level test strategy Detailed test strategy
People you know. Knowledge you need24
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Business Driven Test Management
The key to BDTM is balanceThe key to BDTM is balance
People you know. Knowledge you need25
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Characteristics of BDTM
• Gives client grip on test process• Uses client language • Bases testing on product risks• Delivers appropriate test coverage on right spot
• Makes test results visible
People you know. Knowledge you need26
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Agenda
Sogeti
Situation of TMap® Next
Business driven test management
Structured test process
Complete toolbox
Adaptivity
Conclusion
Questions
People you know. Knowledge you need27
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
V-model
People you know. Knowledge you need28
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Master test plan - Managing the total test process
Planning phase
Master test plan
System test
Development test
Acceptance test
Reviews
Special attention for:• Business driven test management• Set up organisation
Master test plan
Test plan per test level
control, report and adjust
execute
and Control phase
People you know. Knowledge you need29
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Master Test Plan (MTP)
Main purposes of MTP:
• Total overview of test process: what to test (when and how) in which test levels – alligning test levels
• Minimizing risks of overlap or gaps in test coverage
• Detect most important defects as early as possible (critical path)
• Ensure uniformity in processes (e.g. defect procedure)
• Optimal distribution of available resources (testers, test environment, tools, expertise)
• Agreements with stakeholders
• Approach, planning, estimated effort and activities anddeliverables in relation to total test process
People you know. Knowledge you need30
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Structured test process
Phasing model as a base
P P S E C
I
Planning
Control
Setting up and maintaining infrastructure
Preparation Specification Execution Completion
C
>> Mainly for acceptance and system testing
People you know. Knowledge you need31
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Structured test process
Specify infrastructureRealise infrastructureSpecify intake infrastructureIntake infrastructureManage infrastructureConserve infrastructure
Specify infrastructureRealise infrastructureSpecify intake infrastructureIntake infrastructureManage infrastructureConserve infrastructure
InfrastructureInfrastructure
P S E C
I
C
P
Collect test baseSet up checklistAssess test baseSet up report detail intake
Collect test baseSet up checklistAssess test baseSet up report detail intake
PreparationPreparation
Set up specificationsDefine situationsSpecify intake test object
Set up specificationsDefine situationsSpecify intake test object
SpecificationSpecification
Intake test object (pretest)Set up situationExecute (re)testsControl and assess
Intake test object (pretest)Set up situationExecute (re)testsControl and assess
ExecutionExecutionControlControl
Manage, monitor, report, steerManage, monitor, report, steer
Planning Planning
Determine assignment……Assign test techniques……Fix test plan
Determine assignment……Assign test techniques……Fix test plan
Master te
st p
lan
Evaluate test procesConserve testware
Evaluate test procesConserve testware
CompletionCompletion
People you know. Knowledge you need32
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
BDTM in TMap® Next phasing model
1. Establishing the assignment
2. Understanding the assignment
3. Determining test base
4. Analysing the product risks
5. Determining the test strategy
6. Estimating the effort
7. Determining the planning
8. Assign test units and test techniques
9. Defining the test products
10.Defining the organisation
11.Defining the infrastructure
12.Organising the management
13.Determining the test project risks
and adjustments
14. Feedback and consolidation of the plan
1. Management
2. Monitoring
3. Reporting
4. Adjusting
BDTM
BDTM
BDTM
P S E C
I
C
P
People you know. Knowledge you need33
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
TMap® Next processes
Master test plan, managing the total test process
Acceptance and system test
Development test
People you know. Knowledge you need34
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Main differences from system/acceptancetesting:
• Not an independent process by an independent team, part of development process
• Other type of defects found (technical aspect)
• Each statement in the code has been touched once
• Communication on defects is minimal (tester and developer are often the same person)
• Approach: all found defects should be resolved beforedelivery of software (minimal reporting)
• First test traject: cheap and fast defect resolving(flexible environment needed)
• Mindset of developer and tester is different
Development test
People you know. Knowledge you need35
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Using knowledge of the technical
implementation of the system
Unit test: testing the smallest parts of the system
Unit integration test: testing the integration with the other units
• Test-driven development
• Pair programming
• Code review
• Continuous integration
• Application integrator
approach
Development test
People you know. Knowledge you need36
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Characteristics of structured test process
• Applicable for all test levels• Coordinates and synchronizes different test levels
• Assigns tasks and responsibilities among parties involved
• Split in phases, activities and products
People you know. Knowledge you need37
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Agenda
Sogeti
Situation of TMap® Next
Business driven test management
Structured test process
Complete toolbox
Adaptivity
Conclusion
Questions
People you know. Knowledge you need38
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Complete toolbox
Techniques
• How is tested?
Infrastructure
• Where is tested?
• With what is tested?
Organisation
• By whom is tested?
People you know. Knowledge you need39
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Techniques
Test design techniques
Estimation techniques
Defect management
Metrics
Templates
Product risk analysis
Evaluation techniques
Etc.
People you know. Knowledge you need40
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test design techniques
Appropriate coverage on according system part
Various types of coverage
Defineassignment
Riskanalysis
Test basis Test cases
Test strategy
Coverage
Test designtechniques
Specification oftest cases
People you know. Knowledge you need41
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test design techniques
Test basis
Test cases
Test situations
Test designtechnique
Coverage
Coverage type
Coverage ratio
Basic technique
Detailed overview
People you know. Knowledge you need42
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test design techniques
Coverage types & basic techniques
.
.
.
Test design techniques
People you know. Knowledge you need43
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test design
Test situation
Test case
Test script
Show that the requested coverage has been met
Concrete development of test casePreparation of test execution
Logical test case
Physical test case
Logical test design
Physical test design
Relation between test design concepts
People you know. Knowledge you need44
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Coverage types
Paths
Decision points
Equivalence classes
Boundary value analysis
Pairwise testing
Create/Read/Update/Delete (CRUD)
Right paths/Fault paths
Checklist
Orthogonal arrays
Operational profiles
Load profiles
People you know. Knowledge you need45
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test design techniques
Decision table test (DTT)
Data combination test (DCT)
Elementary Comparison test (ECT)
Error guessing (EG)
Exploratory testing (ET)
Data cycle test (DCyT)
Process cycle test (PCT)
Real life test (RLT)
Semantic test (SEM)
Syntactic test (SYN)
Use case test (UCT)
People you know. Knowledge you need46
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Complete toolbox
Techniques
• How is tested?
Infrastructure
• Where is tested?
• With what is tested?
Organisation
• By whom is tested?
People you know. Knowledge you need47
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Infrastructure
Main components
• Test environment
• Tools
• Workplace
People you know. Knowledge you need48
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Test tools
Situation in the V-model
RequirementsSpecification
FunctionalDesign
TechnicalDesign
Realisation
AcceptanceTest
IntegrationTest
SystemTest
UnitTest
Test designtools
Staticanalysis tools
Coveragetools
Performanceand simulatortools
Test execution andcomparison tools
Management tools
People you know. Knowledge you need49
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Complete toolbox
Techniques
• How is tested?
Infrastructure
• Where is tested?
• With what is tested?
Organisation
• By whom is tested?
People you know. Knowledge you need50
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Organisation
Project
Managed
Testing
Services
Managed
Resourcing
Time &
Material
People you know. Knowledge you need51
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Agenda
Sogeti
Situation of TMap® Next
Business driven test management
Structured test process
Complete toolbox
Adaptivity
Conclusion
Questions
People you know. Knowledge you need52
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Adaptivity
Adaptive on different levels
• Client
• Application
• Tool box
People you know. Knowledge you need53
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Adaptivity
React on changes
(Re)use
Learn from experience
Try before use
People you know. Knowledge you need54
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Be adaptive!… Use it!
More than 400 tips, cases and examples
www.tmap.net
People you know. Knowledge you need55
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Agenda
Situation of TMap® Next
Business driven test management
Structured test process
Complete toolbox
Adaptivity
Conclusion
Questions
People you know. Knowledge you need56
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
What does TMap® Next offer?
Ready to go tool set
Tips and examples
Insight and support to tailor your solution
Find as many defects as possible as early as possible
Focus on risk!
Keep testing off the critical path
Testing is more cost-effective and efficient
Coverage without overlap
Repeatable process
Test organisation prepared and efficient
Industry-standard methodology
People you know. Knowledge you need57
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Questions?
People you know. Knowledge you need58
© Copyright Sogeti 2009 All Rights Reserved - Proprietary and Confidential
Thank you for your attention!
Sogeti BeLux nv
Prins Boudewijnlaan 53B-2650 Edegem
Tel.: +32 (0)3 231 12 90
Sogeti BeLux nv
Jules Bordetlaan 160B-1140 Brussel
Tel.: +32 (0)2 538 92 92
www.sogeti.be
www.tmap.net [email protected]
Technical manager Testing Services