Improve Quality Services B.V. - Process Improvement for Present … · 2016-04-04 · Improve...

Process Improvement for Present and Future

Erik van Veenendaal

Test Improvement with TMMiTest Improvement with TMMi

Improve Quality Services BV 2

Increasing size and complexityAmount of software in consumer products doubles every 24 months (Philips)Number of requirements for mobile phones doubles every 6 months (Nokia)

# Defects hardly decreasesDefect density per KLOC is almost constant in the last 10 years (Les Hatton)

Deadlines, Competition & OutsourcingTesting often takes 30 - 40% of project costs

Computable, Sept.’08

ChallengesChallenges


PeoplePeople

Test Test ProcessProcess

Infrastru

cture

Infrastru

cture

Test Improvement DirectionsTest Improvement Directions


People SkillsPeople Skills

Test Test knowledgeknowledge-- Test principlesTest principles

-- TechniquesTechniques

-- Tools, etc.Tools, etc.

IT IT knowledgeknowledge-- Software developmentSoftware development

-- Requirements Requirements (IREB)(IREB)

-- Configuration mgt.Configuration mgt.

DomainDomainknowledgeknowledge-- Business processBusiness process

-- User characteristicsUser characteristics

Soft skillsSoft skills-- CommunicationCommunication-- Critical mindsetCritical mindset-- Presentation & reportingPresentation & reporting

via (exchanging) via (exchanging) Practical Practical ExperiencesExperiences, , CoachingCoachingand (formal) and (formal) TrainingTraining


CCMMMM((II)) MMaattuurriittyy

lleevveell

DDeessiiggnn FFaauullttss // KKLLOOCC

((KKeeeennee))

DDeelliivveerreedd DDeeffeeccttss // FFPP

((JJoonneess))

SShhiippppeedd DDeeffeeccttss // KKLLOOCC

((KKrraassnneerr))

RReellaattiivvee DDeeffeecctt DDeennssiittyy

((WWiilllliiaammss))

SShhiippppeedd DDeeffeeccttss

((RRiiffkkiinn))

55 00,,55 00,,0055 00,,55 00,,0055 11

44 11 00,,1144 22,,55 00,,11 55

33 22 00,,2277 33,,55 00,,22 77

22 33 00,,4444 66 00,,44 1122

11 55--66 00,,7755 3300 11,,00 6611

Trends in Software Quality Trends in Software Quality


Readiness for Test ImprovementReadiness for Test Improvement

Some fundamental Some fundamental practicespractices availableavailableRequirementsRequirements documented and documented and managedmanagedProjectProject planning existsplanning existsConfigurationConfiguration managementmanagement

ManagementManagement ““needsneeds”” / supports better testing/ supports better testingQualityQuality is important enoughis important enough

A budget for process improvementA budget for process improvement

or reverse quality pushor reverse quality push…………..!!..!!


TMMi modelTMMi model

Test Maturity Model (TMM) FrameworkTest Maturity Model (TMM) FrameworkIlinois Institute of Technology Ilinois Institute of Technology -- CMM basedCMM based

TMMi SourcesTMMi SourcesCMMI, ISTQB, TMM, IEEE, TPICMMI, ISTQB, TMM, IEEE, TPI

Starting pointsStarting pointsStaged representation, CMMI structureStaged representation, CMMI structureIndependent or in combination with CMMIIndependent or in combination with CMMI

TMMi released January 2008TMMi released January 2008

BB

AA


TMMi process areas by levelTMMi process areas by level5 : Optimization5 : Optimization-- Test process optimizationTest process optimization-- Quality controlQuality control-- Defect preventionDefect prevention

3 : Integration3 : Integration-- Software test organizationSoftware test organization-- Test training programTest training program-- Test life cycle and integrationTest life cycle and integration-- NonNon--Functional testingFunctional testing-- Peer ReviewsPeer Reviews

2 : Managed2 : Managed-- Test policy and strategyTest policy and strategy-- Test planningTest planning-- Test monitoring and controlTest monitoring and control-- Test design and executionTest design and execution-- Test environmentTest environment

±± 2 years2 years

±± 1 year1 year

Helps to set priorities and focus the tpi processHelps to set priorities Helps to set priorities and focus the tpi processand focus the tpi process

4 : Management and measurement4 : Management and measurement-- Software quality evaluationSoftware quality evaluation-- Test measurement programmeTest measurement programme-- Advanced reviewsAdvanced reviews


CertificationCertification

Requirements defined regardingRequirements defined regardingAssessment process Assessment process (TAMAR) based(TAMAR) based on ARC (B/C) on ARC (B/C) and ISO 15504and ISO 15504LeadLead--Assessor Assessor andand Assessor Assessor

OrganizationsOrganizations and persons and persons to beto be accredited by the accredited by the TMMi FoundationTMMi FoundationRegister of (lead) assessorsRegister of (lead) assessorsFormal TMMi Certification possibleFormal TMMi Certification possible


Lead Assessor / AssessorLead Assessor / Assessor

Training Training 10 days practical experience10 days practical experience

Training Training 20 days practical experience20 days practical experience

AssessmentsAssessments

TrainingTrainingPractical experiencePractical experience

TrainingTrainingPractical experiencePractical experience

TMMiTMMi

> 1 years> 1 years> 2 years> 2 years(2 years SPI = 1 year TPI)(2 years SPI = 1 year TPI)

Test Process Test Process ImprovementImprovement

> 3 years> 3 yearsMultiple levels & domainsMultiple levels & domainsISTQB FoundationISTQB Foundation

> 5 years> 5 yearsMultiple levels & domainsMultiple levels & domainsISTQB AdvancedISTQB Advanced

TestingTesting

AssessorAssessor(informal assessments)(informal assessments)

Lead assessorLead assessor(formal assessments)(formal assessments)


Level 2 : ManagedLevel 2 : Managed

Test Policy and StrategyTest Policy and StrategyTest PlanningTest PlanningTest Monitoring and ControlTest Monitoring and ControlTest Design and ExecutionTest Design and ExecutionTest EnvironmentTest EnvironmentPr

ojec

t orie

nted

proc

ess a

reas

Ensures CMMI complianceEnsures CMMI complianceVerification & ValidationVerification & Validation

TMMi level 2Lead time

reduction !!

TMMi level 2TMMi level 2Lead timeLead time

reduction !!reduction !!


to provide visibility regarding the quality and outstanding riskto provide visibility regarding the quality and outstanding risks of software productss of software products

to contribute to the software product quality by finding defectsto contribute to the software product quality by finding defects

to contribute to customer satifaction regarding the released softo contribute to customer satifaction regarding the released software productstware products

……..

Test Policy 20XX

.

Mission / Ambition

Definition Testing

Test targets

Employees

Organization

Test process

Customers

Test Process ImprovementTest Process Improvement-- objectives, performance indicators, modelobjectives, performance indicators, model

based onbusiness policy

based onbusiness policy


Example Example PerformancePerformance IndicatorIndicator

0

510

15

20

2530

35

40

w eek 1-10 w eek 11-23 w eek 24-36

Development.percentage of TOTAL FAT.percentage of TOTALUAT.Percentage of TOTAL Production.Percentage of TOTAL

““this is what we are doing !!this is what we are doing !!””

Improvement focusImprovement focus


14

Level 3 : IntegrationLevel 3 : Integration

Test OrganizationTest OrganizationTest Training ProgramTest Training ProgramTest Life cycle and IntegrationTest Life cycle and IntegrationNonNon--Functional TestingFunctional TestingPeer ReviewsPeer Reviews

orga

nisa

tiona

lpr

oces

s are

as

reqreq gdgd dddd implimpl testtest operoper

InstitutionalizationInstitutionalization


Life cycleLife cycle

Req.

CodeDesign

TestTest planTest design

Test script

Execution


Test ImprovementsTest Improvements

Testing as a profession Testing as a profession Clear responsibilitiesClear responsibilitiesTest plan(ning) Test plan(ning) RiskRisk--based testing based testing Testing part of development life cycleTesting part of development life cycleEarly involvement Early involvement Test design Test design techniquestechniquesControlled Controlled testtest processprocess

Risk Matrix ISR 2.2

0

8

15

0 25 50probability

Impa

ct

Q/RUS/MR

ECGS/RVasc

USVWLicPtrac

LogSrchImpdMerge

DBDCMlibDbkill

Vwregr

Top PriorityTop Priority

engineeringview

PRISMA®


Reported ResultsReported Results

0

5

10

15

20

Y1 Y2 Y3 Y4

alpha / beta test lead time

0

20

40

60

80

100

120

140

160

% deviation - spent vs. planned

50556065707580859095

100

Y1 Y2 Y3

Defect Detection Percentage

TMMi 1

TMMi 2

TMMi 3

Employee satisfaction


““Getting StartedGetting Started””

Test policyTest policyManagement commitmentManagement commitment

Improvement is a projectImprovement is a projectResources and effort percentageResources and effort percentage

Maturity of development organizationMaturity of development organization

1

2

3


““Getting the job doneGetting the job done””

Set both long term and short term goalsSet both long term and short term goalsUse what is availableUse what is availableDonDon’’t make an external consultant t make an external consultant responsibleresponsibleInstitutionalize the improvementsInstitutionalize the improvementsChange management skillsChange management skillsReview against the overall objectives Review against the overall objectives (policy) almost continuously(policy) almost continuously


Test Improvement ManifestoTest Improvement Manifesto

•• FlexibilityFlexibility over Detailed Processes

•• Best PracticesBest Practices over Templates

•• Deployment orientationDeployment orientation over Process orientation

•• ReviewsReviews over Quality Assurance (departments)

•• Business drivenBusiness driven over Model driven

Making TMMi work Making TMMi work ……..


TMMi Supporting OrganizationsTMMi Supporting Organizations

More information on TMMiavailable at www.TMMiFoundation.org

Thank You !!Thank You !!

Improve Quality Services B.V. - Process Improvement for Present … · 2016-04-04 · Improve...

Documents

Transcript of Improve Quality Services B.V. - Process Improvement for Present … · 2016-04-04 · Improve...