Post on 13-Apr-2018
Building on SuccessBeyond the obvious; Practices for “Agile Testing”
1
Erik van Veenendaal
www.erikvanveendaal.nl
Erik van VeenendaalErik van Veenendaal
� Founder and major shareholder ImproveQS� In testing since 1989 working for many different
clients and in many different roles� Author “TMap”, “The Little TMMi” and
many other books and papers
www. erikvanveenendaal.nl
Improve Quality Services BV 2
many other books and papers� Vice-President International Software Testing
Qualifications Board (ISTQB) 2005 - 2008� Vice-Chair TMMi Foundation� Keynote speaker, e.g., EuroSTAR, STAReast� Winner of the European Testing Excellence
Award (2007)
ImproveImprove QualityQuality Services BVServices BV
�Service organisation in the area ofTesting, Requirements Engineering and Quality Management
�Consultancy, Subcontracting and Training
www. improveqs.nl
Improve Quality Services BV 3
�Consultancy, Subcontracting and Training
SW Process ImprovementQuality AssuranceIT-AuditingRequirements Engineering& management (IREB)
Testing (TMap, TMMi)Test Process ImprovementCertification (ISTQB)- incl. Advanced !!Inspections / Reviews
CommonCommon TestingTesting PracticesPractices
At least 50% of the organizations:� No Test Policy / Strategy� Less usefull Test Plans� No Test Design techniques� Few apply Formal Reviews
The Answers“We do Exploratory Testing …….”� Few apply Formal Reviews
� Test Automation not successful� Re-usable testware hard to find� Testers not trained for the job�Most of us are still at (TMMi) level 1
Improve Quality Services B.V. 4
4
“We do Exploratory Testing …….”“We do Agile Testing ………..”“We do SCRUM ……………..”
“We do Lean and Mean SW development”
The ParadoxThe Paradox
� We still release our systems (most of the times) .....− ‘A little bit too late’− ‘A few too many defects’− ‘Budgets are slightly overrun’
For many yearsFor many years only only 2525% of the IT projects % of the IT projects
are are successfullsuccessfull
− ‘Budgets are slightly overrun’
� How to be ‘good enough’ and ‘survive’
� A tool box for agility in testing?!
Improve Quality Services B.V. 5
What is agile testing?What is agile testing?
� Testing in an immature (test) organzation− The great excuse !!
� Testing in an agile development project� Testing in an agile development project− Re-shaping traditional practices
� Testing following the statutes of the agile manifesto
− A different mind set to start from
Improve Quality Services B.V. 6
AgileAgile ManifestoManifesto
� Individuals & Interactions over processes and tools
� Working Software over comprehensivedocumentation
�� Customer Collaboration over contract negotiation� Responding to Change over following a plan
� Agile Values (Kent Beck)− Communication, Feedback, − Simplicity, Courage
Improve Quality Services B.V. 7
Core Practice # 1: Risk BasedCore Practice # 1: Risk Based
Setting Clear Test Priorities
CommunicationCommunicationCommunicationCommunicationCommunicationCommunicationCommunicationCommunication, , , , , , , , Customer CollaborationCustomer CollaborationCustomer CollaborationCustomer CollaborationCustomer CollaborationCustomer CollaborationCustomer CollaborationCustomer Collaboration
Improve Quality Services B.V. 8
� Risk identification and analysis� Determine a risk-based test approach� Risk based reporting (stakeholders’ language)
Testing is Risk Based - Hetzel
Working software & Responding to changeWorking software & Responding to changeWorking software & Responding to changeWorking software & Responding to changeWorking software & Responding to changeWorking software & Responding to changeWorking software & Responding to changeWorking software & Responding to change
Setting Priorities: Risk AssessmentSetting Priorities: Risk Assessment
�Risk identification− Requirements based (e.g. PRISMA)− Brainstorm workshops (e.g. FMEA)− Faliure history− Faliure history
�Risk analysis− Risk = impact x likelihood
�What is the impact for the business ?●damage, usage intensity, external visibility, ….
�What is the likelihood that there are defects ?●new technology, complexity, interfacing, ….
Improve Quality Services B.V. 9
Product Risk MatrixProduct Risk Matrix
IIIIII
H
xx
x
Must TestCould Test
X1
MoSCoW priorities
Improve Quality Services B.V. 10
IVIVIIIIIIL
M
L HM
x
x
x
Likelihood
Impact
Should Test“Won’t Test”
X2
RiskRisk--BasedBased Test Test ApproachApproach
IIIIII
HH
MM
xxx
Must TestCould TestTest design techniquesReview designSupport module testsReview module testsFull re-test – 2 iterations
Exploratory TestingSupport module testsReview module testsRe-test defect only
Improve Quality Services BV 11
IVIVIIIIII
LL
MM
LL HMM
x
x
xLikelihood
Impact
Should Test“Won’t Test” Test design techniquesReview designFull re-test
Exploratory TestingRe-test defect only
Agile “One Page Test Plan”
Must Test….. Test Approach …..
Should test…… Test Approach …..
should includea definition of “done”
Improve Quality Services B.V. 12
…… Test Approach …..
Could Test….. Test Approach …..
Would Test….. Test Approach …..A 3 hour session
to achieve a one pageMaster Test Plan
RiskRisk--BasedBased ReportingReporting ((exampleexample))
TS1 TS2 TS3 TS4 TS5 TS6 TS7 TS8
Risk item 1
Risk item 2
Improve Quality Services B.V. 13
Risk item 2
Risk item 3
Risk item 4
Risk item 5
CanCan we release the product?we release the product?
TS1 TS2 TS3 TS4 TS5 TS6 TS7 TS8
Risk item 1
Risk item 2
Management view
Improve Quality Services B.V. 14
Risk item 2
Risk item 3
Risk item 4
Risk item 5
Core Practice # 3: ReviewsCore Practice # 3: Reviews
Upstream Focused Reviews
Testing is preventing defects - Hetzel
CommunicationCommunicationCommunicationCommunicationCommunicationCommunicationCommunicationCommunication & Feedback& Feedback& Feedback& Feedback& Feedback& Feedback& Feedback& Feedback
Improve Quality Services B.V. 15
� Priority to the profitable� Apply review practices that make
the difference
CommunicationCommunicationCommunicationCommunicationCommunicationCommunicationCommunicationCommunication & Feedback& Feedback& Feedback& Feedback& Feedback& Feedback& Feedback& FeedbackCustomer collaborationCustomer collaborationCustomer collaborationCustomer collaborationCustomer collaborationCustomer collaborationCustomer collaborationCustomer collaboration
Why? and What?Why? and What?
� 60% of the defects have already been made before coding/implementation has started
� Upstream defects are the most important− for costs, but also for project success− for costs, but also for project success− requirements, architecture documents
� Focus on the high risk items − you usually can’t afford anything else− a ‘risk based’ review plan helps enormously
�Limited number, but thorough reviews
Improve Quality Services B.V. 16
Don’t fool yourself ....Don’t fool yourself ....
� Different review types for different objectives (e.g. Walkthrough / Informal Review / Inspection)
� Trained moderators
− ‘moderator du jour’ does not work− ‘moderator du jour’ does not work
� Apply roles and kick-off
� Entry check
� Not too many pages (± 20) or lines of code
� Reasonable checking rate (± 10)Improve Quality Services B.V. 17
Core Practice # 3: Unit TestingCore Practice # 3: Unit Testing
Perform Real Unit Testing
WorkingWorkingWorkingWorkingWorkingWorkingWorkingWorking software over software over software over software over software over software over software over software over
Improve Quality Services B.V. 18
� Buddy Testing (XP)� Different from debugging� Pair Inspection (or Static Analysis)
Testing requires independence - Hetzel
WorkingWorkingWorkingWorkingWorkingWorkingWorkingWorking software over software over software over software over software over software over software over software over comprehensivecomprehensivecomprehensivecomprehensivecomprehensivecomprehensivecomprehensivecomprehensive documentationdocumentationdocumentationdocumentationdocumentationdocumentationdocumentationdocumentation
Unit Test Unit Test Process (IEEE 1008)Process (IEEE 1008)
PlanCheck
coverageCheckresults
Runtests
Writetests
Unit test plan- technique- environment- exit criteria- level of independence
Will often refer to:Project Unit Test Plan
Unit test spec.- test input- expected result- test objective
May act as:automatedtest Script
Unit test log- test cases- test results- test coverage- tester- environment- etc
Improve Quality Services B.V. 19
Practical Practical ImplicationsImplications
� Buddy (independent) testing� Buy and install a coverage tool� Use formal exit criteria
Remember: most high level testing will only achieve 30 - 40% code coverage
− Risk based / Statement and Decision Coverage (be realistic!!)
� Free format test designs (code)� Developers are testing and it is fun!!� Higher quality delivered to the testers
Improve Quality Services B.V. 20
Core Practice # 4: Test DesignCore Practice # 4: Test Design
Use Informal Techniques Formally
RespondingRespondingRespondingRespondingRespondingRespondingRespondingResponding to to to to to to to to changechangechangechangechangechangechangechange, , , , , , , , CustomerCustomerCustomerCustomerCustomerCustomerCustomerCustomercollaborationcollaborationcollaborationcollaborationcollaborationcollaborationcollaborationcollaboration and No and No and No and No and No and No and No and No comphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensive doc’sdoc’sdoc’sdoc’sdoc’sdoc’sdoc’sdoc’s
21
� Test Use Cases
� Exploratory Testing
� Classification Tree Method
collaborationcollaborationcollaborationcollaborationcollaborationcollaborationcollaborationcollaboration and No and No and No and No and No and No and No and No comphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensivecomphrehensive doc’sdoc’sdoc’sdoc’sdoc’sdoc’sdoc’sdoc’s
Testing is finding defects - Myers
Improve Quality Services BV 21
Exploratory TestingExploratory Testing
� Simultaneous exploration, test design and execution� Finding the most important defects in the time available
Let yourself be Let yourself be distracteddistracted
‘cause you never ‘cause you never knowknowwhat what you’ll find ….’you’ll find ….’
newnew
PRODUCT
tests
2222
distracteddistracted what what you’ll find ….’you’ll find ….’
But periodically take But periodically take stockstockof of your status against your missionyour status against your mission
test ideastest ideas
newnew
test ideastest ideas
Improve Quality Services B.V. 22
Exploratory Testing ProcessExploratory Testing Process
Test Plan
Risk Matrix
Keep track of
bug/fix ratio !!
To be reviewed with
business & developmentCharter
Sessions
- exploration
- design
- execution
Debriefing
notes
Charter
HeuristicsList of
commondefects
Improve Quality Services B.V. 23
Are you really doing ET or just EG?Are you really doing ET or just EG?
� Are you using test charters?� Are stakeholders involved in producing
testcharters?� Is there a standard list of ideas / common defects?� Do you work in pairs?
Illustration only
� Do you work in pairs?� Is the maximum length of a test session 1 day?� Are test logs available from the test sessions?� Are new risk, ideas and issues documented?� Are test experiences discussed on a daily basis
during debriefing sessions?
Improve Quality Services B.V. 24
Core Practice # 5: PeopleCore Practice # 5: People
Build Experienced & Skilled Testers
IndividualsIndividualsIndividualsIndividualsIndividualsIndividualsIndividualsIndividuals and and and and and and and and interactionsinteractionsinteractionsinteractionsinteractionsinteractionsinteractionsinteractionsover over over over over over over over processesprocessesprocessesprocessesprocessesprocessesprocessesprocesses and toolsand toolsand toolsand toolsand toolsand toolsand toolsand tools
25
� Training (-on-the-job)
� Exchanging Experiences
� Test Team Dynamics
� Not only for Testers !!
over over over over over over over over processesprocessesprocessesprocessesprocessesprocessesprocessesprocesses and toolsand toolsand toolsand toolsand toolsand toolsand toolsand tools
Testing is an extremely complex and
intellectually challenging task - Myers
Improve Quality Services BV 25
Testing SkillsTesting Skills
Test knowledge- Test principles
- Techniques
IT knowledge- Software development
- Requirements (IREB)
via (exchanging) via (exchanging) Practical Practical ExperiencesExperiences, , CoachingCoachingand (formal) and (formal) TrainingTraining
Improve Quality Services BV 26
- Techniques
- Tools, etc.
- Requirements (IREB)
- Configuration mgt.
Domainknowledge- Business process
- User characteristics
Soft skills- Communication
- Critical mindset
- Presentation & reporting
Test Team DynamicsTest Team Dynamics
Effective teams don’t just happen!�Correct experience and skills�Correct attitude�Common purpose�Common purpose�Team management�The right team mixture, e.g. Belbin profiles
Balanced teams are more effectiveNobody is perfect but a team can be ……..Improve Quality Services B.V. 27
SummarySummary
�Focus ‘agile testing’ on …−Risk-Based Test Approach & Reporting−Upstream Focused Formal Reviews−Apply Real Unit Testing−Use Informal Techniques Formally−Build Experienced & Skilled Testers
�This will make you survive in almost any project ….
Improve Quality Services B.V. 28