Agile Testing in aFormal Organisation
Egbert Bouman & Robin Esman
Twitter: @egbertbouman
Valori & De Nederlandsche Bank
About us
Robin Esman
• 20+ years in IT, 9 years at DNB
• Test community lead
• Business Analist
Egbert Bouman
• Valori
• Author SmarTEST
• Certified risk auditor
• Test automation consultant
• Agile test coach & trainer
Our message today
YES…
… formal organisations do agile
• Because it’s mainstream now
• And rightly so: agility is good for everyone
BUT…
… this poses extra challenges
• Be aware of that challenges
• And adopt your agile strategy accordingly
SO…
… take advantage of our story
• With seven best practices we’ve selected in retrospective
• They worked for us, and may work for you
3
In the context of better testing!
DNB INTRO
In
Behind Closed Doors
Collectors value >> Gold value Collectors value >> Gold value
DNB Working on trust
6
Het Agile Manifesto
77
1. Our highest priority is to satisfy the
customer with early and continuous
delivery of valuable software.
2. Welcome changing requirements, even
late in development.
3. Deliver working software frequently,
from a couple of weeks to a couple of
months.
4. Business people and developers must
work together daily.
5. Build teams with motivated
individuals. Give them the
environment and support, and trust
them.
6. Most efficient and effective is face-
to-face conversation.
7. Working software is the primary
measure of progress.
8. Promote sustainable development. Agile
teams should be able to maintain a
constant pace indefinitely.
9. Continuous attention to technical
excellence and good design.
10.Simplicity: the art of maximizing the
amount of work not done.
11. The best results emerge from self-
organizing teams.
12.Regularly, the team reflects on how
to become more effective, then adjusts
its behavior accordingly.
The 12
‘A
gile Principles’
How to merge agile into
a formal organisation’s DNA?
9
vs
Risk
aversion
& control
How did we
address the
challenge?
We implemented a lot of bright
(and not so bright) idea’s
10
And selected the top-7 for You
Our advice: 7 ‘agile test essentials’
11
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
Beware of scrumdamentalism
Maximise agility, but stick to ‘good old’ best test practices
• Risk analysis
• Refinement: critical thinking, ‘what if’
• Exceptions, rainy days, negative testing
• Sound defect management
• Overall test management
These and other proven practices are OK
• Don’t treat them with contempt
• Beware of short sighted labels like SINO, AINO, Zombie Scrum, …
12
But be clear about the tough choices
More trust, less ‘governance’
More exploratory, less detail in advance
More collaboration, less toll gates
13
Our advice: 7 ‘agile test essentials’
14
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
Started with the
‘Agile Growth Accelerator’
and Agile Scorecard
MaakMaakBuild
Architecture
Six improvement
areas
Testing still the bottleneck
Gartner:
“Testing repeatedly emerges as the most formidable constraint
impacting velocity”
Forrester:
“As organizations adopt (…) Agile and DevOps, testing has
become the biggest constraint impacting quality and speed ”
16
Continued with the
‘Test Growth Accelerator’,
and ‘SmarTEST scorecard
Seven improvement areas:
1.Agile test strategy
2.Scope, risks and test goals
3.Test design & execution
4.People & culture
5.Test environments
6.Automated testing
7.Planning & control
The SmarTEST scorecard in more detail
Best of breed ‘waterfall’ frameworks
• Test Process Scorecard (SmarTEST®)
• TPI (TMap®), CMMi, TMMi
Made agile
• Address all 12 ‘agile principles’
• Kniberg self assessment checklist
Fit for non-hardcore software companies
• Fully agile
• Respecting pre-agile proven practices
‘Challenged Self Assessment’
• Team decides upon improvement focus and required maturity level
• Supported and challenged!! by agile coach18
Real Growth AccelerationPresented 2016Q4 by the pilot teams themselves
Deposito guarantee system
WSS Wall Street System F&A Peoplesoft(no improvement data)
Cash Operations(late boarding)
CRM Financial Supervision
Scorecard process retrospective
Management happy
• Control
• Direction
• Clear & concise
Teams happy
• Challenging whilst helpful
• Freedom of choice
• ‘Do-able action items’
20
Our advice: 7 ‘agile test essentials’
21
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
DeveloperQuality & business aware
Tester(More or less) independent
CustomerFinal proof & verdict
Respect separation of duties: DTC model
No walls, only roles!!
Design&
Build
Operations
Developer Tester Customer
The test ‘Shift Left’
Unit & integration test (CI)
Acceptancetest
Independent functional (chain)test new & existing
(ST, SIT, FT)
Developer Tester
Unit Test
Customer
A succesfull shift left path
Test New & OldAcceptance
testWAS
Unit Test & CI
Test New
Test Old
AcceptancetestNOW
Unit test & CI
Test New
Test Old
Acctest
GOAL
Insufficient coverage, production incidents.
Beter coverage,testing still on critical path.
Faster, same coverage,less business capacity.
Our advice: 7 ‘agile test essentials’
25
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
WoW cards
Lean Way of Work (WoW!)
• Two-sided with wow-effect
• Kanban: Japanese for ‘Visual card’
Lean, accessible, visual
• Color & appearance
• No bulky documents
Keep them alive
• Proces en ownership
• Expiration date26
27
Visualize
Kanban = visual card!
WoW: make testing colourful again!
Our advice: 7 ‘agile test essentials’
28
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
Tool does the job
Faster, cheaper
Dependable, repeatable
Everybody does it
Demo impressed
me!
29
Top-down driver: bosses like it
Fiddling with tools
I like hightechstuff
Manual testingis boring
Automationis cool
Mymarketvalue
30
Bottom-up driver: boys will be boys (girls too)
Manage your Test Automation ‘just enough’!
Choose what to automate
• And what not to automate
Manage the top of the pyramid
• And choose ‘middle & top of the pyramid tools’
Let developers do it their way
• Trust & let go
• But be aware of ‘too many tools’
31
32
Our advice: 7 ‘agile test essentials’
33
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation tool landscape
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
Improve your improvement
Problem: continuous improvement is immature
Solution: agile working secures continuous improvement
Unfortunately it doesn’t work that way
34
What did we do?
We solved this problem by continuously organising ‘just do Scrum’
Stimulating, Challenging, Facilitating
Use guilds, audits, events, coaches => time to focus
Build a community
Keep looking for leverage opportunities (e.g. Cobit audit)
And make it fun!
35
Make it fun
Test community:40 members
‘pepernoten’ test‘broodje kroket’
Our advice: 7 ‘agile test essentials’
37
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
The Cobit
audit framework:
Overview
38
Still a popular framework!
DNB uses a sub-set
Level 3: formalise & proof
Level 4: continuous improvement
Mapping Cobit at the agile/scrum process
39
• Include non-functional acceptance criteria
• Like Security and Performance
• Atention for out-of-the-box risks
Refinement and Sprint planning
• Security and Performancetest done
• Fall back plan available
• Anticipated and demo ’ed with your DoD
Review(= productevaluation)
• Did we follow the proces agreed upon?
• Do we do a sound risk management job?
• Is testing sufficiently Independent (separation of duties)?
Retro(= process evaluation)
This
example:
AI7.6
‘Testing of
Changes’
40
So far our story
41
1. No scrumdamentalism
2. Use a framework for 'challenged self-assessment'
3. Respect separation of duties: DTC model
4. Make visual WoW Cards
5. Manage your test automation
6. Improve your improvement capacity
7. Take advantage from the next (Cobit) audit
43
Thank you
www.valori.nl
www.dnb.nl
Top Related