Apply automation tests effectively in agile development teams
Using flow approaches to effectively manage agile testing at the enterprise level
49
Testing Flow Using flow approaches to effectively manage agile testing at the enterpri se level Yuval Yeret – anban Lead - AgileSparks #LSSC1 1 /CC @yuval yer et
-
Upload
yuval-yeret -
Category
Technology
-
view
2.124 -
download
1
description
Slides from my upcoming LSSC11 talk:More and more organizations want to become more agile these days. When the theory hits the shores of reality, few organizations can get to an idealistic agile feature team that does all testing within sprints, has no need for release-level processes, and where everything is fully automated continuous deployment style. Usually the testing organization is in the eye of the storm when talking about Big Batches, Wastes, ineffective handoffs and mountains of rework, as well as high transaction costs. I’ve recently been using Lean/Kanban flow based approaches to provide a way to evolve testing organizations to a more effective way of working, so that they can better support earlier feedback and higher flexibility. I will present this work as well as case studies from enterprise-level product development companies that are starting to use these approaches. We will deal with the following challenges:* Complex environments when it is not realistic to finish all required work within a sprint * How to visualize and reduce testing batch sizes within sprints/releases using CFD * How to deal with the testing bottleneck so common in product development organizations – practical suggestions and how to deal with the mindset issues * How to run stabilization/hardening periods using Flow-based thinkinghttp://lssc11.crowdvine.com/talks/18074
Transcript of Using flow approaches to effectively manage agile testing at the enterprise level
Testing FlowUsing flow approaches to effectively
manage agile testing at the enterprise
levelYuval Yeret – Kanban Lead - AgileSparks
#LSSC11 /CC
@yuvalyeret
Repeat after me:VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE
UE-VAL == YUVAL
What we covered TestingElasticity
Why Flow for Testing
Test-specificChallenges
To Flow
Issues at Scale
Distress call!
“We need to release much more frequently”“the business cannot live with months of
hardening”
We want Agile to save us!
Sufficient?
Not for Enterprise
scale
Necessary?
Not all of it
Pragmatic?
So so
So why not just do Scrum+XP?
DONE DONE – The Enterprise Testing Challenge
“We are doing agile iterations for development and testing and it is going quite well. One of our greatest challenges is dealing with the stuff we cannot finish within the iteration – currently its one big pile at the end”
R&D GMLarge Enterprise
Software Group
Longer Sprints
SprintSynchronize
dHandoffs
Flow-based Handoffs
What are the alternatives?
DONEDONE DONE
READY
Flowing all the way to DONE DONE
9
ComplementSolutions
FocusPragmati
c Pace
So... We started to use a recipe based on Flow
Current Reality Tree
• Let’s dive into why this is a problem
LOW PREDICTABILITY
QUALITY @ THE
END
LATE FEEDBACK
HIGH STABILIZATION COSTS
HEAVY MANUAL TESTING
BIG BATCHES
BIG FEATURES
LONG CYCLE TIMES
LARGE WIP
FEATURE PER
PERSON
SPECIALIZAT-ION
LONE WOLFS
LIMIT OPEN
DEFECTS
SMALLER FEATURES
AUTOMATE AS PART OF
DONE
CONTINUOUS INTEGRATION
DONE DONE
LIMIT FEATURES IN PROGRESS
COLLECTIVE OWNERSHIP
FIX M/L DEFECTS?
NOT EFFECTIVE
FEATURE FREEZE
POLICY TOO STRICT
LOSE SIGHT OF THE BIG PICTURE
LARGE OVERHEAD
HIGH COST
OUR BUILD IS
NOT STABLE
EXISTING GAP IS TOO
BIG
IMPROVEMENT BACKLOG
TRANSITION BASED ON NECESSITY
EFFECTIVE SLICING
WHOLE TEAM
AUTOMATION PYRAMID
INVEST IN REMOVING INSTABILITY
PRAGMATIC RISK
PROFILING
START AS IS, SHORTEN
WHEN PREDICTABLE
http://www.flickr.com/photos/librarygoblin/4307610546/lightbox/
http://blog.crisp.se/henrikkniberg
LIMIT AT THE FEATURE
LEVEL
Is this Evolution or Revolution? k/K?
Perfo
rman
ce
Time
Distress Calls + Israeli Impatience?
Evolution
This is what we see in Israel...
Lets see how it looks like in practice
“What is the gap between dev and test? What are we doing to close it?”
Limit Defects in Process – Early Quality is Cheap Quality
BAD GOOD
“We used to feel comfortable to
release quarterly and anxious to
release every two weeks. “ “Now releasing
every two weeks is natural. We are anxious when we can’t do it [due to holiday freeze]”FiftyOne.com
Always go for Continuous Deployment???Not necessarily
What we covered TestingElasticity
Test-specificChallenges
To Flow
Hard
en
ing
Limiting WIP – pull according to the bottleneck
R D C TTime
R D C TR D C T
R D C TR D C T
R D CDONE
WIP Limits SLACK TIME ???* No coaches were idle due to the preparation of this presentation
Hard
en
ing
R D C TR D C T
R D C TR D C T
R D C TR D C T
DONECoding Done
Feature Freeze
Identify
Exploit
Subordinate
to
Elevate
Prevent
Inertia
What LIMITED WIP Does
DD
D
TT
T
3
How can we help current work?
Help us with
Blocker
Fix open defects
on current work
Help us automate tests for
current work
WIP Limit!Can’t start
new DEV work!
What LIMITED WIP Does
DD
D
TT
T
3
How can I help you be more
efficient?
Ways to share the test load –
e.g. automation/set
ups work
Automate Setups and Test DataHalf of our work
is not core test work. Maybe you can take some of it, or
help us reduce waste there
Identify
Exploit
Subordinate toElevate
Prevent Inertia
Improve quality you deliver to
QA!
Testers
DevelopersDevelopers
Where Kanban is used, Whole Team, ATDD and other Agile Testing approaches emerge MUCH faster
problem solved?
“Testing was always like a pillow – it took whatever space was left. We used to have a cap on testing because there was a deadline...
...Now with the new Flow mode the inmates are running the asylum [and we have no cap on how long testing takes]”
A worried Dev Manager
Testing/Verification is a highly
Elastic activity
Lets look at a few ways to cope ...
Cadence of
releases/drops, will
drive completion
Have an explicit policy regarding the desired/acceptable size of work
Care about Cycle Time & Done
TestingEnd to End
Due Dates can help(Especially if doable and agreed upon with delivery team)
http://yuvalyeret.com/2010/09/19/kanban-early-warning-using-a-predictive-variant-of-spc/
http://yuvalyeret.com/2010/09/19/kanban-early-warning-using-a-predictive-variant-of-spc/
No REAL external Due Date?try Internal Cycle Time Soft Goals based on learned capabilities
Avoid local due dates!
Try...
In-Flight
Control Chart
Can testing be
overdone?
Talk about what is Done
Establish Explicit Policies
Experiment with using an Explicit Policies board documenting your
process
Different work types / CoS can
have different
Done/Release Criteria
Goals+Visibility enhance Purpose
3
2
1
What we covered TestingElasticity
Issues at Scale
Will visualizing and managing flow in
each of those roads be enough?
Need to visualize and manage the global end to end flow across shared resources
Sharing
Manage flow at the level at which resources are shared
3
Example Policy - Classes of Service for Downstream Involvement
Risk Profiling Involvement mode of the shared resource
Classes of Treatment
What we covered TestingElasticity
Questions ?
[email protected]@yuvalyeret
Get the slides at http://
www.slideshare.net/yyeretBlogging at
http://yuvalyeret.c
om
