StarWest 2012 - Agile Defect Management: Focus On Prevention
29
Agile Defect Management Focus On Prevention David Jellison Director, Quality Engineering [email protected] Twitter: davidjellison
-
Upload
david-jellison -
Category
Technology
-
view
1.869 -
download
3
description
Efficient agile organizations focus on defect prevention rather than downstream defect discovery, because discovering defects during or after testing adds to development costs. Delaying discovery and repair of defects can make an agile team feel like they are operating in a mini-waterfall. Sharing his experience with Scrum/ Kanban teams, Leveraging the lean concept of limiting work in progress, David Jellison describes how grouping defects into two major categories—work-in-progress defects and escaping defects—reduces development costs and improves reliability in the field. Dave illustrates how to manage problem discovery early and minimize the existence of escaping defects. Treating escaping defects as the exception rather than the norm results in a much smaller defect backlog and increased customer satisfaction. This approach encourages increased collaboration between quality engineers and developers, and shifts the focus of team measures from defect counts to product delivery velocity and cycle time, with increased confidence in quality as the work is completed.
Transcript of StarWest 2012 - Agile Defect Management: Focus On Prevention
Agile Defect ManagementFocus On Prevention
David JellisonDirector, Quality Engineering
[email protected]: davidjellison
Twitter: davidjellison
You Will Learn Categorizing defects drives defect context
management Testing is part of delivering a story or feature Detecting and resolving defects as the work is
delivered prevents backlog Attaching an in-progress defect to a current
story encapsulates the defect in the current work
Escaping defects become the exception rather than the norm
2
Twitter: davidjellison
The One Finding The Most Bugs Wins!
Lord of the Rings: Legolas & Gimli counting kills
Downstream Testing Approach
Twitter: davidjellison
Preventing Bugs is Craftsmanship
upstream Testing Approach
Twitter: davidjellison
Quest for Zero Defects You can only manage what is known
Crosby’s “Zero Defect” standard manages to requirements (http://en.wikipedia.org/wiki/Zero_Defects)
Agile changes the game, managing business value to the customer frequently Need a pattern to manage defects that we
inevitably discover during and after release
Never really reach 0 Defects
5
Defect Discovery
Moving Defect Discovery Earlier
6
Twitter: davidjellison
Traditional Defect Discovery Trends
7
Twitter: davidjellison
Late Defect Discovery Costs Opportunity cost
Not working on other things Developer Context switching
Time to get acclimated and then back again Rebuild and test time
More cycles to deliver the fix Damage control
Support time and reputation Other hidden costs…
8
Twitter: davidjellison
Traditional Defect Discovery Trends
Defect debt accumulation
Delay in resolution
Defects resolved
Dev/QE aligned
9
Twitter: davidjellison
Defect Discovery Improvement Plan Stop defect debt accumulation
Move defect discovery up in the development cycle Embed QE into teams with Dev/QE alignment Prevent defects through design review & test
automation Write failing regression tests as defects are discovered
Test in sandbox Don’t wait for final integration environment to test Continuous build and continuous integration patterns
Whole-team responsibility for quality Clear acceptance criteria for all new work Everyone tests Automated test code reviews with developer
10
Defect Prevention
Automating Along The Way
11
Twitter: davidjellison
Defect Discovery Late
Downstream test teams focus on defect discovery
12
Twitter: davidjellison
Defect Discovery Early
Embedded test teams focus on defect prevention
Regression tests assure no system failures
13
Twitter: davidjellison
Agile Practices Smaller
cycles Collaboration Design
reviews Test early and
often Test Driven
Development Continuous
Integration Automated
Regression
Scrum
Kanban
14
Twitter: davidjellison
Automated Regression Tests
UnattendedTest Runs
AvailableTest Time
15
Defect Categorization
Introducing WIP Defects
16
Twitter: davidjellison
Categorizing Defects Escaping Defects
Discovered defects allowed into the field Undiscovered defects found in the field
17
Twitter: davidjellison
Categorizing Defects WIP (work in progress) Defects
Defects discovered in new code not yet delivered to field
Becomes additional acceptance criteria for delivery
Treat as subtasks of feature work Caged by the parent work in progress
18
Twitter: davidjellison
Managing Defect Discovery Its OK to not write
defects if there are failing automated tests to manage
Promote failing tests to WIP defects if need to manage separately
Promote WIP defects that are still failing at time of delivery to escaping defects and added to the backlog
19
Focus on Prevention Develop failing
regression tests when discovered manually
Continuous feedback through failing test run reports
Associate WIP defects with feature work in progress
20
Managing Escaping Defects Delivering business
value over no defects
No unknown failures in regression tests
Coordinate planned escaping defects with customer support
Reduce escaping defects to less than you can count on two hands21
Twitter: davidjellison
Managing Escaping Defects
“Jellison Jazz Hands”
Twitter: davidjellison
Driving escaping defects down
23 Number s for illustration only
Twitter: davidjellison
Escaping Defect Reduction Plan Set expectation in the teams that fixing WIP
defects is part of the acceptance criteria Add escaping defects in the field to the
backlog and rank along with other work Plan to fix some number of escaping defects
from the backlog in each development cycle Treat escaping defects as the exception rather
than the norm
24
Examples
Using WIP Defects
25
Twitter: davidjellison
Example: WIP Defects/Escaping Defects Shifting the focus to WIP defects results in far
fewer escaping defects in the field, even with increased discovery
26
WIP Defects [blue]Escaping Defects [red]
Twitter: davidjellison
Example: WIP Defects/Escaping Defects Each team may be in a different place with
business risk and can manage escaping defects differently
WIP Defects [ red & blue]Escaping Defects [green]
27
Develop a theme strategy
Twitter: davidjellison
What We Covered Categorizing defects drives defect context
management Testing is part of delivering a story or feature Detecting and resolving defects as the work is
delivered prevents backlog Attaching an in-progress defect to a current
story encapsulates the defect in the current work
Escaping defects become the exception rather than the norm
28
Twitter: davidjellison
Contact: David JellisonDir, QEConstant Contact, Inc.1601 Trapelo RoadWaltham, MA [email protected]: 339-222-5712
Blog: http://davidjellison.wordpress.com Twitter: davidjellison http://ConstantContact.com/Careers
29
