Validation Missteps Making Us Full Time Firefighters
21
-
Upload
dvclub -
Category
Technology
-
view
139 -
download
2
Transcript of Validation Missteps Making Us Full Time Firefighters
Presented to Silicon Valley DVClub January 26th, 2010 Allison Goodman, [email protected]
My Background
Products that survived the fires to market (some minus a few features)
Products were consumed by fires late in development
4
SSD
3
What missteps along the way turn us into firefighters?
Insufficient Planning Not Designing for Test Not creating and integrating
Feedback Loops Lack of Transparency
Insufficient Planning- Don’t have what you need to test Original Test Coverage Not Enough
Undocumented Assumptions
Missed Dependencies
SCOPE
Scope
Lack of Coverage
If you don’t plan for it…it will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
Plan! (in writing!)
Take the time to write it all down -who does what (owners!) -how does a feature really work -what “done” is -checkpoints along the way -success criteria
Keep track of the assumptions
And what everyone really needs
test
Not Designing for Test Very difficult to repeat tests for debug Waste time
My Design is Perfect No plan tests- late implementation
Pass? Fail? Who knows?
Getting lucky vs knowing it works
Getting test to fail again
If you don’t design for test, you won’t find or replicate issues … they will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
Design for Testing!
Hooks & Bread Crumbs • Where you are (state) • How you got there • How to get back
Ability to Artificially inject fails
Figure out coverage
Define Pass & Fail with Team Partner with Architects & Design, you are helpful!
tests?
Not creating & integrating Feedback Loops-
Planned Testing
New Market / Customer Requirements
Planned Testing Stale
Testing
Hardware Software Firmware Marketing
Working in Silos Implementation != Documentation
If you don’t create and implement feedback loops, you’ll miss issues…they will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
Create & integrate Feedback Loops
Validation Team
Tight & Ongoing Feedback -Problems found on current product -new info about system interactions -actual implementation changes
Hardware Software Firmware Marketing
Planned Testing
New Market Requirements
Additional Testing
Adaptable to new requirements/usages
Is it Effective? Efficient?
Change it!
Run Test
Lack of Transparency
Team telling mgmt…or worse each other, it’s all fine
What you feel is really happening
DONE!
Schedules run late, validation is squeezed
Fewer iterations, narrower
coverage, less thorough
investigations
If your team is not transparent, you’ll ignore risks and coverage gaps…escapes will surprise you.
Every surprise will end up as a fire …
As you get closer to trying to ship a product you become a full time firefighter.
Foster a Transparent Team
Everyone plays a role…
Don’t declare DONE until it is really there…. Don’t punish for calling it out
Tell the real story, Encourage others to do the same
Stop the Firefighting!
Plan Design for Test Create & Integrate Feedback Loops Foster a Transparent Team
