Test Plan Documentation Strategies - Mind Maps, Test Scripts, and More
61
Documenting Software Testing Instructions - A Survey of Successful Approaches STP Con October 25th, 2011 Justin Hunter
-
Upload
justin-hunter -
Category
Technology
-
view
1.914 -
download
0
description
Summary from the STP Con Testing Conference: Documenting Tester instructions – a Survey of Successful approaches Track: Test Process Justin hunter, CEO, Hexawise This session provides an overview of several different approaches that are successfully used by testers to address the questions of: What type of testing documentation should you create; and how much detail is appropriate to include in order to guide testers and/or satisfy compliance requirements? This session will discuss pros and cons of several approaches, including: • Checklists • Mindmaps • Test ideas • Specification by example • Detailed test scripts In some software testing presentations, the speaker will explain why they think one particular approach is particularly attractive. This won’t be one of those talks. Justin is a test design specialist who has enjoyed teaching testers how to improve the efficiency and effectiveness of their test case selection and documentation approaches.
Transcript of Test Plan Documentation Strategies - Mind Maps, Test Scripts, and More
Documenting Software Testing Instructions - A Survey of
Successful Approaches
STP Con October 25th, 2011Justin Hunter
Topics
Tradeoffs
Considerations
Documents
Idea Triggers
Context Objectives
Mind Maps“Straw Man”
ChecklistsPersonas
“Kan Ban Cards”
Combinations
Topics
3
Context is Critical
4
Tester’s experience
level / Exploratory
Testing skills
Importance of keeping detailed records (e.g., for regulatory compliance or lawsuit defense)
Context is Critical
5
Tester’s experience
level / Exploratory
Testing skills
Importance of keeping detailed records (e.g., for regulatory compliance or lawsuit defense)
Relatively littledetail
Enough detail to satisfy lawyers
Very detailed
Enough detail to avoid tester confusion
Appropriate Amt. of Detail
6Sources: Key Terms (“Cloud Level” / “Clam Level”, etc.)
http://alistair.cockburn.us/Use+casesImage: accelerateddeliveryplatform.com
Objectives
7
Objectives & Assumptions
8
Who? W
hen?
Prescriptive vs. Exploratory?
Tradeoffs
9
Practical Tradeoffs
10
ManyFew
Reqs Docs and the SUT
Reqs Docs Only
Test ideas from
looking at...
Detailsin tests
Practical Tradeoffs
11
ManyFew
Reqs Docs and the SUT
Reqs Docs Only
Test ideas from
looking at...
Detailsin tests
Prescriptive /Designed Before
Exploratory /Designed During
Documentation Strategy
12
“Straw Man”
13
Every test scriptMUST have:
• Test Name• Test Number• Tester’s Name • Tester’s Dept.• Type of Test • Requirement Number• Pre-conditions• Configuration Details• Test Data• Specific Steps• Designer• Approver• Execution Time (est.)• Expected Results
Time Management
14
Overly-Detailed Documentation
15
ActualTesting
Mind Maps
16
Mind Maps - Test Plan Template
17http://www.softwaretestingclub.com/forum/topics/the-two-
page-test-plan?commentId=751045%3AComment%3A105353
18http://www.softwaretestingclub.com/forum/topics/the-two-page-
test-plan?commentId=751045%3AComment%3A105353
Clear Communication Example
Evolution of a Mind Map
19
“1. Identify Properties you're interested in”
http://testerkiwi.blogspot.com/2011/04/building-exploratory-test-plan-redux.html
Evolution of a Mind Map
20
“2. Start Collecting Intel”
http://testerkiwi.blogspot.com/2011/04/building-exploratory-test-plan-redux.html
Evolution of a Mind Map
21
3. After Talking to Developer...
http://testerkiwi.blogspot.com/2011/04/building-exploratory-test-plan-redux.html
Mind Maps
22
Benefits of mind maps so far... Benefits of mind maps so far...
Creation
Quick and Easy Maintenance
Quick and Easy Collaboration
Quick and Easy
Reporting
Quick and Easy
But would mind maps be detailed enough...
... For junior testers?
... For testers who will join team in six months?
“MindMapping 101 for Testers”
23
http://www.bettertesting.co.uk/content/?p=956
Test Details into Test Mgmt Tool
24
25
Collaboration is Easy
Planning & Prioritizing Risks
26
http://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg
27
http://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg
Determining Scope - What’s In
28
http://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg
Determining Scope - What’s Out
Documentation
29
Kan Ban Boards
30
31
http://www.flickr.com/photos/roxocubes/5786672702/
Paul Holland - (Testing-Focused)
32
Paul HollandAlcatel-Lucent
(personal photo of histesting board, used with permission)
Paul Holland - (Testing-Focused)
33
Each note represents a half-day of testing activity (via Session-Based Testing)
Highly recommended reading: For a detailed description of Paul’s experiences with this approach and his lessons learned, check out Michael
Bolton’s “A Sticky Situation” in an upcoming version of STP Magazine. Also, Google “Session-based Test Management”
Kan Ban-ish Tool: Pivotal Tracker
34
More Details on Hover-Overs
35
More Detail Still by Clicking
36
Including Recent Test Results
37
Test Descriptions / Size Estimates
38
39
Soap Opera Testing
40
http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf
Tests should be fun and aggressive.
Write scenarios that are:
• Reality-based• Exaggerated, and • Condensed
Soap Opera Testing
41
http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf
42
http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf
Soap Opera Testing
Soap Opera Example
43
http://www.logigear.com/logi_media_dir/Documents/Soap_Opera_Testing.pdf
Personas and “Substruction”
44
http://www.software-testing.com.au/blog/2006/07/30/personas-substruction-and-other-trades-tricks/
Behavior-Driven Development
45
Behavior-Driven Development
46
47
How to Select?
48
49
... How to Handle More Details?
50
... and still MORE Details?
How to Document?
51
Test inputs from all 3
mind maps are entered.
jklhlkjhkjhkjh
52
Pairwise coverage in 19 highly-varied tests
Auto-populate Mad Libs-like tests
53
Turned into Detailed Test Scripts
54
Detailed test scripts: quickly generated by Hexawise.
Checklists
55
Checklists
56
• Cem Kaner - The Value of Checklists and the Danger of
Scripts: What Legal Training Suggests for Testers http://
www.kaner.com/pdfs/ValueOfChecklists.pdf
• Justin Hunter - http://hexawise.wordpress.com/2009/11/04/
checklists-good-test-scripts-bad/
• Thoughts from the Test Eye - Software Quality
Characteristics
• Michael Hunter, “You Are Not Done Yet”
• Elizabeth Hendrickson - Testing Cheat Sheet
Software Quality Characteristics
57
http://thetesteye.com/blog/2010/11/software-quality-characteristics-1-0/
Rapid Reporter (Free Tool)
58
http://testing.gershon.info/reporter/
Checklists
59
www.kaner.com/pdfs/ValueOfChecklists.pdf
“Easy-Access Executive Summary”
60
http://bitly.com/bundles/hexawise/2
An organized summary of all the links referred to in this presentation is
available in the link below:
Invitation to SlideShare Viewers
61
https://app.hexawise.com/signup-1-5
If you have read this far, odds are very good that you’re the kind of tester who is interested in trying out promising new testing tools.
Hexawise is a powerful test design tool that allows testers to zero in on a small set of unusually-powerful tests. We’d like you to try it and let us
know what you think about it.
We have an unusual - some have even said bizarre - pricing policy. While enterprise licenses of Hexawise start at $50,000 / year, we make fully-featured versions of Hexawise available at no cost for teams of 1 to 5
testers. It’s like having a rich uncle with a $50,0000 BMW who gave you the keys to his car indefinitely. And he wouldn’t get upset even if you kept it or ruined it. Wouldn’t you at least want to take it out for a test drive?
Sign up now
