Testing Unexplained: The Importance of Tacit Knowledge in Testing and How to Develop it - James Bach
-
Upload
ho-chi-minh-city-software-testing-club -
Category
Presentations & Public Speaking
-
view
1.048 -
download
3
Transcript of Testing Unexplained: The Importance of Tacit Knowledge in Testing and How to Develop it - James Bach
7/14/2014
1
Testing Unexplained (The Importance of Tacit Knowledge in Testing and How to Develop It)
James Bach @jamesmarcusbach [email protected]
Consulting Software Testing, Satisfice, Inc.
Tacit and Explicit Knowledge
EXPLICIT means it can be represented completely in the form of a string of bits: words, pictures, even actions can be explicit. (software is explicit)
TACIT means it is not manifested in a form that can be equated to a string of bits: it is unspoken, unwritten, unpictured. • Relational Tacit Knowledge is tacit by convenience. • Somatic Tacit Knowledge is tacit in your body. • Social Tacit Knowledge is tacit in a community. (see Collins, Tacit and Explicit Knowledge)
7/14/2014
2
Examples of the Need for Tacit Knowledge
• You consider how users will interact with and adjust to a product.
• You consider what a specification and product was intended to say or do, not just what they literally say and do.
• You notice what the specification and product strangely omits.
• You focus on business risks even when no one tells you what they are.
• You notice changes in the product’s behavior over time—none of which are
failures in and of themselves—you form conjectures of why that happens, and you connect those conjectures with an evolving
understanding of plausible error and failure patterns.
• You use tools, you notice when those tools misbehave, and you make
adjustments or workarounds to get the job done.
• You report your test results based partly on how you think your clients will react. You anticipate their questions based on your insight about
how they will understand you.
“2+2=“
7/14/2014
3
“2+2=“
What is the Point? You have MANY expectations about how a calculator should work or might work…
(you acquire them automatically)
…but you aren’t aware of many of them...
(they are encoded or generated in your mind, but not in words or pictures)
…and that means no explicit test procedure can ever duplicate the value of tacit knowledge of a skilled human tester.
We need a process that respects tacit knowledge.
7/14/2014
4
Fact Gathering
Result Checking
Tester Thinking
(theorizing about meaning
of results)
Tester Thinking
(failure modeling and test design)
Tester Thinking
Tester Thinking
Tester Thinking
Tester Thinking
Tester Thinking
Tester Thinking
Tester Thinking Tester
Thinking
Tester Thinking
Tester Thinking
Tester Thinking
Tester Thinking
Tester Thinking
Tester Thinking
If testing is an intellectual process that requires deep mental
engagement, personal motivation, and constant learning to do well,
how do we manage the situation in order to foster that?
7/14/2014
5
Managing Tacit Knowledge
10
Heuristics bring useful structure to problem-solving skill.
• adjective:
“serving to discover.”
• noun:
“a fallible method for solving a problem or
making a decision.”
“The engineering method is the use of heuristics
to cause the best change in a poorly understood situation within the available resources.”
-- Billy Vaughan Koen, Discussion of The Method
7/14/2014
6
11
-Idea -Idea
…
Heuristics They are applied, not followed.
1. Do this 2. Then do this 3. Then do this 4. Then do this 5. And then this…
This…
…not this.
The skilled tester remains in control of
the process.
“2+2=“
7/14/2014
7
Tacit Explicit O
ther
Peo
ple
Te
ster
Your Feelings &
Mental Models
Shared Artifacts (specs, tools, etc.)
Stakeholders’ Feelings &
Mental Models
Inference
Observable Consistencies
Reference Conference
Experience
Oracles From the Inside Out
Consistency (“this agrees with that”) an important theme in oracles
• Familiarity: The system is not consistent with the pattern of any familiar problem.
• Explainability: The system is consistent with our ability to explain it.
• World: The system is consistent with objects and states, in the world, that it represents.
• History: The present version of the system is consistent with past versions of it.
• Image: The system is consistent with an image that the organization wants to project.
• Comparable Products: The system is consistent with comparable systems.
• Claims: The system is consistent with what important people say it’s supposed to be.
• Users’ Expectations: The system is consistent with what users want.
• Product: Each element of the system is consistent with comparable elements in the same system.
• Purpose: The system is consistent with its purposes, both explicit and implicit.
• Statutes & Standards: The system is consistent with applicable laws and standards.
7/14/2014
8
A General Statement to Protect Role of Tacit Knowledge/Skill
A system testing event in Malmo, Sweden. Maria Kedemo, manager, is serving food.
7/14/2014
10
Three Roles of Testing
Test Leads/Managers
Cultivate and support
Helpers
Cultivate and supervise
Heuristics and Tools
Use
Use
Clients and Context
Respond to
Respond to
“I am ABLE to make decisions that hurt people (or me).”
“I want to AVOID hurting people (including me).”
“I KNOW how my decisions might hurt people (or me).”