It’s Not Just It’s also Talking about Testing · • SysTest Labs will be scheduling a free...

Anymore, It’s Not Just Testing, It’s also Talking about Testing

SQuAD PresentationJef Knutson, SysTest Labs, Incorporated

13 April 2010

Why is it ‘Talking about Testing’ Now?Why is it Talking about Testing Now?

• Anecdotes about Testing and CommunicationsAnecdotes about Testing and Communications

• Communications Changes in SW Test / QA

• Practical Specifics about Test Communications

– Test Planning / Scheduling / Executiong / g /

– Test Results Reporting

Speeding up Time Intensive Communications– Speeding up Time‐Intensive Communications

• 5 Communication Rules for SW Test / QA

2(C) 2010 SysTest Labs

Typical Manual Test Schedule / UpdateTypical Manual Test Schedule / Update

• Product Tests 1 ‐ 56(Done Mon 7 June)

• Portal Tests 200 ‐ 226(Done Wed 9 June)

• Catalog Tests 57 ‐ 68(Done Tue 8 June)

I t T t 69 112

• Integration Tests (ALL)(Done Mon 7 June)

D f t R i T t (ALL)• Inventory Tests 69 ‐ 112(Done Wed 9 June)

• Supplier Tests 113 ‐150

• Defect Repair Tests (ALL)(Done Fri 11 June)

• Inventory Reports A ‐ GSupplier Tests 113 150(Done Tue 8 June)

• Ordering Tests 151‐199

Inventory Reports A G(Done Thu 10 June)

• Admin Tests UR 1 – UR 12(Done Thu 10 June) (Done Fri 11 June)


Better Manual Test Schedule / UpdateBetter Manual Test Schedule / Update

• Infrastructure TestsDone Wed 9 June)

• Test Cycle Wrap‐up Tests Done Fri 11 June

(Integration, Portal) • Supply Chain Tests

(Defect Repair, Admin)

Done Wed 9 June (Supplier, Inventory, Reports)

• User Order TestsDone Thu 10 June ( C O )(Product, Catalog, Ordering)


Typical Defect Status ListTypical Defect Status List

261. Order Accessory Dropdown BoxNot working [3] OPEN

266. Product Order Radiobutton FailsChoice fails on 1 item [2] OPEN

h262. Inventory Entry Report ABMissing 2 listings [2] CLOSED

263. Catalog Effective Date Control

267. Missing shipping options ORDERNo Overnight [2] CLOSED

268. Customer Shipping Address LostgDate is End of Year [2] OPEN

264. User Security‐Customer MonitorAccess is View Only [2] OPEN

pp gUpdates not saved [2] CLOSED

269. Customer Comments too shortAllow 500 char [3] CLOSEDAccess is View Only [2] OPEN

265. Portal Access – Upload FailsProcess starts, but hangs; no

Allow 500 char [3] CLOSED

270. Order –SELECT button is erraticNeed multiple mouse clicks to

data into database [2] OPEN send form to server [3] OPEN


Better Defect Status ListBetter Defect Status List

Portal Access – OKUpload still fails, but won’t be used this month

ADMIN – OKCustomer – Monitor still at VIEW‐ONLY; will need to supportused this month

Inventory Reports ‐ OKAll OPEN defects were CLOSED;OPEN catalog bug narrow scope

ONLY; will need to support customers via Help Desk

OPEN catalog bug narrow scope

Ordering Process – OK w/ RISKSOverall, the process works;

?

some product‐related controls and buttons still not operational, COMMENTS, SHIPPING OPTIONS and ADDRESS defect are CLOSED


Typical Cumulative Test Results ReportTypical Cumulative Test Results Report


Better Cumulative Test Results ReportBetter Cumulative Test Results Report


Communications Changes in SW Test / QACommunications Changes in SW Test / QA

• Challenge: Communications need to speed up andChallenge: Communications need to speed up andimprove despite that projects are trending toward fewer testers possibly without leads ormanagersfewer testers, possibly without leads or managers.

• Response: To manage role change, SW Testers can:

– Recognize that communications is a core skill

– Understand communications at various levelsUnderstand communications at various levels

– Build communications skills where gaps exist


Recognize Communications is a Core SkillRecognize Communications is a Core Skill

E l iExpect less timeto prepare for communicationscommunications, but to present the same content more conciselyand achieving easier bettereasier, better comprehension


Communications at Various LevelsCommunications at Various Levels

• Manager : What andWhenManager : What and Whenanswers respond to project Budget, Schedule questions

WHAT &WHEN

• Lead: Where, Why and How answers respond to logistics

WHEN

WHERE and coordination questions

• Tester : How answers respond

WHERE& WHY

pto test technique and software design properties questions

HOW


Update resumes w/ communications skillsUpdate resumes w/ communications skills

• SysTest Labs will be scheduling a free workshop to explore this topic in more detailexplore this topic in more detail.

• If you are interested, email Jef Knutson atjknutson@systest [email protected] date and time.

• Limited seating available; if you plan to attend,Limited seating available; if you plan to attend, please notify us within the next week.


Practical Specifics ‐ Test CommunicationsPractical Specifics Test Communications

• Challenge: Even as funds decrease and SDLC / TestChallenge: Even as funds decrease and SDLC / Test processes change, small teams or even single testers still need to understand / manage / improvecommunications at a Test / QA group‐level

• Responses: Testers will need to address:

– SW Test / QA Logistics (Signaling Status)

– Time‐Intensive Communications (e g Requirements)Time Intensive Communications (e.g., Requirements)


Communications Need: SW Test / QA LogisticsCommunications Need: SW Test / QA Logistics

• Challenge: Test efforts with fixed scope are easy to g p ypresent, but communicating ‘on‐the‐fly’ changes in scope, schedule and risks is much harder.

‘ h fl ’• Response: ‘On‐the‐fly’ communication requires prior planning for flexibility, audience ‘frame of reference’: 1 Framing test scope as core and supplemental test cases1. Framing test scope as core and supplemental test cases

2. Organize schedule based on ‘prune‐able’ test cases

3. Presenting test‐related risks in terms of work results3. Presenting test related risks in terms of work results

4. Performing tests in a “watch‐over‐my‐shoulder” mode

5. Reporting test progress via ‘on‐demand’ dashboards


Frame Test Scope: Core / Supplemental TestsFrame Test Scope: Core / Supplemental Tests

• IDEA: Tests progress from simple complete work tasksIDEA: Tests progress from simple complete work tasks to increasingly complex complete work tasks.

• Strategy: Tests clarify the level of work productivityStrategy: Tests clarify the level of work productivity that the current ‘build’ will support, if deployed– CORE: Simplest complete ‘pieces of work’ do‐ablep p p

– Supplemental: Complete ‘pieces of work’ that involve increasing levels of user choice and/or task length

• Benefit: Lets the Tester report PASS / FAIL results in a way that ‘real‐world’ deployment implications are

dil d d b M d Creadily understood by Managers and Customers16(C) 2010 SysTest Labs

Organize Schedule via ‘Prune‐able’ TestsOrganize Schedule via Prune able Tests

• IDEA: Prune test starting with most complex workIDEA: Prune test starting with most complex work tasks, progressing to simplest complete work tasks.

• Strategy: Preserve test value in a shortened scheduleStrategy: Preserve test value in a shortened schedule by completing highest number of available tests.– Caveat 1: If highest value tests are also most complex, have g p

management numerically prioritize tests for execution

– Caveat 2: If necessary, prepare (in advance) other pruning i h h M l h d i dstrategies; have the Manager select the strategy desired

• Benefit: Lets Tester present schedule changes in a way that Managers Customers can recognize and approvethat Managers, Customers can recognize and approve


Present Testing Risks as Work ImpactsPresent Testing Risks as Work Impacts

• IDEA: Don’t cite risks for untested features; do identifyIDEA: Don t cite risks for untested features; do identify risks to work progress if untested features should fail

• Strategy: Citing work‐related impacts is more readilyStrategy: Citing work related impacts is more readily understood by managers; is also much easer to defend– Caveat: For this method to work, Tester and/or Business

Analyst must clearly grasp how system features relate to types and volumes of work that will be done via system

fi i k i h bl• Benefit: Lets Tester state risks in a way that enables Managers and Customers to take ‘ownership’ of both risk decision and related outcome consequencesrisk decision and related outcome consequences


Test Risks as Work Impacts: Sample ScaleTest Risks as Work Impacts: Sample Scale

If a feature isn’t tested and fails:If a feature isn t tested and fails:1. High Risk: [ specific tasks ] will be stopped until feature is

restored; undone tasks will accumulate and may be lost if ; y[ specific time period ] elapses without correction

2. Medium Risk: [ specific tasks ] will be delayed by [ % ] until feature is restored; undone tasks will accumulate or occur at [ low %] rate via [ alternate path ] until correction

3. Low Risk: [ specific tasks ] will be delayed [ time period ] until feature is restored; tasks can occur at [ high % ] productivity via [ alternate path(s) ] indefinitelyproductivity via [ alternate path(s) ] indefinitely.


Perform Tests: ‘Watch‐over‐my‐shoulder’Perform Tests: Watch over my shoulder

• IDEA: Tests are open for monitoring, if desiredIDEA: Tests are open for monitoring, if desired

• Strategy: Fosters Manager and Customer engagement; allows ‘outsiders’ to follow current developmentsallows outsiders to follow current developments– Caveat 1: Pertains to ‘traditional’ and ‘hybrid’ SDLCs than

to Agile or JAD processes that encourage involvement

– Caveat 2: Can sometimes create more headaches than benefits; people who monitor decide to become ‘Critics’

• Benefit: Intangible, yet can be substantial; creates an opening for skeptical individuals to assess how much testing adds value particularly for specific test casestesting adds value, particularly for specific test cases


Report Test Progress via DashboardsReport Test Progress via Dashboards

• IDEA: Report results on‐line with Summary View and p ya ‘drill‐down’ capability for further details, if desired

• Strategy: Continuously visible results help Testers avoid interruptions to write reports that may not be carefully read and may only have a ’shelf‐life’ of hours

Caveat: Sure Quality Center (QC) does this for $$$;– Caveat: Sure, Quality Center (QC) does this for $$$; most testers don’t know that QC by itself isn’t adequate

– DIY Option: What’s surprising is that real‐time dashboards to delivery are possible in Excel VBA, with a minimal effort

• Benefit: Lets Tester give simple, accurate, rapid status; lets Managers and Customers to recheck ‘on demand’lets Managers and Customers to recheck on‐demand


Dashboard Design: Information “Stages”Dashboard Design: Information Stages

• Answer the question: “Tell me more about this Test here…”Answer the question: Tell me more about this Test here…– Dashboard gives the top‐level, overall status view

– Dashboard links to Graphs show more details on key topics

– Graph links to Summary Tables enables in‐depth views, if needed


Communications Risk: Time-Intensive ProcessCommunications Risk: Time Intensive Process

• Challenge: Within SW Test / QA work, time‐intensiveChallenge: Within SW Test / QA work, time intensive communication processes are now unaffordable

• Response: SW Test / QA must avoid time‐intensiveResponse: SW Test / QA must avoid time intensive processes by using higher‐bandwidth approaches

– Example 1: When creating tests (existing system)Example 1: When creating tests (existing system), reviewing project artifacts is time‐intensive; high‐bandwidth tactic ‐ video capture test cases

– Example 2: When reporting on system test progress, generating and reviewing tables is time‐intensive; high‐bandwidth tactic – diagram test case sequences


High‐bandwidth – Video Test Case CaptureHigh bandwidth Video Test Case Capture

Step 14:S d d bSet dropdown box value to HOLD

Step 15:Set radio button

• If review is needed document screen shots; for

to Ship Overnight

• If review is needed, document screen shots; for test cases, identify controls and available choices.

• If review is NOT needed just replicate video steps• If review is NOT needed, just replicate video steps.


High‐bandwidth – Diagram Test SequencesHigh bandwidth Diagram Test Sequences

• Number Sequence is too abstract; reviewers can’t follow

• Diagram ‘paints a picture’ – provides needed context


5 Communication Rules ‐ SW Test / QA5 Communication Rules SW Test / QA

1. “Contextualize”: Paint a picture (but not by numbers)1. Contextualize : Paint a picture (but not by numbers)

2. Frame of Reference: Put it in audience’s terms

3. Short: Enough said

4. Think about diagrams: Applies 1, 2 and 3 above

5. Inform the decision: Don’t give book reports!


‘Talking about Testing’ in a NutshellTalking about Testing in a Nutshell

• Do you have Test Communications anecdotes?

• Communications is a Core SW Test / QA Skill

Be practical i i ti b t T ti• Be practical in communicating about Testing

– Know how to handle WHAT, WHY and WHERE

– Know how to signal status of SW Test / QA

– Find / Speed up Time‐Intensive Communications/ p p

• Use 5 Communication Rules for SW Test / QA


Questions / Answers / DiscussionQuestions / Answers / Discussion

Thanks for coming!Thanks for coming!


It’s Not Just It’s also Talking about Testing · • SysTest Labs will be scheduling a free...

Documents

Transcript of It’s Not Just It’s also Talking about Testing · • SysTest Labs will be scheduling a free...