Agile Testing, Risk, Uncertainty, and Why It All Works(the updated for AGILEEE, almost all new edition)

Elisabeth HendricksonQuality Tree Software, Inc.www.qualitytree.comesh@qualitytree.com

Last updated September 22, 2011Copyright © 2011 Elisabeth Hendrickson, Quality Tree Software,

Inc.

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

What Does Agile Really Mean?

Agile software teams…

…Deliver value in the form of releasable software at frequent regular intervals (at least monthly)…

…At a sustainable pace…

…While adapting to the changing needs of the business.

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Source: Wikimedia Commons, courtesy of Ray KurzweilSee http://www.kurzweilai.net/the-law-of-accelerating-returns

Increasing Complexity

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Speculation BuildupS

pecu

lati

on

RISKRISK

AnalyzeDesign Implement StabilizeBig BangRelease

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c. Intention

s

Actual Need Implementation

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Speculation, Lack of Alignment, and Risk

Intentions

Implementation

Actual Need

?

??

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Agile eliminates speculation buildup…

Iteration

Iteration

IterationIteration

Iteration

Specu

lati

on

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

…only if we eliminate the speculation

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c. “frAgile”

Specu

lati

on

RISKRISK

Iterations…

Stabilize

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Every. Single. Time.

EmpiricalEvidence

speculation

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

The organizations best positioned to succeed

are the ones that

LEARN

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Build

MeasureLearn

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

How do we check alignment?

Intentions

Implementation

Actual Need

Get out of the building!

Steve Blank, author of Four Steps to the Epiphany

?

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

How do we check alignment?

Begin with the End in Mind. Capture examples

with expectations. Automate checking of

expectations. Explore to discover risks.

Intentions

Implementation

Actual Need

?

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

The Letter and Spirit of a Story

Explicit Expectations

Implicit Expectations

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Check that all previously met expectations are still met even as the

solution grows in capability and complexity.

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

How do we check alignment?

Intentions

Implementation

Actual Need

Demo. Rehearse delivery. Deliver frequently. Explore with the

customer or proxy.

?

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Reducing Feedback Latency

Latency

Develop

Demo

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

What Does Agile Really Mean?

Agile software teams…

…Deliver value in the form of releasable software at frequent regular intervals (at least monthly)…

…At a sustainable pace…

…While adapting to the changing needs of the business.

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Intentions

Implementation

Actual Need

✔

✔✔

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

(OK, now let’s talk about “testing.”)

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Checking Alignment

Intentions

Implementation

Actual Need

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Every. Single. Time.

speculation

EmpiricalEvidence

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

The Whole Team Succeeds or Fails TOGETHER

“Hey! There’s a hole in your side of the

boat.”

“Hey! There’s a hole in your side of the

boat.”

There is no “Us” and “Them.”

There is only Us.

As Alistair Cockburn says:

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Nine Key Technical Testing-Related Agile Practices

ATDD TDDExploratory

Testing

Collective Ownership

Automated Unit Tests

Automated System Tests

Continuous Integration

RehearseDelivery

Automated Deploy

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Initiative

Test-First Mindset: Begin with the End in Mind

Code

StrategyHow will I know my efforts have the result I intended?

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Acceptance-Test Driven Development (ATDD)

User Story Workshop

Implementation

Agile-Friendly Test Automation Framework

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Continuous Integration (CI)

CI tools do automated builds, execute tests, and report the results

Developers practicing CI merge their changes locally & execute tests before checking in

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Version ALL Technical Assets with the Code

Unit Tests

Production Code

Source Control Repository

(Plus All other

technical assets)

Test “Fixture”

Code

Acceptance Tests

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Increase Visibility

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Automated Deploy

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Fail Early, Fail Fast

Failing early & fast gives us time

to fix the problems.

Local Unit Tests Secs-Mins

MinsCI Build

Mins-Hrs

Automated Regression

Hrs-Days

Exploration

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Red Build Means Stop the Line

If a previously passing expectation fails, there’s a bug. Bugs slow everything down. To keep

sustainable pace, fix bugs fast.

We can just throw that bug on the pile

with the others.

Yuck.

But that will increase

technical debt & slow velocity.

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Exploratory Testing

Simultaneously…

…learning about the software

…designing tests

…executing tests

using feedback from the last test to inform the next

(See Jon and James Bach’s work on Session-Based ET)

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Done Includes Tested

Implemented+ Checked+ Explored== DONE

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

Every. Single. Time.

speculation

EmpiricalEvidence

Copyri

ght

© 2

01

1 E

lisabeth

Hendri

ckso

n,

Qualit

y T

ree S

oft

ware

, In

c.

LEARNTEST

Intentions

Implementation

Actual Need