ALN DC Chapter MeetingMarch 2012

Agile Testing for Mere MortalsPresented by Dave Haeffner

ALL BOOKS OR SERVICES MENTIONED OR RECOMMENDED ARE COOL AND WORTH SHARING. DAVE IS NOT AFFILIATED WITH THEM IN ANY WAY

SHAPE OR FORM.

IF ANYONE HAS A MEDICAL CONDITION, OR IS ALLERGIC TO FIRE, DON'T SIT IN THE FRONT ROW.

AND PLEASE, FEEL FREE TO ASK QUESTIONS THROUGHOUT.

Show of hands -- How many of you are human beings?Keep them up. Ah, good we have our control.Keep your hand up if you've had experience with Agile Testing, QA… or Automated Web Testing.OK. Keep your hand up if the thought of it leaves a bad taste in your mouth.

Look around. You're not alone.

Thanks for coming out.

A little bit about me…While working at The Motley Fool, I accidentally ended up in this profession.I used to be in IT Operations -- a Systems Administrator.Through a twist of fate, I ended up trying out QA for a change of pace. It was supposed to be temporary…That was over 3 years ago…When I started out, I had a lot of questions. A big one was -- "Ummmm, what is Agile Testing?”

I'd like to share with you some of what I've learned since then. It should only take about 5 minutes.

And I'll deliver it in 3 parts so it's more digestable -- So I'll give you the text book version, the "street" version, and Automation.

I tend to leave Automation last because it's a good excuse to read you a quote. This one's from Randy Pausch's "The Last Lecture":

"No, I did not make it to the National Football League, but I probably got more from that dream and not accomplishing it than I got from any of the ones that I did accomplish. I had a coach, I signed up when I was nine years old. I was the smallest kid in the league, by far. And I had a coach, Jim Graham, who was six-foot-four, he had played linebacker at Penn State. He was just this hulk of a guy and he was old school. And I mean really old school. Like he thought the forward pass was a trick play. So, we showed up for practice the first day, and you know, there’s big hulking guy, and we were all scared to death of him. And he hadn’t brought any footballs. One of the other kids said, excuse me coach, but there’s no football. And Coach Graham said, right, how many men are on a football field at a time? Eleven on a team, twenty-two total. And Coach Graham said, all right, and how many people are touching the football at any given time? One of them. And he said, right, so we’re going to work on what those other twenty-one guys are doing. And that’s a really good story because it’s all about fundamentals. Fundamentals, fundamentals, fundamentals. You’ve got to get the fundamentals down because otherwise the fancy stuff isn’t going to work.”

Let's dig in.

Ah, Agile Testing -- The White Whale of the Agile World. Everyone knows it exists, but it can be hell on earth to catch it. If you're successful, the benefits are *massive*. If you fail, then your abilities as a team can be severely limited, even if you don't know it.

So let's unpack the idea of "Agile Testing" and focus on what it is, it's key elements, and the key players involved in it's delivery. For this, I'm going to consult "The Book"

Agile Testing -- what is it?

Here's a good definition -- "doing the simplest tests possible to verify that a piece of functionality exists or that the customers' quality standard (e.g. performance) has been met”. Sounds simple enough, but there's obviously more to it, or else you wouldn't all be here.

I look at that definition as the outcome you want. But you can't get there by dictating it from the top-down. It requires more... finesse.

Team

Crispin & Gregory advocate that you should put it to the team, letting them work together to come up with a "software quality strategy". They go on to abstract this concept further, saying you need to keep stoking that initial fire, and evolve it into a "Quality Philosophy" -- making software quality a part of the team's DNA -- or else your efforts will not be as effective.

! Team

While a main tenant of Agile Testing is a whole team focus, Crispin & Gregory really emphasize the importance of the "Tester" or "QA” role (they use both names interchangeably). Their primary responsibilities are varied, and include:* Seeing the big picture, looking at the application more from a user or customer point of view* Being an integral member of the customer team -- helping elicit requirements and examples, and helping the customers express their requirements as tests* Being embedded on the developer team* Should look for unique ways to facilitate communication

x^3

One tool for facilitating communication is The Power of Three, or (since I was just in Costa Rica) The Three Amigos. NEXT SLIDE

3 amigos

It means that all discussions about a feature need to have a Developer, a QA, and the Product Owner present. And it is the responsibility of each person to make sure that each group is properly represented.

Team Bridge

Value & Risk

So if you follow their model and have successfully ignited a passion for quality -- then you should have an autonomous whole team approach to quality and someone on the team working to help bridge the gap between tech & biz. If you take this and keep it simple, focusing on value for the business, then things start to get interesting.

(Here's a quote -- "while we have skills to identify test cases beyond the 'happy path' we still need to start by making sure the happy path works.")

Seems simple enough. Let's all do that.

While it sounds simple, it's not easy. Because let's be honest, Agile Testing is a lot like the classic 1981 video game "The Oregon Trail”

Think about it…

It's a long journey (takes time)You think you know where you're heading -- greener pastures and so on (everyone talks about it – a UTOPIA)You try to make it while keeping your family together (whole team)Only to be attacked by natives along the way (funding, politics, other priorities, technology woes)And you'll likely end up dying of dissentary (give up and stick with the status quo)

Let’s draw some parallels (above in parens)

But all joking aside, there are additional challenges that prevent Agile Testing from working. Here are some more subtle ones

When dealing with building out QA, you’ll find you need both a technical and analytical set of resources. Such skills exist in a single person, but it’s hard to find. And odds are, they won’t stay in the QA role for long. Unless they’re 6’2” with brown hair and has a coffee obsessions

An approach to work around this is, pick one, and leverage other resources to fill in the gaps. (e.g. what The Fool did) – makes staffing *a lot* easier

Need more QA’s and having trouble sourcing external candidates?

Higher from w/in – Customer Service is an excellent place

And why not? They know your product inside and out and interact with your customers on a regular basis

Throwing it over the wall *shudder*

The problem is that often times the feedback that QA’s give to developers is out of band with their work flowAnd there’s often not enough QA’s to go around, they are spread too thin, and not truly embedded on the team

The short game - use a CI serverThe long game – use a CI server and get the team using ATDD and well written step definitions (more on this later)

“The one thing worse than screwing up, is screwing up and thinking you’re doing well.”

– J.B. Rainsberger

In antithesis of this, Dev's treat QA Regression as a safety net w/o knowing what is actually covered, QA's do the same for unit tests

I say we get rid of the words "regression" and "smoke". I've found that they tend to mean different things to different people

Effectiveness = Quality * Acceptance

But also, you need to tie behavioral and process change back to valueThat way you can motivate people in the right direction w/o forcing it – autonomy will reign supreme

Start with why, then sort out the how and whatThis will help your effectiveness formula – have you heard of this formula?(Effectiveness = Quality * Acceptance)

QA as a gatekeeper & defects still getting out?

Try a Proper process

IDEATION Planning EXISTENCE

Story Discussion, Sprint Goals, Sizing, Tasking, and a dash of

Definition of Done (depending on the team)

Work’s done! Time to throw it over the wall to be “QA’d”

Pre

WIP QA Review Done Released

Stories reviewed , if issues found, kicked back to Devs

Happens more often that you think

Automated tests written here

Business Owner review, if they find an issue – it gets kicked back to Devs

Sometimes happens without QA knowing – causing duplicative effort and additional context switching for Devs

Often times, things get released that bypass the previous steps

This reinforces a passive acceptance of the inefficiencies of our process

It’s time for a change

IDEATION Planning EXISTENCE

Create Acceptance Tests

Becomes the Specification

Magically becomes an automated web test

Require rep’s. from each group (Biz Owner, Dev, QA, UXD)

Automated Acceptance Tests need to pass in order to proceed

Post

Shovel Ready

Can be done at the same time

Batches feedback for Dev

Expl. Testing done by QA (or someone that didn’t work on the story)

Review done by Stakeholder

WIP

Everyone can rest easy, because this software is high quality baby!

Exploratory Testing Review Done Released

This is *the* most important PART

My train of thought spawned from a tremendously good read by Gojko Adzic where he introduced me to the concept of ATDD

It solves so many problems and just makes so much sense.

Different dialects from team to team & btwn biz & tech? create a ubiquitous languageIf you get this right, then you win the game, you have completed the oregon trail and did not die of dysentary

Code

Commit

TestREDGREENREFACTOR(repeat)

Given <precondition(s)>When <action>

Then <result>

http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/

Automation frees you to do more exploratory testing!

Acceptance Test Driven Development (ATDD) or BDD

Test Driven Development (TDD)

Not ready to automate? That’s fine.If you start to capture new features in the gherkin syntax, you will be heading in the right direction.It will act as a communication and collaboration tool between your team and the business.And it will enable you to have a shared, ubiquitous language that business and tech can both agree on and understandIf you add automation later, then you get the retroactive benefit of already captured features.

Traditionally w/ automated web testing using a free tool like Selenium there were two paths

IDE -> big sweet -> slow -> brittle -> false positives

IDE -> RC -> abstraction, logic, smarter -> more technical (read “custom testing harness”)

And the big complaint among Devs is that it’s too slow – they call it full stack testing since it loads the browser to execute

The funny thing about Selenium

Jason Huggins built Se to solve a problem much like Nike, then founded a company to solve the problem he created.

By Aslak Hellesøy

Right around the same time, Cucumber came about. It’s a ruby implementation of BDD. It enables you to automate gherkin feature sets

Where Gojko evolved the approach of Crispin & GregoryAslak provided automation for Gojko’s approach to capturing featuresAnd you can drive Selenium through CucumberAnd push it to the cloud, or run things parallel locally

By taking the ATDD & Cucumber (or other BDD implement) route, you short-circuit the entire headache chain that has plagued Agile Testers that have gone before you

Granted, there are new challenges, given that the step definitions need to be crafted to fit the context of your environment, and that is code. But the benefits, IMO, greatly outweigh the cost

But but…

I often get the question – but what if we need to move fast? Or have a legacy code base? We don’t have time for this, what do you recommend?

Focus on the 3 buckets

1) SETIAre the foundational pillars of your system alive and responding?

2) Money MakersWhat in your app is responsible for directly

generating value to the business? Focus on that.

3) Back breakersWatch out for that hole in the Death Star

Keep it simple

This advice is more for people who are going to be writing the glue code

Focus on IDs -- XPath is a last resort

Hard to test apps are a symptom of poor design

Take steps towards speed (and long term sanity)

Make each test scenario fast – 1-2 min

Keep them autonomous (e.g. unique accounts, set up built in, no cross dependencies, etc.)

Single assertions per scenarios

One interpretation of test results

The prior will make parallelization possible

Options abound for both local and cloud offerings

But all that feedback is pretty useless unless you do something with it

Build a feedback loop

Make sure that developers are getting feedback in line with their work flow, else hear the dreaded words "Works on my machine"

A checklist for you

Make Quality The Whole Team’s responsibilityTie things back to WhyBuild a ubiquitous languagePromote from withinKeep it simpleFocus on the 3 buckets (SETI, Money Makers,

Back Breakers)Parallelize for speed

@TourDeDave

http://www.slideshare.net/tourdedave