QA Transformation

“Get on board or risk NOW!”Ray Scott

QA Transformation Consultantrayscott65@sky.com

Ray.scott@qualityagile.co.uk

Who am I?• 25 years I.T (15 QA, 10 Agile)

• Sr. QA Manager & Agile – Walt Disney Theme Parks & Resorts

• Director Agile and professional services – Grid Tools

• Bank of Boston, Fidelity Investments

• QA Transition Manager/Interim Head of QA• Ladbrokes/Aditi

• UK Leading airline

• Head of Performance SOA, Consultant

• Speaker at:• Agile and QA conferences throughout Europe and USA

2

AGILE – True of False?CULTURE METHODOLOGYSET OF PRACTICESPracticing SCRUM is AGILEPerforming TDD, BDD and Pair programming means

you are AGILEIs hard to do!

What is Software Testing?

4

Testing practices

• Test plans

• Test cases/scenarios

• Aligning to requirements

• Wait for code to be delivered

• Automation

• Expect defects and report

• KPI

5

• Defect management

• Triage manager

• Test manager

• Test lead

• Tester

• QA Manager

€€ £££ $$$

What is the cost of

exposing defects?

6

Costs of Issues

7

Req. Arch. Dev. Prod

• Total 400 hrs ~ £40,000• - Clients £ £ £ £ £

£300

Detail Design.

£800 £1800 £3800 £6000

SIT UAT

£10,000

PreProd

£20,000

Off-shoring has become BIG-BUSINESS

What are we trying to save?

9

Problems!

Pushing “LEFT”

10

Req. Arch. Dev. ProdDetail Design. SIT UAT PreProd

1X 6X 13X 20X 35X 65X 140+X3X

Traditional testing€€ £££ $$$Quality

AssuranceTesting finds defects,

QA prevents defects

How does it all change for a

tester in an Agile

environment?

11

Some core concepts

12

• Collaboration

• Communication

• Transparency

• Innovation

How we work together

How we move information

Know what is going on

Trying something different

Does this sound like Testing?

Traditional Tester

13

Transforming

14

WoW - Ways of Working

15

Testing finds defects, QA prevents

defects

Quality Assurance agenda

16

• Testing is the responsibility of the TEAM

• Testing occurs ALL THE TIME

• “Push-Left”

• Reduce risk and increase confidence with predictability

• Alignment to business goals in the SPRINT

• Validates a potential release candidate

• DONE is DONE

How can testing be the responsibility of the

TEAM?• Development testing practices, TDD and BDD

• Non-Functional requirements

• Product Owner feedback

• Test Scenarios over Test cases

• Manage technical debt (Sev 3 and 4s)• Identifying Sev 1 and 2s early leaves time for 3s and 4s

17

Quality Assurance agenda

18

• Testing is the responsibility of the TEAM

• Testing occurs ALL THE TIME

• “Push-Left”

• Reduce risk and increase confidence with predictability

• Alignment to business goals in the SPRINT

• Validates a potential release candidate

• DONE is DONE

How can testing occur ALL THE TIME?

• Continuous Integration (Build & Test) - TDD, BDD

• QA reviews

• Product Owner reviews

• Plan for the Unplanned

• Avoid more than needs to be tested

19

Quality Assurance agenda

20

• Testing is the responsibility of the TEAM

• Testing occurs ALL THE TIME

• “Push-Left”

• Reduce risk and increase confidence with predictability

• Alignment to business goals in the SPRINT

• Validates a potential release candidate

• DONE is DONE

Push Left = How do we detect defects earlier?

• The best protection is EARLY PROTECTION

• Understand the Sprint delivery

• Create relationships with team

• Communication plans

• Participate in user story maintenance (Func. & NFR)

• Perfection is only a test case away

21

Quality Assurance agenda

22

• Testing is the responsibility of the TEAM

• Testing occurs ALL THE TIME

• “Push-Left”

• Reduce risk and increase confidence with predictability

• Alignment to business goals in the SPRINT

• Validates a potential release candidate

• DONE is DONE

How can RISK be reduced?

• “The probability of LOSS” AND “ Exposing to harm or

danger”

• Understand the limitations of what is under test

• Testing is NOT trying to break it

• Shorter, incremental & cumulative

• Understand the data

23

Quality Assurance agenda

24

• Testing is the responsibility of the TEAM

• Testing occurs ALL THE TIME

• “Push-Left”

• Reduce risk and increase confidence with

predictability

• Alignment to business goals in the SPRINT

• DONE is DONE

How to align with the business

• Know your Product owners (Business Representative)

• Prove the value you bring to the product development

• Intelligently challenge

• Walk through demo’s

• Introduce Quality Controls (KPI’s)

25

Quality Assurance agenda

26

• Testing is the responsibility of the TEAM

• Testing occurs ALL THE TIME

• “Push-Left”

• Reduce risk and increase confidence with

predictability

• Alignment to business goals in the SPRINT

• DONE is DONE

When are we DONE?

• Define DONE

• Work towards DONE

• Communicate DONE to all

• Stop when DONE enough

• DONE-DONE-DONE

27

Joining an Agile team

• Testers are pack animals• Joining an Agile can be a lonely existence

• Reporting into a Dev. Manager

• Reduced structure, stage gates and documentation

• Loss of control

28

The new world of Agile

• Testers no longer need to be siloed or feel alone in Agile

• Work with support groups

• Influence team members

• Focus on Quality not quantity of test cases

• Testing finds defects, QA prevents defects

• Requires courage to move out of comfort zone

• Learn new skills (Customer focus)

• Team up with those that build the quality into product delivery

29

WoW - Ways of Working

30

A real world worked example

Environment

• Agile-like

• 3rd party development team, distributed

• 3rd party testing team, distributed (Not same as above)

• Product Owner engaged and committed

• Some SCRUM practices encouraged

• Too much technical debt• Sev 3 and 4 pushed aside

• UAT duplicated SIT testing

31

WoW – original lifecycle

32

Development SIT

UAT

Goals

• Reduce cycle delivery time for Sprints

• Reduce number of defects found late in the cycle

• Increase confidence with delivery of quality product

33

KPIs –Cumulated defects

10/1/2

013

10/2/2

013

10/3/2

013

10/4/2

013

10/5/2

013

10/6/2

013

10/7/2

013

10/8/2

013

10/9/2

013

10/10/2

013

10/11/2

013

10/12/2

013

10/13/2

013

10/14/2

013

10/15/2

013

10/16/2

013

10/17/2

013

10/18/2

013

10/19/2

013

10/20/2

013

10/21/2

013

10/22/2

0130

5

10

15

20

25

30

35

40

45

Sprint 1 & 2

WoW

• Early engagement of QA with Dev and Business team• Sharing product and testing objectives

• Added value to Dev testing effort

• Last few days of Sprint became a Pre-SIT period• Execution of “Happy Path”

• Early detection of Sev 1s & 2

• Entry stage gate criteria met

• Fast turn around of defects

35

WoW

• Entry to formal SIT with “Known Knowns”

• More confidence of product quality• Less “Unknown Unknowns”

• Next Sprint Technical debt (Sev 3/4)

• Product owner can “Go-Live” earlier• UAT cycle shorter

• Less technical debt

• More communication….more reliability….earlier value

36

WoW – Modified lifecycle

37

Development SIT

UAT

Development SITPre-SIT

UATQA involvement

KPIs –Cumulative KNOWN defects

11/1/2

013

11/2/2

013

11/3/2

013

11/4/2

013

11/5/2

013

11/6/2

013

11/7/2

013

11/8/2

013

11/9/2

013

11/10/2

013

11/11/2

013

11/12/2

013

11/13/2

013

11/14/2

013

11/15/2

013

11/16/2

013

11/17/2

013

11/18/2

013

11/19/2

013

11/20/2

013

11/21/2

013

11/22/2

0130

5

10

15

20

25

30

35

40

45

Sprint 3 & 4

10/1/2

013

10/2/2

013

10/3/2

013

10/4/2

013

10/5/2

013

10/6/2

013

10/7/2

013

10/8/2

013

10/9/2

013

10/10/2

013

10/11/2

013

10/12/2

013

10/13/2

013

10/14/2

013

10/15/2

013

10/16/2

013

10/17/2

013

10/18/2

013

10/19/2

013

10/20/2

013

10/21/2

013

10/22/2

0130

5

10

15

20

25

30

35

40

45

Sprint 1 & 2

WoW – comments from team

“Much more engagement created a collaborative environment”“Gut feel entering SIT stage was superior”“Quality product delivered – but more effort needed”

39

In closing

• AGILE is HARD to do!• Agile is NOT for everyone• Leaders over Mangers• A Pat on the back cost nothing and Motivates

40

Questions ????

Good reading :

41

Published in ComputerWorld.co.uk

“Say hello to QA and goodbye to testing”“Failure is only a stepping stone to success”“Test data governance”“Test data warehouse”“What does quality look like”“Test data management is the new Automation”