Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes...

33
1 Agile Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services [email protected]

Transcript of Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes...

Page 1: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

1

Agile Tune-Up: Improving Your Current Testing Processes

Jeff Tatelman Sr. Director, Professional Services

[email protected]

Page 2: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

2

AGENDA

• Agile Assessment Approach

• Findings

• Testing Challenges

• Agile Testing Model

• Agile Testing Improvements

• Questions and Answers

Page 3: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

3

People

Process

Technology

Level 0: InitialLevel 1: RepeatableLevel 2: DefinedLevel 3: ManagedLevel 4: Optimizing

0

0.5

1

1.5

2

2.5

3

3.5

4

Requirements Fulfillment -2.5

Software Development - 3

Defect Management - 3

Test Planning - 3.5

Test Management - 1

Performance Testing - 1

Unit Testing - 2

Functional/End-to-EndTesting - 1

User Acceptance Testing -1.5

Build / ReleaseManagment - 0

Data/EnvironmentManagement - .5

Reporting/Governance - 1

Software Quality Maturity Level Example

Agile Assessment Approach

Page 4: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

4

Agile Assessment Findings

• Test planning and test cases still using waterfall approach

• User Stories & Acceptance Criteria not standardized

• Unit Testing is not automated or performed

• Testing of Sprints not complete

• Automation Regression Framework not built with Agile in mind

• Regression test coverage percentage unknown or not performed

• Build and Release process primarily a manual effort

• Self directed teams are not assuring tools and processes are followed consistently

Page 5: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

5

Agile Testing Challenges

• Requirements (stories) will continue to change

• Development should create automated unit test

• Continuous testing during the lifecycle with unstable or incomplete code

• Automate as much as possible

• Regression testing cycles increase

• Regression tests difficult to maintain

• Assurance of consistency

Page 6: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

6

Waterfall / Iterative Approach on Agile Projects

Test Case:Click ‘down’ arrow on search Menu appears- ‘Artist,

combo box Album Title, Music Type’

Select ‘Artist’ ‘Artist’ appears in combo boxEnter ‘Santana’ in search text ‘Order’ page is returned with 3

box, click ‘Search’ button entries- ‘Santana – Abraxas, Santana – Santana, Santana Supernatural’. Selection

check box is unchecked, Quantity is ‘1’, etc., etc.

Page 7: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

7

ProductionRedefine

Requirements

Production

Redefine Requirements

Sprint Testing

Page 8: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

8

Agile Testing Improvements

Page 9: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

9

Agile Testing Model

Build

Design

Story 1

Test

ProductBacklog

Sprint IterationPlanning

Sprin

t Review

Additional Stories based on customer

review, changing requirements, etc.

Sprint – 2-3 week timeline

Shippable Product

Build

Design

Story 2

Test

Build

Design

Story N

Test

Story 1Story 2

.

.Story N

Test the Release

Test the Build

Page 10: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

10

Agile Test Planning Lifecycle

Plan Develop Test Build Test Release Deploy

Page 11: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

11

Agile Test Planning Strategy ExamplePlan

Page 12: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

12

UAT Test Plan Mind map Example

Page 13: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

13

Agile Test Planning

Plan Develop Test Build Test Release Deploy

User Stories

Acceptance Criteria

Test Scenarios

StoryEstimation

Page 14: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

14

Use a Standard User Story Template

Story narrative: Pay for a CD using credit card payment process Priority: High

Short Story name: Credit card payment process

As a: Web Customer

I want: To pay for purchase with credit card Size: Large

So that: web customers can easily buy items.

Acceptance Criteria:

Scenario: Pay for a CD

Given: Item(s) are ready for purchase

When: Items are in shopping cart and ready for checkout

Valid card number is entered

Valid expiration date is entered

Valid code is entered for the card

Then: Authorize Payment

Pay for Purchase

Plan

Page 15: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

15

Breaking down User Stories - Are they testable?

• Theme, Epic, Story, Task - help simplify some discussions

• Story - future conversation between product owner and team

• Epic - larger user stories to communicate additional meaning• More than 2 sprints to develop and test

• Broad in scope, short on details

• Split into multiple, smaller stories

• Theme - is a collection of related user stories.

• Prioritized based on their value to the organization

• Acceptance Criteria – confirmation on how delivery team will complete the story

• Tasks – breakdown of work with different effort estimates

Plan

Page 16: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

16

• Ideal Time

• Amount of time something will take with no interruptions

• Often expressed in days

• Story Points

• Relative number based on other stories

• Tells team the difficulty the work requires

• A measure of the relative size of a feature

• Based on the total effort involved

Testers Involved In Estimating the Product BacklogPlan

Page 17: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

17

• Collaborative with entire team

• Each estimator is given cards.

• Each card has a valid estimate written on it.

• Product Owner reads a story and it is discussed.

• Each estimator selects a card.

• Cards are revealed.

• Differences are discussed.

• Re-estimate until converge.

Story Point Estimation Method – Planning Poker

Mike Cohen: https://www.planningpoker.com/ - https://

Plan

Page 18: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

18

Agile Test Development

Plan Develop Test Build Test Release Deploy

User Stories

AutomatedUnit Testing

Acceptance Criteria

Test Scenarios

StoryMapping

Automated UnitTesting

Page 19: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

19

Story Mapping Example

Log On Customer Service

Delete aCustomer

Add aCustomer

OrderProcessing

Add anOrder

Hold anOrder

Sequence Variations

Modify anOrder

ShippingInventoryMgt.

ReplenishInventory

Audit Inventory

Ship toCustomer

Bill aCustomer

Visualizing the relationships between stories

Page 20: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

2020

Test1 Add customer

Test2 Add Order

Test3 Modify Order

Test4 Change Address

Test5 Ship Order

Test6 Approve Order

Test7 Bill Order

Acceptance Criteria

R1 Customer Service

R2 Order Processing

R3 Order Change

R4 Shipping

R2 add order

R3 Mod Order

R4 Chg Addr

Test1 Test2

R5 Ship

Test3

TC6 Approve order

Test Planning

Story Mapping Matrix

User Stories

Mapping Stories to Tests

R1 Add customer

Develop

Visualize the relationships between stories and tests

Page 21: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

21

Coverage Of Stories

Inventory Control System Example

Story ID User Stories Risk Test 1 Test 2 Test 3 Test 4

1.0. Log On

1.1 Valid User Id H X X X

1.2 Invalid User Id L X

2.0. Customer Service

2.1 Add A Customer H X X

2.2 Delete A Customer L X

2.3 Modify Existing Customer M X X

3.0. Order Processing

3.1 Add an Order H X X

3.2 Modify an Order M X X

3.3 Hold an Order M X

4.0. Inventory Management

4.1 Replenish Inventory L X

4.2 Audit Inventory Stock On Hand M X X

5.0. Shipping

5.1 Bill A Customer H X X X

5.2 Ship To All Customers H X X

Develop

Page 22: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

22

Test Automation

Plan Develop Test Build Test Release Deploy

User Stories

AutomatedUnit Testing

Acceptance Criteria

Test Scenarios

StoryMapping Automated

UI Testing

EnvironmentReadiness

Testing

Automated UnitTesting

Automated SmokeTest

Page 23: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

23

Top Down – Bottom Up Automation Frameworks

• A Top Down Framework is a design methodology that starts at the high level business process and breaks logical pieces apart according to their highest level of reusability to create a test looking from the “top down”. Examples: Action/data driven (Gherkin)

• Bottom Up Framework is a design methodology that starts at the basic building blocks for functionality and creates tests from the “bottom up”. Examples: keyword driven or function based (Page Object Modeling)

Test Scenario

Test Driver

Test Data

Actions

Test Scenario

Test Driver

Test Data

Functions/Actions

Develop

Page 24: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

24

• Top Down Framework to describe software behavior

• Created for behavior driven descriptions translated to Development (BDD)

• Write business-readable test cases based on acceptance criteria

• Given / When / Then format

• Like Python, YAML, it’s a line oriented language to define structure

• Serves the purpose to document and automate tests

• Support tools: Cucumber, Eggplant, ALM Octane, Hiptest, SpecFlow

Bridging Testers with Automation - GherkinDevelop

Page 25: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

25

BDD Data Driven Test - Gherkin Example

• As an Admin User,I would like to change the password of other user’s accounts.

Feature: Update passwordScenario: Admin user can update user passwordGiven I am in the HR system with an Admin accountWhen I update password of another userThen I receive a message for updating password successfullyAnd user’s password is updated to the new password

• Automated test:

Scenario Outline: Verify Updating user password featureGiven I am in the HR system with “<account_type>” account

and there is another user with “<old_password>” passwordWhen I update password of the user to “<new_password>”Then I got the message “<message>”

and the user password should be “<final_password>”

• Examples:

|account_type |old_password |new_password |message |final_password |||Admin |$Test123 |@Test123 |Password changed..|@Test123 ||Viewer |$Test123 |@Test123 |Invalid right access.. |$Test123 |

Develop

Page 26: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

26

Exploratory Testing

Plan Develop Test Build Test Release Deploy

User Stories

AutomatedUnit Testing

Acceptance Criteria

Test Scenarios

StoryMapping

ExploratoryTesting

Page 27: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

27

Exploratory Testing Benefits

• Helps testers keep up with the Agile development pace

• Takes less preparation

• Faster feedback including finding critical defects

• Testers become familiar with the application/efficient on future iterations

• Beyond automated unit tests

• Empowering testers in achieving high quality products

Test

Page 28: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

28

Role of Testers working with DevOps

Continuous Integration

Continuous Delivery

Plan Develop Test Build Test Release Deploy

Continuous Deployment

Test each change automatically in your codebase early and often.

Automatically deploy code into a production like environment for further testing.

Update production environments automatically.

Leadership Knowledge Teamwork

Automated UnitTesting

Automated SmokeTest

Page 29: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

29

Testing Continuous Integration, Delivery, Deployment

Continuous Integration

Continuous Delivery

Plan Develop Test Build Test Release Deploy

Continuous Deployment

Automated Testing

EnvironmentReadiness

Testing

Automated UnitTesting

Automated SmokeTest

Page 30: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

30

TestExecution

Code Analysis

Automated UnitTesting

Agile Related Tools

CollaborationTest & Defect Management

IDEIntegrated Development

Environment

Source Control

Plan Develop Test Build Test Release Deploy

Automated Builds for

ContinuousDelivery/

Deployment

PerformanceTesting

Automated Builds for

ContinuousIntegration

Reporting

Environment Management

Test Automation

Coded UI

Allure

Automated SmokeTest

Page 31: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

31

Skills For Agile Testers

• Know how to:

• play well with others:

• Take a developer to lunch

• Pair up with developers to write unit tests

• Celebrate milestones – 100 automated unit tests written

• focus on what business stakeholders care about

• ask the right questions, and when to ask them

• have a positive attitude (over aptitude)

• listen (and be a good communicator)

• deal with difficult personalities

• manage your time effectively

• trust your judgment — and your intuition

Page 32: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

32

Summary

Continuous Integration

Continuous Delivery

Plan Develop Test Build Test Release Deploy

Continuous Deployment

User Stories

AutomatedUnit Testing

Acceptance Criteria

Test Scenarios

StoryMapping Automated

Testing

EnvironmentReadiness

Testing

Automated UnitTesting

Automated SmokeTest

ExploratoryTesting

Page 33: Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes Jeff Tatelman Sr. Director, Professional Services ... Santana –Santana, Santana

33

Jeff Tatelman Sr. Director, Professional Services

[email protected]