Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes...
Transcript of Agile Tune-Up: Improving Your Current Testing … Tune-Up: Improving Your Current Testing Processes...
1
Agile Tune-Up: Improving Your Current Testing Processes
Jeff Tatelman Sr. Director, Professional Services
2
AGENDA
• Agile Assessment Approach
• Findings
• Testing Challenges
• Agile Testing Model
• Agile Testing Improvements
• Questions and Answers
3
People
Process
Technology
Level 0: InitialLevel 1: RepeatableLevel 2: DefinedLevel 3: ManagedLevel 4: Optimizing
0
0.5
1
1.5
2
2.5
3
3.5
4
Requirements Fulfillment -2.5
Software Development - 3
Defect Management - 3
Test Planning - 3.5
Test Management - 1
Performance Testing - 1
Unit Testing - 2
Functional/End-to-EndTesting - 1
User Acceptance Testing -1.5
Build / ReleaseManagment - 0
Data/EnvironmentManagement - .5
Reporting/Governance - 1
Software Quality Maturity Level Example
Agile Assessment Approach
4
Agile Assessment Findings
• Test planning and test cases still using waterfall approach
• User Stories & Acceptance Criteria not standardized
• Unit Testing is not automated or performed
• Testing of Sprints not complete
• Automation Regression Framework not built with Agile in mind
• Regression test coverage percentage unknown or not performed
• Build and Release process primarily a manual effort
• Self directed teams are not assuring tools and processes are followed consistently
5
Agile Testing Challenges
• Requirements (stories) will continue to change
• Development should create automated unit test
• Continuous testing during the lifecycle with unstable or incomplete code
• Automate as much as possible
• Regression testing cycles increase
• Regression tests difficult to maintain
• Assurance of consistency
6
Waterfall / Iterative Approach on Agile Projects
Test Case:Click ‘down’ arrow on search Menu appears- ‘Artist,
combo box Album Title, Music Type’
Select ‘Artist’ ‘Artist’ appears in combo boxEnter ‘Santana’ in search text ‘Order’ page is returned with 3
box, click ‘Search’ button entries- ‘Santana – Abraxas, Santana – Santana, Santana Supernatural’. Selection
check box is unchecked, Quantity is ‘1’, etc., etc.
7
ProductionRedefine
Requirements
Production
Redefine Requirements
Sprint Testing
8
Agile Testing Improvements
9
Agile Testing Model
Build
Design
Story 1
Test
ProductBacklog
Sprint IterationPlanning
Sprin
t Review
Additional Stories based on customer
review, changing requirements, etc.
Sprint – 2-3 week timeline
Shippable Product
Build
Design
Story 2
Test
Build
Design
Story N
Test
Story 1Story 2
.
.Story N
Test the Release
Test the Build
10
Agile Test Planning Lifecycle
Plan Develop Test Build Test Release Deploy
11
Agile Test Planning Strategy ExamplePlan
12
UAT Test Plan Mind map Example
13
Agile Test Planning
Plan Develop Test Build Test Release Deploy
User Stories
Acceptance Criteria
Test Scenarios
StoryEstimation
14
Use a Standard User Story Template
Story narrative: Pay for a CD using credit card payment process Priority: High
Short Story name: Credit card payment process
As a: Web Customer
I want: To pay for purchase with credit card Size: Large
So that: web customers can easily buy items.
Acceptance Criteria:
Scenario: Pay for a CD
Given: Item(s) are ready for purchase
When: Items are in shopping cart and ready for checkout
Valid card number is entered
Valid expiration date is entered
Valid code is entered for the card
Then: Authorize Payment
Pay for Purchase
Plan
15
Breaking down User Stories - Are they testable?
• Theme, Epic, Story, Task - help simplify some discussions
• Story - future conversation between product owner and team
• Epic - larger user stories to communicate additional meaning• More than 2 sprints to develop and test
• Broad in scope, short on details
• Split into multiple, smaller stories
• Theme - is a collection of related user stories.
• Prioritized based on their value to the organization
• Acceptance Criteria – confirmation on how delivery team will complete the story
• Tasks – breakdown of work with different effort estimates
Plan
16
• Ideal Time
• Amount of time something will take with no interruptions
• Often expressed in days
• Story Points
• Relative number based on other stories
• Tells team the difficulty the work requires
• A measure of the relative size of a feature
• Based on the total effort involved
Testers Involved In Estimating the Product BacklogPlan
17
• Collaborative with entire team
• Each estimator is given cards.
• Each card has a valid estimate written on it.
• Product Owner reads a story and it is discussed.
• Each estimator selects a card.
• Cards are revealed.
• Differences are discussed.
• Re-estimate until converge.
Story Point Estimation Method – Planning Poker
Mike Cohen: https://www.planningpoker.com/ - https://
Plan
18
Agile Test Development
Plan Develop Test Build Test Release Deploy
User Stories
AutomatedUnit Testing
Acceptance Criteria
Test Scenarios
StoryMapping
Automated UnitTesting
19
Story Mapping Example
Log On Customer Service
Delete aCustomer
Add aCustomer
OrderProcessing
Add anOrder
Hold anOrder
Sequence Variations
Modify anOrder
ShippingInventoryMgt.
ReplenishInventory
Audit Inventory
Ship toCustomer
Bill aCustomer
Visualizing the relationships between stories
2020
Test1 Add customer
Test2 Add Order
Test3 Modify Order
Test4 Change Address
Test5 Ship Order
Test6 Approve Order
Test7 Bill Order
Acceptance Criteria
R1 Customer Service
R2 Order Processing
R3 Order Change
R4 Shipping
R2 add order
R3 Mod Order
R4 Chg Addr
Test1 Test2
R5 Ship
Test3
TC6 Approve order
Test Planning
Story Mapping Matrix
User Stories
Mapping Stories to Tests
R1 Add customer
Develop
Visualize the relationships between stories and tests
21
Coverage Of Stories
Inventory Control System Example
Story ID User Stories Risk Test 1 Test 2 Test 3 Test 4
1.0. Log On
1.1 Valid User Id H X X X
1.2 Invalid User Id L X
2.0. Customer Service
2.1 Add A Customer H X X
2.2 Delete A Customer L X
2.3 Modify Existing Customer M X X
3.0. Order Processing
3.1 Add an Order H X X
3.2 Modify an Order M X X
3.3 Hold an Order M X
4.0. Inventory Management
4.1 Replenish Inventory L X
4.2 Audit Inventory Stock On Hand M X X
5.0. Shipping
5.1 Bill A Customer H X X X
5.2 Ship To All Customers H X X
Develop
22
Test Automation
Plan Develop Test Build Test Release Deploy
User Stories
AutomatedUnit Testing
Acceptance Criteria
Test Scenarios
StoryMapping Automated
UI Testing
EnvironmentReadiness
Testing
Automated UnitTesting
Automated SmokeTest
23
Top Down – Bottom Up Automation Frameworks
• A Top Down Framework is a design methodology that starts at the high level business process and breaks logical pieces apart according to their highest level of reusability to create a test looking from the “top down”. Examples: Action/data driven (Gherkin)
• Bottom Up Framework is a design methodology that starts at the basic building blocks for functionality and creates tests from the “bottom up”. Examples: keyword driven or function based (Page Object Modeling)
Test Scenario
Test Driver
Test Data
Actions
Test Scenario
Test Driver
Test Data
Functions/Actions
Develop
24
• Top Down Framework to describe software behavior
• Created for behavior driven descriptions translated to Development (BDD)
• Write business-readable test cases based on acceptance criteria
• Given / When / Then format
• Like Python, YAML, it’s a line oriented language to define structure
• Serves the purpose to document and automate tests
• Support tools: Cucumber, Eggplant, ALM Octane, Hiptest, SpecFlow
Bridging Testers with Automation - GherkinDevelop
25
BDD Data Driven Test - Gherkin Example
• As an Admin User,I would like to change the password of other user’s accounts.
Feature: Update passwordScenario: Admin user can update user passwordGiven I am in the HR system with an Admin accountWhen I update password of another userThen I receive a message for updating password successfullyAnd user’s password is updated to the new password
• Automated test:
Scenario Outline: Verify Updating user password featureGiven I am in the HR system with “<account_type>” account
and there is another user with “<old_password>” passwordWhen I update password of the user to “<new_password>”Then I got the message “<message>”
and the user password should be “<final_password>”
• Examples:
|account_type |old_password |new_password |message |final_password |||Admin |$Test123 |@Test123 |Password changed..|@Test123 ||Viewer |$Test123 |@Test123 |Invalid right access.. |$Test123 |
Develop
26
Exploratory Testing
Plan Develop Test Build Test Release Deploy
User Stories
AutomatedUnit Testing
Acceptance Criteria
Test Scenarios
StoryMapping
ExploratoryTesting
27
Exploratory Testing Benefits
• Helps testers keep up with the Agile development pace
• Takes less preparation
• Faster feedback including finding critical defects
• Testers become familiar with the application/efficient on future iterations
• Beyond automated unit tests
• Empowering testers in achieving high quality products
Test
28
Role of Testers working with DevOps
Continuous Integration
Continuous Delivery
Plan Develop Test Build Test Release Deploy
Continuous Deployment
Test each change automatically in your codebase early and often.
Automatically deploy code into a production like environment for further testing.
Update production environments automatically.
Leadership Knowledge Teamwork
Automated UnitTesting
Automated SmokeTest
29
Testing Continuous Integration, Delivery, Deployment
Continuous Integration
Continuous Delivery
Plan Develop Test Build Test Release Deploy
Continuous Deployment
Automated Testing
EnvironmentReadiness
Testing
Automated UnitTesting
Automated SmokeTest
30
TestExecution
Code Analysis
Automated UnitTesting
Agile Related Tools
CollaborationTest & Defect Management
IDEIntegrated Development
Environment
Source Control
Plan Develop Test Build Test Release Deploy
Automated Builds for
ContinuousDelivery/
Deployment
PerformanceTesting
Automated Builds for
ContinuousIntegration
Reporting
Environment Management
Test Automation
Coded UI
Allure
Automated SmokeTest
31
Skills For Agile Testers
• Know how to:
• play well with others:
• Take a developer to lunch
• Pair up with developers to write unit tests
• Celebrate milestones – 100 automated unit tests written
• focus on what business stakeholders care about
• ask the right questions, and when to ask them
• have a positive attitude (over aptitude)
• listen (and be a good communicator)
• deal with difficult personalities
• manage your time effectively
• trust your judgment — and your intuition
32
Summary
Continuous Integration
Continuous Delivery
Plan Develop Test Build Test Release Deploy
Continuous Deployment
User Stories
AutomatedUnit Testing
Acceptance Criteria
Test Scenarios
StoryMapping Automated
Testing
EnvironmentReadiness
Testing
Automated UnitTesting
Automated SmokeTest
ExploratoryTesting