!"
#
$
%
&'
$./run
{
Created by Fabrício Epaminondas ©2015-2020
ContinuousTESTING
Testing in an agile environment
• Continuous Testing • Agile principles • Why Continuous Testing? • Creating your first test project in minutes • How it works • Test process • Effective Continuous Testing
15/10/2015 - 20:30 Location: Cinema didático - Facisa
Agenda
The goal
AGILE PRINCIPLES
The Big Picture of General SDLC 4
3
21
SDLC
Requirement Initiation
Analysis & Specification
Risks
Delivery User Acceptance (UAT)
Deployment
Maintenance and Support
Operations
4
4 }
( '
Planning Scope, Time and Cost
Prioritization
Development Design / Prototype
Implementation
Integration
Testing
Documentation
Release
Software-Development-Life-Cycle
5Agile process: Scrum
SPRINT REVIEW /RETROSPECTIVE
2-4 Weeks
1 2 3
4
6
“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”
The Agile Manifesto, First Principle
WHY CONTINUOUS
TESTING?The metaphor
8Software-Development-Life-Cycle Metaphor
1.Jack is a happy programmer, he has just done his first commit
2. After a few days, he receives the test report from QA Team reproving his changes and pointing a new BUG X in production.
3.Jack is sad now, but he fixes the bug and submit his changes again.
4. Now it is the week of a new release and Jack receives other report confirming the BUG X is fixed
5. But the report also contains new BUGS: A, B, C.
9
6.Jack tries to figure out if his changes caused the other bugs
7.But Jack is not alone, he tries to figure out if someone else changes caused the other bugs
8.Jack tries a new solution to get done and re-tested in all environments before the deadline
9.Poor, poor, Jack… it is too late… the deadline has come.
Software-Development-Life-Cycle Metaphor
10
What does Jack do?
Software-Development-Life-Cycle Metaphor
11
Let’s think for a moment…
What are bugs? What do they eat? Where do they live? How to avoid new bugs?
What if bugs were living entities…
What are they thinking?
Bugs….
12
"I WILL BE BACK"(The Bug)
)
13
TRUST ME, I CAN DO IT.. ( Developer in a good day)
14
• To control side effects • To avoid software defect snowball• To efficiently support multiple platforms,
environments and release versions • To get fast results in a short time cycle • To anticipate risks• To create an agile, reliable and repeatable process• Release in hours not months
Use Continuous Testing…
CREATING YOUR FIRST PROJECT
IN MINUTES
Hands-On
16Hands-On
• Let’s create our first project for continuous testing using an amazing and totally free web hosting platform with continuous integration support:
Let’s start from the test project template:
https://gitlab.com/fabricioepa/gitlab-ci-quickstart
gitlab.com
17Hands-On•You can learn how to implement your own test platform
in the training:
https://fabricioepa.wordpress.com/2015/02/02/quickstart-for-continuous-integration/
•For more advanced continuous integration practices see the link:
https://fabricioepa.wordpress.com/2015/02/03/continuous-integration-practices-to-improve-the-software-quality/
HOW IT WORKS
Overview
19Overview
• Test Environment• Test Data Management • Automatic Deployment • Multiple environment configuration profiles •Virtualization (production-like) •Dynamic Infra-structure
•Test Automation • Orchestration
*Continuos Testing
20Related Topics
• Continuous Integration (CI) • Continuous Delivery • Continuous Deployment • See 3 high levels of continuous testing adoption
*Continuos Testing
Design development pipelines 21
Commit Stage Unit Tests
Code Analysis
Code Review
'
Triggers
/branches/product-1.0
Development Unit Tests
Code Analysis
Development pipeline
Build Package
Sanity Tests
Verify Metrics
Publishing Test Report
Unit Test Coverage
Project Metrics
Release Package
Fast Builds pattern ,
Artifact Repository$
01
22
Setup Environment
Infrastructure & Data
QA automation pipeline
Deploy Install packages
Automated Test Plan Integration,
Functional, Regression,
Performance, etc.
Publishing Test Report
Test Coverage
Performance Benchmark
○ Push-buttonsScheduled buildsRepository changes{
' /branches/product-1.0
Triggers
Design QA pipelines
Self-service " + Nightly-builds {
Artifact Repository$
02
23
Setup Environment
Infrastructure
Data
Testing Manual
Exploratory
○
Deploy Install packages
Health check
○
. QA Team
UAT
Self-service " deployments
Design delivery and deployment pipelines
Delivery / Deployment pipelines
User Acceptance Demonstrations
Consumer Preview
Beta testing
GA releases
03
Artifact Repository$
AGILE TEST PROCESS
The real deal
25Main Testing Stages
Planning Scope, Time, Cost
Process guideline
Specification Test Scenarios
Test Cases
Prioritization and
Classification
Execution Manual
Automatic
Closure Test Report
Bug Report
Qualification
Documentation
1 2 3 4
26Test plan
A good test plan should answer at least 3 main questions:
• What? • How? • When?
01
s27Test plan: What?
Guideline for macro activities•Test execution
• Defect detection (Bugs) • Enhancement areas
)
01
• Strategies and methodology• Environment and tools
28Test plan: How?01
• Traceability
29Test plan: How?01
30
Schedule
Test plan: When?
/" 0Team and Resources
Test Scope and Risks
4
01
31
•Black-box / White-box •Unit / Component /
Integration •Smoke / Sanity •Acceptance / Functional •Regression •System / End-To-end •Exploratory
•Load / Stress / Performance
•Parallel / Concurrent •Usability •Alpha / Beta •Compatibility •Destructive / Recovery •Internationalization /
Localization
Choose the right test techniques01
32Test specification
• Requirements study •Test Scenarios and Estimative of Test Cases •Test Case Creation, Review and Maintenance •Prioritization and Classification
•Automatic vs. Manual •Others: Acceptance, Functional, Performance,
etc. •This information will be used for creating
execution test plans.
02
33Executing test plans
•Manual execution •Automatic execution (by QA test pipeline) •Whatever the way you take, you must choose a
strategy to compute all test results in the final analysis
•To use a tool like TestLink may help you to manage all tests.
03
34Closure
•Reports • Test Execution Status • Defects by Priority • Requirements Test Coverage • Other quality aspects:
•Performance, accuracy, security issues, etc.
04
35Closure
•Final report and Qualification: •Approval Status of Deliverables
04
1
X1
Feature A
Feature B
Feature C
L Partial release approval
The real value
EFFECTIVE CONTINUOUS
TESTING
• Let’s remember the agile principle:
37Continuous Quality Improvement
“…continuous delivery of valuable software.” The Agile Manifesto, First Principle
• Testing != Software Quality Assurance • You must understand how to use tests to enhance
the process, reduce rework, increases the quality and business value of the software product.
• This is Continuous Improvement, one of the most important QA practices.
38Continuous Improvement in 4 Steps
Plan Targets, Methodology,
Process, etc…
Do Process execution
Verification / Study Measure Results
Study evidences and deviations
from original plan
Action Diagnostic
Corrective and Preventive Action Points
Improvement Areas
Notes: Continuous Improvement and Agile
• Retrospectives • Learned Lessons • Action Points • Use the PDCA Cycle 2
40Effective Continuous Testing and Agile
•Objective •For each sprint the deliverable must potentially
be production-ready •Quality metrics and daily feedback
•Build status and reports •Statistics and Trends
•Automatic Tests report •Code violations •Code coverage •Project metrics
•Dev&QA teams must understand and maintain the quality metrics
3 Displays + Dashboards
% Email notifications
Continuous Monitoring
Effective Continuous Testing and Agile
42
•Balance test levels according to the project needs on each sprint
Effective Continuous Testing and Agile
Unit Test
Integration Test
System Test
Acceptance Test
Test individual component
Test integrated system
Test component groups
Test final system stack
V-model
43
•Continuously improve your automated testsEffective Continuous Testing and Agile
Unit
Integration
E2E
• Recommended test distribution pyramid
Cheaper automation
Complex Automation
UI / Manual / Exploratory Tests
Reference: [1] http://googletesting.blogspot.com.br/2015/04/just-say-no-to-more-end-to-end-tests.html
44
•Unrelated activities (to the backlog) •Implementing automation scripts •Status monitoring •Multiple environment maintenance: Testing,
Staging, Production… •Sprint Retrospective Tasks
•Use CI feedback to continuous improve the process and the product quality
•Diagnosing pitfalls and registering sprint action points
Continuous Testing and Agile Planning
45Business Value
•The cost of adoption has a fast return on investment
•Rework reduction •Increase team productivity. •Release in days not months •Reduce at about 50% of development, testing
and deployment costs.
!"
#
$
%
&'
$./run
{
Criado por Fabrício Epaminondas ©2015
ContactFABRICIO EPAMINONDAS 4 linkedin.com/in/fabricioepa
5 fabricioepa.wordpress.com
6 fabricioepa
7 @fabricioepa
Top Related