Succesful Testing - The Continuous Delivery Process

30
codecentric Nederland BV @huibschoots & @mieldonkers SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS

description

Presentation on testing for continuous delivery models. Enabling DevOps methodologies.

Transcript of Succesful Testing - The Continuous Delivery Process

Page 1: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

@huibschoots & @mieldonkers

SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS

Page 2: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

INTRODUCTION

Miel Donkers Developer @mieldonkers

Huib Schoots Tester @huibschoots

Page 3: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

• Experience with Continuous Delivery? • As a tester, do you need to wait for an

acceptable release from developers? • Loose time doing all repetitive manual stuff?

• Maintain different environments and software versions manually, over-and-over-and-over…

•  Is this really release 1.6.5?

• All sounds familiar?

TYPICAL

Page 4: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

Prod

BUILDING A RELEASE

TEST

Local Build

Integration server

Release build

DEV

Acc

Test

Users

DEV DEV

Page 5: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

WHY CONTINUOUS DELIVERY

Page 6: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

WHAT IS CONTINUOUS DELIVERY

•  Continuous Delivery pipeline

•  Automation

•  Manual control

•  Feedback

Page 7: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

GOALS

•  Predictability

•  Quality

•  Speed

Page 8: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

WHAT ARE THE KEY BENEFITS OF CONTINUOUS DELIVERY?

•  Low-risk releases

•  Faster return on investment in software projects

•  Improvement of competitiveness and responsiveness

•  Quality improvement of new software versions

Page 9: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

CONTINUOUS IMPROVEMENT CYCLE

Requires: Discipline & Responsibility

Page 10: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

STOP THE LINE!

Page 11: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

PUSH VS. PULL

Page 12: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

ANATOMY IN DETAIL

Source Code

Commit Stage

Compile Commit Tests

Assemble Code Analysis

Artifact Repository

Acceptance Stage

Configure Environment Deploy Binaries

Smoke Test Acceptance Tests

UAT Configure Environment

Deploy Binaries Smoke Test

Capacity Stage Configure Environment

Deploy Binaries Smoke Test

Run Capacity Tests

Production Configure Environment

Deploy Binaries Smoke Test

Env.&App Config

Env.&App Config

Version Control

Testers Self-service deployments

Operations Perform push

Button releases

Developers See code metrics and test failures

reports binaries metadata binaries

reports metadata binaries

reports metadata

Page 13: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

WHAT ARE THE MAIN STEPS OF CONTINUOUS DELIVERY?

•  Build – compile, unit test, version, package

•  Quality – metrics, documentation

•  Test – acceptance-, regression- and performance tests

•  Provision environments – deployment to test- and staging environment

•  Production – green/blue deployment to production

Page 14: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

MAIN STEPS TO CREATE A CONTINUOUS DELIVERY PIPELINE?

•  Getting started with virtual environments, e.g. Amazon EC2

•  Configure your Continuous Integration Server

•  Provisioning your Test, Staging and Production environments

•  Configure your Continuous Delivery Pipeline

•  Create a Dashboard of your Systems

Page 15: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

EXAMPLE

! Elke stage een slide.

Page 16: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

WHAT ARE THE MAIN STEPS OF CONTINUOUS DELIVERY?

Page 17: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

TOOLING THAT CAN HELP YOU

Page 18: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

EXAMPLE – TECHNOLOGY STACK

•  Amazon EC2 (Virtualization, Cloud) •  Jenkins (CI Server) •  Git/Github (Version Control) •  Sonar (Code Quality) •  jUnit (Unit Tests) •  jBehave (Acceptancetests) •  Selenium (UI Tests)

•  Puppet (Provisioning) •  Tomcat (Application Server) •  Artifactory (Artifact Repository) •  Maven (Build Tool) •  jMeter (Performance Tests) •  AppDynamics (Performance Tests) •  Liquibase (DB Versioning)

Page 19: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

OUR SETUP

https://github.com/...

CD Server

Test Server

User Acceptance Test Server

Production Cluster Blue / Green

CD Slave

Page 20: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

EXAMPLE – DELIVERY PIPELINE 1/2

Push to Git Repo 1

Provision-Deploy Acceptance Commit

Trigger CD Pipeline 2

Build Source Code 3

Run Unit Tests 4

Start EC2 Image 1

Provisioning 2

DB Create Update 3

Deploy WAR 4

Start Tomcat 5

Start Selenium 1

Acceptance Test

Run AATs 2

Version Git Repo 6

Release WAR 7

Run Code Analyse 5

Page 21: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

EXAMPLE – DELIVERY PIPELINE 2/2

Run JMeter 1

Provision-Deploy UAT

Start EC2 Image 2

Provisioning 3

DB Create Update 4

Deploy WAR 5

Start Tomcat 6

Performance Test

Select Version 1

Provision-Deploy PRODBLUE/GREEN

Start EC2 Image 2

Provisioning 3

DB Update 4

Deploy WAR 5

Start Tomcat 6

Select Version 1

Page 22: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

ASK YOURSELF

•  Which parts are tested how?

•  What remains to be tested?

Page 23: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

APPLICATION STACK

Apps / Services / Components

Application Configuration

Middleware Middleware

Configuration

Operating System

OS Configuration

Hardware

TESTING

Page 24: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

WHAT DO WE TEST WHERE AND WHEN?

•  Automation is key

•  Do what we do best

•  Team effort

Page 25: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

MANUAL TESTING

Still required: •  “Show me”

•  Exploratory testing

•  User acceptance testing

Page 26: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

ANATOMY OF A DEPLOYMENT PIPELINE

Release

•  Every change results in a trigger of the deployment process •  Software is build once and only once

•  The same deployment process for every environment •  Deployment in production-like environments

Commit Automated Acceptance Test

Automated Capacity Test

Manual Testing

Fast Feedback

Trust in Application Stability

Done

Page 27: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

EXAMPLE

! Elke stage een slide.

Page 28: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

SUMMARY

•  Continuous delivery is a way to create a stable release process •  Automated testing makes continuous delivery possible

•  Manual testing is (still) necessary •  Pull vs Push

•  Dashboard

Page 29: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

THANK YOU FOR YOUR ATTENTION – ANY QUESTIONS?

Page 30: Succesful Testing - The Continuous Delivery Process

codecentric Nederland BV

[email protected]

+31 (0) 6 24 64 10 33

@huibschoots

[email protected]

+31 (0) 6 51 19 77 38

@mieldonkers

See our website for more information: http://www.codecentric.nl/portfolio/continuous-delivery/

CONTACT INFORMATION