W17  DevOps  &  Testing  5/4/16  15:00              

The  Road  to  DevOps:  Data,  Environment,  and  Test  Automation  

 Presented  by:  

 

Tanya    Kravtsov  

Audible    

Brought  to  you  by:        

   

   

350  Corporate  Way,  Suite  400,  Orange  Park,  FL  32073    888-­‐-­‐-­‐268-­‐-­‐-­‐8770  ·∙·∙  904-­‐-­‐-­‐278-­‐-­‐-­‐0524  -­‐  info@techwell.com  -­‐  http://www.stareast.techwell.com/      

 

       

Tanya    Kravtsov  Audible    Tanya  Kravtsov  has  recently  joined  Audible  as  a  Director  of  QA.    Previously,  as  the  head  of  automation  and  continuous  delivery  at  ROKITT,  Tanya  focused  on  internal  Automation  and  DevOps  practices  and  worked  with  customers  to  transform  their  development  and  testing  processes.  Tanya  is  passionate  about  process  automation,  which  encompasses  test,  data,  and  environment  automation,  as  well  as  continuous  integration  and  continuous  delivery.  She  is  a  founder  of  the  DevOpsQA  NJ  Meetup  group  and  a  speaker  at  events  including  QUEST,  TestBash,  and  the  Jenkins  User  Conference.  Follow  her  on  Twitter  @DevOpsQA.    

   

The Road to DevOps: Data, Environment, and Test Automation

Star East 2016

Agenda

Introduction

Definition of Continuous Delivery

10 Steps

Target State

Tanya Kravtsov

• Director of QA at

• Head of Automation and Continuous Delivery at

• Founder of DevOpsQA NJ Meetup group

• Developer turned Tester

@DevOpsQA

Definition of DevOps

ULTURE

UTOMATION

ONITORING

HARING

Star East 2016

“Any improvements made anywhere besides the bottleneck are an illusion”

Identify Bottlenecks - Why does it matter?

Bottleneck Identification Techniques

Play the Speed Boat Game

Identify Anchors holding back your process

Draw the Workflow

Identify all players in the systems

Identify all handovers between the players

Communicate

Brainstorm

Mind Maps

Interviews

Retrospectives

Star East 2016

Automated Environment Setup process

Infrastructure as Code

Automated Configuration Management

Virtualization

Infrastructure Monitoring

Availability Monitoring

CPU Utilization

Test Environment Setup and Monitoring

Automated Smoke Test

Automated Environment Readiness test

Application Server

Web Server

Database Server

Automation Framework Artifacts

Authentication

Virtual Machines

Automated Product Readiness test

Validate critical functionality

Star East 2016

Automated Test Data Management

Data Discovery

Synthetic Test Data

Data Subsetting

Data Masking

Data Cleanup

Integrated Automation and Test Data solution

Utilize TDM tool APIs to retrieve data at runtime

Generate fresh data before each execution

Automatically clean up data at the end of the run

Test Data Management

Star East 2016

Automated Unit Testing

Unit Tests

Short independent tests

Tests only fail when a bug is introduced

When tests fail, we know the reason

TDD

1. Write Test First

2. Write enough code to make the first test fail

3. Fix the code to make the test pass

4. Repeat

Automated Build

Automate

Compiling computer code into binary

Packaging binary code

Deploying to test and production environments

Creating documentation

Setup CI Server

1. Install Jenkins CI

2. Configure Jenkins Server

3. Install Jenkins Plugins

4. Configure Version Control System

5. Setup Jenkins Job

Automated Code Quality Metrics

Static Analysis

Security vulnerabilities

Common and critical defects

Coding Standards

Memory Leaks

Code Coverage Analysis

Measure Unit Test Coverage

Measure application-level test coverage

Code Reviews

Pull Requests

Pair Programming

Automated Regression

Test Automation

Automated Web Services Testing

Automated UI Testing

Parallel Execution

Virtualized Test Lab

Central Management Dashboard

Automated Test Data Management

Retrieval

Generation

Cleanup

Selective Testing

Pairwise testing

Searchable Test Case Repository

Star East 2016

Automated Analysis and Reporting

Automated Test Results Analysis

Consolidated Results

Failure Analysis

Defect Logging

Automated Results Reporting

Automated Email Notifications

Reporting Dashboard

Excel, PowerPoint, ALM

Root Cause Analysis

QUEST 2016

Automation and DevOps Myths

Testing can be 100% Automated

Automation is a lot faster than manual testing

Automation should be maintenance free

Automation can be created without any scripting

Adapting DevOps will result in 50 builds a day fully tested and ready for production

21

Target State

Communication