Integrating effective testing & automation for successful ... · Integrating effective testing &...

1© CSS Corp | Confidential | www.csscorp.com© CSS Corp | Confidential | www.csscorp.com

Integrating effective testing & automation for successful adoption of DevOps

20th May, 2014SCQAA-Orange County, CA

2© CSS Corp | Confidential | www.csscorp.com

Authors

Anil Parambath

Vice President - Solution Engineering

PRabhu Meruga

Director - Solution Engineering


Agenda

• What is DevOps-History, Evolution and Prominence

• Why DevOps now? Today’s Application development needs

• Deployment statistics of todays modern world

• Build Failures impact

• Development life cycle survey results

• Challenges – Collaboration, Release and Integration testing

• Drivers for DevOps

• Elements of DevOps

• Testing framework for DevOps

• Integrating continuous testing into DevOps – understand the differences

• how to integrate testing into DevOps? What are the tools available

• Difficulties, Barriers & Benefits

• Q&A


DevOps – history, evolution and prominence

Development

(software engineering)

Technology Operations (IT teams)

Quality Assurance

(testing)DevOps

Started in Belgium as “DevOps”

days in 2009 by Patrick

• Cross departmental integration concept

• Idea • Culture• Practice


Build, listen. Mash-up scenarios. Mobile first

Develop Fast. Succeed Fast

Ease of use. Avoid feature overload. Don’t replace Simplicity

Speed

Innovation

Quality, Performance, UsabilityCoverage

Why DevOps? Today’s dynamic application development needs


Deployment challenges of modern world

• Social networking sites such as face book on average have minimum 2 builds per day and maximum 7 builds per day

• Open source code sharing firms like github.com have 500 builds/day and 100 deploys/day


Development methodology survey results from Agile conference

Gartner: By 2015, DevOps will emerge as niche strategy employed by large cloud providers into a mainstream strategy for 20% of the global 2000 Organizations.


Collaboration challenges between Dev, QA and Ops

Development completed

Deployment completed

Incorrect version deployed

Checked it again – it’s the right version

Oops – we picked the wrong code

Now we got the right one

Scen

ario

1Sc

enar

io 2 Request for new

environment setupPlease wait we are busy

Scen

ario

3

Request for new environment setup

I have daily buildCannot support daily build

Consistent build deployment needed

…and many other scenarios….


Release flavors

Hourly Daily Weekly Fortnightly Monthly Quarterly Half Yearly Yearly

Business Critical systems-Social Media, Space, Defense, e-Commerce, financial

• Some of the changes in business critical systems are pushed on an hourly basis

• Release vary based on the criticality up to quarterly releases

Large scale ERP updates, Operating system patches, Anti virus updates, Embedded systems, traditional internal applications

• Some of the changes are very slow depending on the nature of business and components involved. Eg Hardware/embedded


Integration testing becomes more complex

Dev1

Dev2

Dev3

Continuous Integration server

Submit Code branch

ERP

ch

ange

sPa

tch

u

pd

ates

Mo

bili

ty

enh

ance

me

nts

Deployment

Production Farm

QA Lab

Performance Lab

UAT Lab

Operations• Lack of controls in Parallel Dev methodology

• Heterogeneous channels – PC, laptop, Mobile

• Systems Integration – Mobile apps, ERP, Web, Client

• API testing ( too many bugs escape downstream)

• Lack of performance engineering & automation

• Multiple environments to setup and maintain• Huge number of production applications

monitoring• Lack of visibility into customer experience


Drivers for DevOps implementation

My enterprise has waterfall modelWe are in agile mode

• Need for on the fly multiple environments creation and maintenance

• Code synchronization challenges across environments

• Business demanding frequent releases to catch up with Users demand

• Increased usage of mobile devices and applications

Dri

vers


Elements of DevOps

Continuous Integration

Continuous Deployments

Test Driven Development

Continuous Automation testing


DevOps can resolve the challenges and ensure quality of applications.

• Continuous integration tools- Jenkins, Selenium grid

• Automated build deployment process• Automation build

qualification(testing)

Developers

IT Operations

QA/Testing team

User Community

• Automated monitoring of the deploys

• Jump into action in case of deadlocks

• Automated daily builds into production• Faster release cycles for new features• No dependency on IT operations human

resources• Alert management systems


With and Without DevOps – understand the differences

• Automated monitoring of the deploys leveraging DevOps tools

• Has visibility into application monitoring for better root cause analysis

• Reduces dependency on operations

• Very focused on development and coding

• Line of responsibility does not encourage cross skill set training

• Raises tickets to deploy code, monitor application logs, set up environment

• Very focused on waiting for the code deploy

• Line of responsibility does not encourage creating code snippets for any kind of automation

• Raises tickets to deploy code, set up environment

• Continuous automation testing for regression

• Has visibility into security testing and application monitoring

• On the fly environments creation for QA needs

• Very focused on operations alone

• No perspective on developing deployment scripts

• Low focus on validation and purpose of monitoring

• Takes time to create environments

• Motivated to create or design code for various support activities

• Environment creation and monitoring made easy

• Roll back, Patch updates – no dependency on Development teams

Complements additional skill set

Sharing of efforts and cross pollination DevOps

Without DevOps – Silo operations

Dev QA Operations


Testing framework for DevOps implementation

Development(agile/iterative)

QA Testing(PC, Laptop & Mobile)

UAT Production

Build Failure

Build Pass

Defect Management Tool & process

QA Pass UAT Pass

Test Cases repository (Automated scripts) –leverage Selenium, Test link integration

Continuous integration coupled with continuous automation


What will it look like after DevOps implementation?

Sprint 1 Sprint 2 Sprint N

Development

Functional AutomationNon Functional Automation

Automated Unit Testing





Continuous Automated regression testing

Continuous Automated integration testing

Release

…..

…..

………


Tools for DevOps implementation

Deployment automation : Fai, KickStart, preseed, Cobbler

Configuration management : Puppet, Chef, cfengine, bcfg2

Build Automation :Jenkins, ant, Cruise control, Hudson

Test Automation : QTP, Autoit, Testlink, Test Complete, CA LISA Test


How do I start Dev Ops implementation

Perks and Benefits – Starbucks cards etc

Join Developer, QA and Operations team members

Start small with a mini project, assess the success factors


Difficulties & Barriers for implementing DevOps

Common Cultural barriers• 49 percent of respondents who

had no plans to implement DevOps identified a lack of manager buy-in as a blocker—this was the most frequently identified barrier for this group, followed by "lack of team buy-in" (38%).

• 48 percent of all respondents indicated that one of the biggest difficulties in implementing DevOps was that the value wasn't understood outside of their group.

Source: Puppetlabs


Benefits of DevOps – What kind of metrics will help assess DevOps?

• Achieve 25 times more frequent code deployments

• Reduce build failures/outages by 60 times

Key Metrics• Number of Deployments• Number of Successful Builds• Change Deployment Lead time• Validation time per change• Quality of deployed change


What is skill set needed for DevOps teams?

Pre-requisites: Cross pollination/training of Developers, IT and QA team

People Management Skills

Version Control Systems/Automation testing tools


Where was/is DevOps implemented? Success stories!!!

US elections: Scott VanDenPlas, the head of the Obama's

DevOps group, put it"4Gb/s, 10k requests per second, 2,000 nodes, 3

datacenters, 180TB and 8.5 billion requests. Design, deploy, dismantle in 583 days to elect the President”

Source: http://gizmodo.com/5960790/inside-obamas-team-techExtract from Keynote, Velocity Conference, Santa Clara, Jun 16, 2011 by Jenkins, Jon;

Amazon AWS: 10,000 maximum deployments/hour; Meantime between deployments: 11.6 seconds;Outages due to build failure: 0.0001%;

http://gizmodo.com/5960790/inside-obamas-team-tech


Big Picture – in simple terms

Each man, in isolation, examines a different area of the elephant — the leg, the tusk, the tail, and so on. When they compare observations, they discover that their conclusions are completely different. Only when they collaborate and bring their ideas together do they begin to understand the full picture.Source: BMC.com


FAQs

1)Is there a benchmark of team size to start implementing DevOps?

2)Is Agile same as DevOps?3)Does DevOps mean Developers learning

testing and operations skill set and vice versa?

4)What will happen to the core skill set of the team members?

5)How is application quality and time to market improving through DevOps?


Q&A


© CSS CorpThe information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.

Thank You


Upcoming Webinar from CSS Corphttps://www.csscorp.com/events/are-you-ready-mobile-revolution

Integrating effective testing & automation for successful ... · Integrating effective testing &...

Documents

Transcript of Integrating effective testing & automation for successful ... · Integrating effective testing &...