Intuit Confidential and Proprietary1

INTEGRATIONCONTINUOUS TESTING

DELIVERYAND THE ROLE OF QUALITY ENGINEERING

CI/CD is the process of continuously releasing quality code to production with confidence.

Tools , Process and People are all critical part of the CI/CD model

Continuous Integration

Continuous Testing

Continuous Deploymen

t

SDLC AGILE CONTINUOUS DELIVERY DEVOPS

Empowered and engaged engineers – able to deliver awesome to customers

CI-CT-CD implemented well looks like this … (Ideal State)

Predictability and fast release cycles (in days) Integration and release is a non-event, happens

everyday ZERO production escapes 99.9 % Uptime Day 1 productive new hires

QUICK POLLHow many of you are working on CI/CD strategy in your project/organization and for how long?

Less than 6 months1 Year

More than 1 YearCompletely Done

Everyone wants CI/CD, Everyone’s doing CI/CD

– Applying CI/CD into existing development flow– Ad hoc process implementations – Standalone view, CI or CT as a standalone practice

But very few are getting it right …. Why?

– Short term view of tools and process – Tribal knowledge– Relapse to the former ‘roles’ and habits– Creating fear-driven, risk averse development

– Framework is too flexible or too rigid for effective adoption– Tools fragmentation – cohesiveness missing

CI/CD @INTUIT A FEW YEARS BACK …

PD Team 4…NPD Team 3

Developer

SCM

CI

DeploymentOrchestration

Tools/Release Engineer

OperationsDefect Tracking

PCET

Team 1Team 2

TestAutomation

QE Engineer

Defects are siloed and using different definitions leading to inefficiency and/or inability to resolve issues end to end

Effort/time wasted maintaining different repositories introducing risks and making it hard to contribute across Intuit.

Duplication of recipes and varying implementations (and degrees of maturity) are delivering limited benefits and low re-use

Different levels of automation, using many different technologies, resulting in low productivity and scalability, and inability to resolve issues end to end

Access to public cloud severely limited and onboarding requires a lot of coordination

We cannot deliver at the scale, speed and ease as a platform and ecosystem company

Scale• 3000+ engineers • 1000 source repos• 10K builds/Test jobs• 100s of CI clones • 5+ instances of tools

Side effects • Fragmentation• Duplication/

Snowflakes• Siloed culture • Not platform centric• Productivity loss• Cost/Effort high• ~20% prod. Leakage

Internal hosting

CURRENT: CI/CD STRATEGY

Design Principles

• Tool agnostic• Modular • Fast and Secure• Standard

platform, secure with flexibility

• Toolchain-ed• Self-service

Tools Portal Process Orchestration

SCM Services(Source, config, binaries, artifacts, …)

DeploymentOrchestration

RuntimeAutomation

TestExecutionFramework

FunctionalTesting

PerformanceTesting

Dashboards, Quality, Logs, M

onitor, Issues

MobileTesting

AppSubmission

FutureCapabilities

ContinuousIntegration

SecurityTesting

commit

Release

Internal Hosting

CI/CD: TOOLING STANDARDS

Develop Build Deploy Test Release

Martini

iTAG is the quality engineering x-team that standardizes testing tools, frameworks and quality standards across Intuit

CI/CT/CD SOLUTION ARCHITECTUREEngineering Principles

• Scale for entire dev. community

• HA/DR• Multi-tenancy• Hybrid cloud• SOA• Buy+OSS v/s build• Open contribution• Fairly obvious,

difficult to make mistakes

CI

CT

CD

Test Execution Platform (AWS/IHP)

nCI/CD

Testers

Test case/Impact management

TEP client/ Martini

Test data management

Analytics

BrowsersDevicesServers

iOTsimulators

Test controller

One set of Standards and Adoption(Automation, standards, onboard)

Enterprise ready and best-in-class (Scale, faster, accurate, reliable)

Truly CI/CD(Feedback loop, feature flags, shorter release cycles)

CI-CT-CD EVOLUTION @INTUITMULTI-YEAR JOURNEY - ONE STEP AT A TIME

HOW TO START ON THE CI-CT-CD STRATEGY FOR YOUR PROJECT?

• Take stock of where you are. List down what happens from the requirement/story stage to production release

• List down steps are automated, tools used, human interventions and the most painful processes

• Talk to each stakeholder to understand their release criteria e.g. Product manager, Performance engineer, QE

• Define optimal release cycle time for your project. Define your pipeline

• Finally, look for exceptions/tailwinds that can slow you down.CI-CT-CD – Big project but take one step at a time …. Bite the bullet

ROLE CHANGES:QUALITY ENGINEERING

• FOCUS Go from tactical to strategic or consulting role Non Functional Testing to be handled by dedicated specialists End to End testing, standards and governance Automation , Automation, Automation !

• MINDSET Everyone in the team is a software engineer – no distinction of roles Ownership & accountability of quality lies with entire team Developing Unit and Functional Test is every Software Engineer's

responsibility

• ROLE ‘Quality Engineers’ enable infrastructure for ‘Software Engineers’ to write Tests

EVOLUTION OF QE INTO 3 AREAS

• Product Specialists Group

Lean resource pool of domain experts involved in managing end-product testing Measuring product feature coverage, calculating risk impact - influencing end user

testing patterns Standards and governance

• Engineering Productivity Group

Develop test enabling infrastructure at various levels Build & configure various quality monitoring & developer productivity tools

• Non Functional Test Specialists Group

A ‘Center of Excellence’ for non functional assessment Evolve QE into SMEs in ‘Ilities’ like performance, security, reliability, usability, serviceability testing

THANK YOU!

Sushma ChakkiralaGroup Manager, Developer Tools

Email: sushma_Chakkirala@intuit.comLinkedIn: https://www.linkedin.com/in/sushma-c-5443873

Twitter: @sushmac

Soumi DattaDirector of Quality and Ops

Email: soumi_datta@intuit.comLinkedIn: https://www.linkedin.com/in/soumi-datta-a8baa22

Q & A

APPENDIX