The Shift to Quality Engineering - Accenture€¦ · QA is asked to delivery transparency into root...
Transcript of The Shift to Quality Engineering - Accenture€¦ · QA is asked to delivery transparency into root...
The Shift to Quality Engineering
Bart Gourley
2016 Accenture Testing Symposium
February 2016
2Copyright © 2016 Accenture All rights reserved.
Today’s
Discussion Industry/market pressures and the quality
challenge
Trending responses for testing organizations• Structural transformations (enabling a quality
factory)
• Rollout of new roles
• Shifts to new/different tools
What’s next?
Industry change in is disrupting internal quality process, putting some testing organization at risk if they fail to adapt.
Industry/Market Pressures And the Quality Challenge
Challenges for Testing OrganizationsIndustry Drivers of Change
Growth of new and potentially disruptive
B2C IT products in historically stable
markets
Increasing business agility requirement pressures from dynamic client demands
Continued downward cost pressures
• Stakeholders: Integration with development, business, and
customer partners within the quality workflow often requires
continuous and active collaboration through non-IT
traditional methods (e.g. waterfall vs. agile/lean)
• Methodologies: Structured multi-speed IT test
methodologies (e.g., internal industrialized agile, waterfall,
extreme, hybrid approaches) are required where they often
do not exist.
• Focus: Engineering ‘for’ quality is replacing testing for
quality. QA is asked to delivery transparency into root
causes of quality impacts both up and downstream from
testing (e.g., shift left and shift right).
• Skills: New/Non-traditional QA roles and skills (e.g., SDET,
Techno testers, Quality Czar, etc.) are required.
• Value: The challenging economic environment retains a
focus on higher value SDLC activities. However, innovation
has been slow to respond to creating more value at lower
costs.Crucible for Transformation
• Uber and Shared Car
Services
• Cable and Content
Streaming Services
• Theaters and Direct to
On Demand
• Market feedback
rates have
increased (app
store, real-time
surveys, etc.)• Budget alignment
with product market
performance and
value
Copyright © 2016 Accenture All rights reserved. 3
Quality organizations are seizing the moment to evolve — not simply to survive, but to elevate their value.
Trending Responses for testing organizations
Copyright © 2016 Accenture All rights reserved. 4
Challenges What actions are others taking? What benefits are they seeing?
• Stakeholders
• Methodologies
• Focus
• Skills
• Value
• Test organizations are moving away from being
external to the development team and towards
being part of the engineering team through the
staffing of roles with new skills (e.g., SDET, QA
engineer, etc.)
• Test organizations are shifting left and right to
influence and to sometimes control quality
across the delivery.
• Centralized test organizations are shifting away
from Testing Center of Excellences towards
Quality Center of Excellences and are held
accountable for more than defect leakage
• Test organizations often need to reorganize
from a structure and capability perspective to
support the new engineering models and/or
requirements
• Tighter integration with internal
and external (market)
stakeholders
• Increased perceived value by
stakeholders
• Decreases in UAT defect leakage
and improvement in market
feedback
• Increased transparency into the
true drivers of quality (e.g.,
development code readiness
‘green builds’, requirement
readiness, and crowd acceptance)
Industry is redefining how software is developed, tested, & delivered. Lean engineering is rapidly evolving the QA space.
Trending Responses for testing organizationsStructural Transformations > Enabling Quality Engineering
Copyright © 2016 Accenture All rights reserved. 5
Core Concepts• Quality is a by-product of the engineering process, not a point-in-time
function
• Quality activities are deeply embedded into release engineering
(DevOps), product performance (business/market definition and
success monitoring), and technical development
• Quality organizations and their activities are lean at the core and scale
variability for execution
Common Traits of Mature Quality Factories• Technical level quality activities (TDD, BDD, etc.) are formalized and
SDETs (or the equivalent function) are prolific supporting these
functions
• Quality oversight and control is injected into each part of the SDLC
• Leaders in the space envision the SDLC as a lean
process that, like any other manufacturing process, has
waste that should be reduced to optimize for cost, time to
market, and market relevance.
• Shifting left and right removes traditional test stages
and gates and blends quality measures into each function
of the engineering process.
The intersection of DevOps, Traditional QA, Development, and Market Feedback
A holistic quality engineering approach places more testing and automation left in the delivery workflow.
Trending Responses for testing organizationsStructural Transformations > Example Run State
6Copyright © 2016 Accenture All rights reserved.
Forming Norming & Storming Performing
• The SDET term is applied to any
non-development/test resource
that is providing testing within the
development team.
• Typical Environmental
Factors/Prerequisites: None
• The SDET term is applied to any engineering
resource that is providing low-level automation
and test coverage that is stored within the source
code repository.
• Typical Environmental Factors/Prerequisites:
Development and development integration
leveraged in testing
• The SDET term is applied to any engineering resource that
is providing low-level automation and test coverage that is
stored within the source code repository, providing
coverage and execution integration into the continuous
delivery solution, that collaborates with integration level
validations from external teams and that delivers within an
agile methodology structure.
• Typical Environmental Factors/Prerequisites: Continuous
Integration solution is leveraged
Establishing a Quality Czar and SDET capability is a journey that typically starts with identifying muti-functional/’utility’
resources and ends with those resources engineering portions of the production solution*.
Trending Responses for testing organizationsStructural Transformations > ‘New’ Role Definition
Copyright © 2016 Accenture All rights reserved. 7
* Conceptual and for discussion only
- Quality Czar: SDLC experienced resource that has a mixed skill background of PM, QA, Development, and
Release Engineering. The resource provides governance and input into all engineering activities to drive for quality
outcomes
- SDET: technical testing resource that frequently provides low level automation and manual testing services within a
sprint team.
- QA Analyst: functional testing resource that responsible to provide behavior and/or business transaction validation
of the solution within an integration team.
SDET Journey Example
Example role functions within an DevOps driven development workflow
Trending Responses for testing organizationsStructural Transformations > ‘New’ Role Definition
Copyright © 2016 Accenture All rights reserved. 8
SDET Activity
QA Analyst Activity
Developer Activity
Automated Activity
Test driven development:
Developer cuts feature branch,
writes test case, tests, writes
code, tests and submits for review
Gulp provides
Automated task
scripting
Bamboo begins
automated build &
deploy process
JS Hint, JSRC, HTMLLINT rand
other code quality run and
perform analysis
Feedback is passed to
developers
QA Testers test on multiple devices
(Automated Regression (selenium) +
Manual Functional)
Once code review is approved, Git
with Atlassian Stash is used to
merge the code update with
development branch
Cod e artifacts, including applications are
stored/retrieved in NPM and Bower JSON
used to integrate with Rest API
Gulp runs automated Karma
behavior with Jasmine assertions
and test reporting with Istanbul
Gherkin
Behavioral driven development:Automation tester writes Gherkins scripts that become inputs for drafting unit test cases.
Istanbul
Test results are sent to
SonarQube server for analysis
Automated Smoke Testing (Build Validation)
Code is packaged & readied for
deploy by Gulp
Code is deployed to environments by
Bamboo via Docker
Analysis of testing
results and
determination of follow-
up / action items
Creates automated and manual tests for
continuous validation activities
Creation and execution of E2E, full
integration, performance, crowd,
and other testing.
Creates and integrates unit tests into
code base
Quality Czar
Tooling needs to support a quality engineering workflow blend functional traditional and technical development tools
Trending Responses for testing organizationsStructural Transformations > Shifts to new/different tools
Copyright © 2016 Accenture All rights reserved. 9
* Conceptual and for discussion only
Tooling focus is on supporting the
continuous and often automated
delivery and analysis of quality
measurements.
Continuous
Delivery
Architecture
Relevant
Tooling landscapes are evolving to
support a structure of core+. The
core is centrally supported where
as the + is supported only within
the scrum or application stack.
Cloud Enabled
Tooling is more frequently on-
demand and cloud supported either
through SaaS or frameworks that
leverage dynamic infrastructure
Tooling scope and breath looks at
the entire SDLC and promotes
adoption of requirement capture
and traceability to deployment
performance. Example: BDD
Cucumber & JBehave.
Workflow
Questions?