Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV)...

15
Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs 1 Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs 1 Enterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish Agrawal Durgesh Joshi

Transcript of Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV)...

Page 1: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs1Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs1

Enterprise DevOps: Testable Business Value (TBV) driving

Continuous Delivery & Deployment

Amdocs Testing Services

Harsh Agrawal

Ashish Agrawal

Durgesh Joshi

Page 2: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs2

Harsh has 23 years of diverse experience in development, infrastructure and testing, and has been

with Amdocs for 17 years. Throughout his career, he has helped Tier-1 and Tier-2 service providers

across the globe achieve successful delivery of testing programs and TEM services.

Harsh Agrawal

A quality thought leader with 18 years of experience in organizational-wide ‘quality function’

deployment and continual improvements, Durgesh has been with Amdocs for nine years. He currently

supports testing pre-sales and leads the Test Centre of Excellence (TCoE) implementation, Test

Assessments and measurements/benchmarking programs.

Durgesh Joshi

Ashish Agrawal

Authors

Ashish has 19+ years of vast experience in development and testing. Ashish has led several testing

programs with the teams spread across multiple global delivery locations. His testing experience

spans across Telecom, Financial Services, Energy Trading and Risk Management, Public Sector,

Airlines domains.

Page 3: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs3

So everyone develops in Agile. What’s the problem?

Until code is in production, no value is

actually being generated, because it’s merely

WIP stuck in the system.

Page 4: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs4

Problem Statement

Product Backlog

Story -1

Story -2

Story -3,4

Story -5,6,

Story -8,9

Story -10,11

Story -12-15

E2E Testable ? Value to customer?

E2E Testable ?

E2E Testable ?

Value to customer?

Value to customer?

PI 01

PI 02

PI 03

Vendor-1 Vendor-2 Vendor-3

Large Complex Multi-Vendor SI environment

Continuous Delivery

DevOps

Continuous Integration(Silo agile cycles)

Constraint at Continuous

E2E/UAT

(due to the

Iterations/Sprints getting

queued up)

Page 5: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs5

Then Each Development Iteration Needs To

Create Something One Can Actually Test & Deploy

Then We Moved To Agile

Need in todays demanding world..

Development and Testing as Waterfall

Page 6: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs6

Testable Business Value (TBV)

Project PI 2

1-2 Weeks

Project PI 3

1-2 Weeks

Project PI 1

1-2 Weeks

Every PI starts from Scoping in which TBVs are defined

Development & Testing are done continuously per TBV, enabling production deployment of each TBV Separately

TBV

(Testable Business Value)

An entity defined by the Test Architect during scoping, to describe the integrated scope element which the testers must receive in order to certify scope which carries business

value and supports E2E tests

TBV Example A loyal customer with multiple connections, looking for reduced bill amount

due to collective usage via all connections

Continuous Testing • Automated/Exploratory testing activities occur seamlessly in parallel using TBVs

• After each stage acceptance move to the next level and finally to production

Page 7: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs7

ESTABLISH TBV

PO, BA, Test Architects,

Solution Manager

PO, BA

DEFINE BACKLOG

TBV Creation Process

Feedback

CUSTOMER

Gives High Level

Requirements

Dev

Design

Commit

Code

Test Design

Test

Auto. Build &

Test

Acceptance

Test

Staging

Deploy To

Prod

Production Value!!

Page 8: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs8

Actual Example

BR-1 : Setup New Family Plan

for Mobile/Data service with

logic as 1st subscriber under

same account will be charge

actual amount, 2nd

subscriber with same Plan will

get 50% discount, 3rd

onwards will be charged with

20% of actual RC rate

BR-2 : Ability to provide this

offer to BYOD(bring on your

device) device

BR-3 : Ability to provide free

voice service & data charges

having Family Plan

TBV01 : As a CSR, I will be able to change Plan to new

Family Plan for subscribers having existing account, so customer will be eligible to avail this offer of Family Plan of Less charge due to multiple subscription

T1F1: Ability to Setup Family PPs in EPC system

Testable Deliverable Deployable

A loyal customer with multiple connections, looking for

reduced bill amount due to collective usage via all

connections

TBV02 : As a CSR, I will be able to provide this offer to

New customer opting for Family Plan with New device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV03 : As a CSR, I will be able to Provide Family offer

to new Customer opting of Family Plan having their own

device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV04 : As a Family Plan Customer, I will be able to

make unlimited voice Call under same account & all data usage free for such subscribers, so Customer has to pay only for Flat Plan charges for all subscribers under ONE account

T1F2: Ability to show all family Plans on CRM

screen, once Agent select Family Plan button for all existing Customers

T1F3: Ability to upgrade subscriber service with

Family Plan.

T1F4: AR routine should be enhanced to calculate

RC for each subscriber such a way, if its 1st subscriber with Family Plan then 100% RC, 2nd sub

having 50% RC & 3rd PP onwards 20% RC

TB01 - FEATURES

“1”

DEFINE

BUSINESS

REQMTS.

“2”

DEFINE

TESTABLE

BUSINESS

VALUE

(TBV)

“3”

DEFINE

FEATURES

FOR

RELEVANT

TBV

Page 9: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs9

Actual Example

BR-1 : Setup New Family Plan

for Mobile/Data service with

logic as 1st subscriber under

same account will be charge

actual amount, 2nd

subscriber with same Plan will

get 50% discount, 3rd

onwards will be charged with

20% of actual RC rate

BR-2 : Ability to provide this

offer to BYOD(bring on your

device) device

BR-3 : Ability to provide free

voice service & data charges

having Family Plan

TBV01 : As a CSR, I will be able to change Plan to new

Family Plan for subscribers having existing account, so customer will be eligible to avail this offer of Family Plan of Less charge due to multiple subscription

T2F1: Ability to Setup Family PPs in EPC system

Testable Deliverable Deployable

A loyal customer with multiple connection, looking for a

solution as reduced bill by leveraging benefit due to

collective usage via all connections

TBV02 : As a CSR, I will be able to provide this offer to

New customer opting for Family Plan with New device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV03 : As a CSR, I will be able to Provide Family offer

to new Customer opting of Family Plan having their own

device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV04 : As a Family Plan Customer, I will be able to

make unlimited voice Call under same account & all data usage free for such subscribers, so Customer has to pay only for Flat Plan charges for all subscribers under ONE account

T2F2: Ability to purchase device & select Family

Plan on CRM screen for new Customer

T2F3: Ability to attached Family Plan with device &

do activation

T2F4: AR routine should be enhanced to calculate

RC for each subscriber such a way, if its 1st subscriber with Family Plan then 100% RC, 2nd sub

having 50% RC & 3rd PP onwards 20% RC

TB02 - FEATURES

Page 10: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs10

Actual Example

BR-1 : Setup New Family Plan

for Mobile/Data service with

logic as 1st subscriber under

same account will be charge

actual amount, 2nd

subscriber with same Plan will

get 50% discount, 3rd

onwards will be charged with

20% of actual RC rate

BR-2 : Ability to provide this

offer to BYOD(bring on your

device) device

BR-3 : Ability to provide free

voice service & data charges

having Family Plan

TBV01 : As a CSR, I will be able to change Plan to new

Family Plan for subscribers having existing account, so customer will be eligible to avail this offer of Family Plan of Less charge due to multiple subscription

T3F1: Ability to Setup Family PPs in EPC system

Testable Deliverable Deployable

A loyal customer with multiple connection, looking for a

solution as reduced bill by leveraging benefit due to

collective usage via all connections

TBV02 : As a CSR, I will be able to provide this offer to

New customer opting for Family Plan with New device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV03 : As a CSR, I will be able to Provide Family offer

to new Customer opting of Family Plan having their own

device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV04 : As a Family Plan Customer, I will be able to

make unlimited voice Call under same account & all data usage free for such subscribers, so Customer has to pay only for Flat Plan charges for all subscribers under ONE account

T3F2: Ability to validate BYOD into Provisioning

system if Customer comes with this own device

T3F3: Ability to allow Customer opting for Family

Plan for BYOD device & do activation

T3F4: AR routine should be enhanced to calculate

RC for each subscriber such a way, if its 1st subscriber with Family Plan then 100% RC, 2nd sub

having 50% RC & 3rd PP onwards 20% RC

TB03 - FEATURES

Page 11: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs11

Actual Example

BR-1 : Setup New Family Plan

for Mobile/Data service with

logic as 1st subscriber under

same account will be charge

actual amount, 2nd

subscriber with same Plan will

get 50% discount, 3rd

onwards will be charged with

20% of actual RC rate

BR-2 : Ability to provide this

offer to BYOD(bring on your

device) device

BR-3 : Ability to provide free

voice service & data charges

having Family Plan

TBV01 : As a CSR, I will be able to change Plan to new

Family Plan for subscribers having existing account, so customer will be eligible to avail this offer of Family Plan of Less charge due to multiple subscription

T4F1: Ability to make voice call & record in Rating

table with free charges if subscribers are belongs to Single Customer

Testable Deliverable Deployable

A loyal customer with multiple connection, looking for a

solution as reduced bill by leveraging benefit due to

collective usage via all connections

TBV02 : As a CSR, I will be able to provide this offer to

New customer opting for Family Plan with New device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV03 : As a CSR, I will be able to Provide Family offer

to new Customer opting of Family Plan having their own

device, so Customer will be charged as per family Plan setup & he will need to pay less for multiple subscription

TBV04 : As a Family Plan Customer, I will be able to

make unlimited voice Call under same account & all data usage free for such subscribers, so Customer has to pay only for Flat Plan charges for all subscribers under ONE account

T4F2: Ability to capture free RC for all data usage if

customer belongs to Family Plan

T4F3: Ability to allow Customer opting for Family Plan for BYOD device & do activation

T4F4: Billing Routine to enhanced to Calculate Flat

charges for all customer belongs to Family Plan and actual bill to show the impact on the change

TB04 - FEATURES

Page 12: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs12

Test Architect Role

• Customer Engagement

• TBV Creation

• 1st handshake with

Testing Team

• Automation

Recommendation

• Proactive Ownership

Scoping

• Post Scrum testing

review

• Participate in

Customer demos

• Teamwork

Testing

• TBV driven iteration planning

• Define Scrum testing ability

• Scrum testing coverage mapping

• Participate in Scrum demos

• 2nd handshake with testing teams

Construct

Page 13: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs13

Results

Value to customer

Product Backlog

Story -1

Story -2

Story -3,4

Story -5,6,

Story -8,9

Story -10,11

Story -12-15

E2E Testable!! Value to customer

E2E Testable !!

E2E Testable !!

Value to customer

PI 01

PI 02

PI 03

Vendor-1 Vendor-2 Vendor-3

Large Complex Multi-Vendor SI environment

Continuous Delivery

DevOps

Continuous Integration(Silo agile cycles)

Continuous E2E/UAT

Sprints getting moved to

next level)

Page 14: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Information Security Level 2 – Sensitive

© 2017 – Proprietary & Confidential Information of Amdocs14

Using BEATTM to enable continuous testing and shift-left

Enabling DevOps

Test-driven

Projects

GINGER Automation

Mobile Reports

Amdocs BEATTM Analytics Analytics

for quality,

speed and

efficiency, dev

and

test metrics

Immediate

feedback;

real-time

test status

DevOps

automation

console:

requires

no coding

knowledge

Amdocs BEATTM Cloud

Design Console

A-TEAM

Methodology-

driven,

cross-site project

management

Business flow

creation broken

into testable

units

Dynamic

environment

configuration

and

management

DevOps - Test-Driven Delivery

Test Data & Environment ManagementDynamic definition and supply in real time

AutomationEliminate Manual Testing

Test Driven DeliverySynchronized Dev-Test Assembly line

Page 15: Enterprise DevOps - Software Testing ConferenceEnterprise DevOps: Testable Business Value (TBV) driving Continuous Delivery & Deployment Amdocs Testing Services Harsh Agrawal Ashish

Thank you