Sigist 2012 Managing your Automation for successful...
Transcript of Sigist 2012 Managing your Automation for successful...
Sigist 2012 –
Managing your
Automation for successful
implementation
Erez Shacham
Automation & Tools
Director
Agenda
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 2
Test Automation introduction
Automation Methodology Life Cycle
Automation Implementation approach
Case study
Test Automation objectives
Shorten time to market (TTM)
Reduce cost of the project
Increase test coverage – Enhance testing scope
Improve quality
Increase efficiency
Improve quality
Shorter TTM
Reduce cost
The Challenge of Automation – Cont.
● Automation scope definition
● How to reduce Automation development effort
● ROI Calculation
● Involvement of manual testing team in Automation
activities
● Automation package readiness for execution
● Use of Automation packages during execution
Agenda
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 6
Test Automation introduction
Automation Methodology Life Cycle
Automation Implementation approach
Case study
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 7
Automation Methodology Life Cycle
Assessment Questioner
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 8
● Request information on manual procedures
● Identify pain points
● Share repetitive work
● Understand technology
● Which tools should we interact with?
● Should we support Java, Power Builder Etc?
● Web Testing
● Mobile Device Testing
Estimations and ROI
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 9
● Evaluate effort and potential ROI
● Based on complexity
● Create and use golden numbers and best
practice experience
Plan Automation - Dashboard
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 10
● Commit to timeline
● Full transparence to the teams you work with, one centralized file which covers all aspects
● Scope under test
● Progress
● Due dates
● Downtime
● Open Issues
Design and Preparation Phase
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 11
● Preparation
● Install \ Validate Tool Automation and AddIns
● Install Framework / ASAP
● Test automation environment configuration
● Connect to management testing tool
● Review each test case, within the scope, define the following:
● Prerequisite
● Script Parameters
● Error handling and recovery mechanism
● Internal script algorithm (Structure and Flow)
● Scripts dependencies
● Data table structure and content
● Verifications list
● Test Data structure preparations
Development Phase
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 12
● Skeleton Scripts – recorded and/or coded
● Tool Automation Coding Standards (Documentation, Naming Conventions, Declarations, Functions)
● Error Handling
● Synchronization Points
● Verification Points
● Common Data Files
● Reporting Results
Unit Test Phase
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 13
● Should be performed for each test script/function separately
● Each script is unit tested in Tool Automation debug mode
● Scripts are tested with all relevant input data options
● The execution report is reviewed
● Both negative & positive cases are checked
Integration Test Phase
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 14
● Check the integration between the scripts (i.e. scripts that share the same data) to make sure that the test set runs as one suite
● Support intermediate failures (this is important for the overnight run)
● Verify test environment
● Integration testing execution is done from Test Management Tool
● Review entire results report
Knowledge Transfer
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 15
● Train Manual team
● Present developed scripts and excel – Data Prep
● Walkthrough shared folder structure
● Configuration files
● BUS and GUI per version
● Log folder
● Review user guide
● Login to Test Management tool
● Create New calendar
● Configure Driver Script
● Initiate execution
● Trouble Shooting
● Maintenance ID SKIP TEST_NAME PARAM0 PARAM1 PARAM2 PARAM3 PARAM4 PARAM5
1 Create Contact Customer with Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME
2 fBusCreateCustomer Contact Customer PostPaid Y Mrs. @FIRST_NAME_CUST1
3 Create Account Customer with Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME
4 fBusCreateCustomer Account Customer PostPaid Y Mrs. @FIRST_NAME_CUST2
5 Create Contact Customer without Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME
6 fBusCreateCustomer Contact Customer PrePaid Y Mrs. @FIRST_NAME_CUST3
7 Create Account Customer without Billing Profile ACTION CUSTOMER_TYPE PRODUCT_TYPE NEW_CUST_INDTITLE FIRST_NAME
8 fBusCreateCustomer Account Customer PrePaid Y Mrs. @FIRST_NAME_CUST4
Knowledge Transfer - Execution
Test Execution
Pre Execution
• Preparations
Execution
• Run Test
Post Execution
• Analysis
Data Preparation -Test Case Example ● TC Name: “Search for a subscriber”
1. Log in Successfully
2. Subscriber phone # found
3. Details found
• Payment Options and User Information
Execution
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 18
● Execute complete packages
● Identify any intermediate failures
● Investigate the failure
● Update excel to resolve
● Re run entire flow
● Raise defects
● Review entire results report
Automation - Manual
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 19
Man
ual
Au
tom
ati
on
Scope Analysis
Plan Design &
Prerequisite Development & Unit Test
Integration Knowledge
Transfer Execution Maintenance
Share
• Details on
pain
points
• Repetitive
work
• Manual
effort
• Test
Cases
Update with
• Roadmap
• Timeline
Provide
• Business Knowledge
• Walk through on tested flows
• Env.
Participate
In status meetings
Be part
of scripts validation
OR
Participate
In status meetings
Learn How to
• Execute automated packages
• Investigate failures and resolve
• Maintain
Execute
Automated packages
Update Scripts
OR
Involve
Automation expert
Contributor
Participant
Depend – Customer’s decision which mode
Agenda
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 20
Test Automation introduction
Automation Methodology Life Cycle
Automation Implementation approach
Case study
Automation Implementation Approach Overview
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 21
Gradual Development
with periodic delivery
to manual testing team
> Environment Checkers
> Sanity – Minimal
Acceptance Test
> Data Generation
> Front End; BackEnd /
SOA automation for
Core Regression
Comprehensive Coverage of
Core Regression
Main Business Activities
Entity Creation
Common Functions Library
APIs
Data
Management
Shared
Package
Initialization
Tools
Version
Control
Reports
Interfaces
ASAP- Amdocs Shared Automation Package Amdocs Layers approach
Application Under Test
Automation Tool
ASAP
Amdocs Shared Automation Package
GUI Functions \ BPT - Business components
ATI = Application Test Interface
BPT- Business Process
22
ASAP – In house developed framework Structure and Methodology
● A collection of foundation scripts and solutions developed over the years to support Amdocs automation testing teams in different projects all over the world
> Calendar file consists of
primarily 2 tabs
> MAIN – Consists of test
cases along with variable
names and fixed data
● KEEP_REFER – Consists of
all the variable data which
changes as per ENV and also
data which is created at run
time like: BAN number,
Subscriber number
Amdocs Automation Framework Parameterization and flexible design
Agenda
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 24
Test Automation introduction
Automation Methodology Life Cycle
Automation Implementation approach
Case study
Automation
PSO
Manual
Tier 1 Quad Play - Case Study
Raise the needs
Brainstorming with
Manual team
Analysis of the
challenges and
search for
available solution
Solution validation
and
Implementation
Engage
Automation team,
Define
requirements
Utilities and
automation scripts
Development
Work with client
business and
testing tool team
Support
Tier 1 Quad Play - Case study – Cont.
Activity Original
Effort
New
Effort
Savings
Execution 157 73 84
157 MM
73 MM
0
20
40
60
80
100
120
140
160
180
Original New
Ma
n M
on
ths
52% Efficiency
•Dry run before testing
cycle
•184 defects were detected
during automated
execution
•Significant Automation
coverage for 1st
implementation
Thank You
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 27
Automation Implementation Approach
Example
Information Security Level 2 – Sensitive
© 2012 – Proprietary and Confidential Information of Amdocs 28
Activity Task September October November December
Regression
AMDOCS and
non Amdocs
Design
Development
Deployment
Execution Sup
Environment
Checker +
Sanity
Design
Development
Deployment
Execution Sup
Data
Preparation
Design
Development
Deployment
Execution Sup