Whitepaper - Automated Functional Equivalence Testing

www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 1

Automated Functional Equivalence TestingA White Paper by: Sagar M. PatilDecember, 2012

WHITEPAPER

http://www.datamatics.com


Table of Contents

3

4

Executive Summary ........................................................................

Automation Approach for Functional Equivalence Testing ....

5

5

1. Design Phase ..........................................................................

2. Script Development Phase ...................................................

6

6

3. Script Execution Phase .........................................................

4. Analysis ....................................................................................

7Functional Equivalence Framework for any Migration Project ..............................................................................................

7

8

Automation Framework for Enhanced ROI ................................

ROI Calculation ................................................................................

8Complete Application Migration is Planned Over Multiple Releases ...........................................................................................

11

12

Conclusion .......................................................................................

About the Author .............................................................................

12About Datamatics Global Services ..............................................

4Typical Migration Testing Challenges .........................................



Executive Summary

There is a growing need for migration from legacy Visual Basic 6.0 to VB.NET or Java technology as Microsoft has discontinued support for VB6. To leverage benefits of new technology and mitigate risk, organizations have started migration of their applications to .NET or Java technology.

Migration projects typically involve extensive testing. The key challenge is to ensure that functional equivalence is achieved on the target application. An Automated Functional Equivalence Testing (AFET) solution therefore becomes a crucial part of the migration process. This is reusable, smooth and cost effective means to achieve 100% functional Equivalence testing on any migration project.

This Paper focuses on methodologies and techniques that Datamatics has developed for automation of functional testing, when a source VB project is migrated to .NET.

For migration projects involving extensive testing, an automated functional equivalence testing solution is crucial



Typical Migration Testing Challenges

When a legacy application is migrated as- is to a newer technology, e.g. VB (Source Application) to Java (Target Application) or VB to .NET, some of the typical challenges faced during testing include:

1. Effort intensive Regression Testing2. Time constraints for development and testing cycle3. Manual testing - being labor intensive, contributes to higher costs4. Insufficient test coverage5. Data consistency verification6. Lack of automation tools for functional equivalence testing

Datamatics’ AFET approach is to use single automation script on source and target platform

Approach for AFET

The functional equivalence tools available in the market support only bitmap comparison and have limited functionality when it comes to verification testing. Datamatics overcomes this constraint through use of a single automation script on source and target platform to achieve functional equivalence verification.

Here is how it works:- The automation script is developed using QTP 11 [Functional Automation Testing Tool]- The business process transaction is scripted on the source platform and then executed on target platform- Any discrepancy from the source application is flagged as a failure on the target application execution- The verification includes comparison test both at controls (buttons, list box etc) and at functional level

The high level steps in a testing process for AFET are:1. Design 2.Script Development 3.Script Execution 4.Analysis

Automated Functional Equivalence Testing Process

Script Dev2

Develop description based script for

reusable for VB as well as .NET

Design1

- Understand functionality- Analyze the

reusable

Script Exec3

First execute script on VB application

and mark results as baseline

Analysis4

Analyse the discrepancies against expected result and

report errors

Figure 3.0



1. Design Phase

a. Source VB “Application under Test” (AUT) is analyzed for identification of reusable functions/actions that need creation of scriptsb. Input and output parameters and dependences are documentedc. Structure of Object Repository is analyzed

The high level steps in a testing process for AFET are:1. Design 2. Script Development 3. Script Execution 4. Analysis

2. Script Development Phase

a. Based on the analysis of reusable actions/functions, either objects from source VB application are added to object repository or a unique description of each object is createdb. Descriptions are created in a manner such that each object is uniquely identifiable on VB (source) as well as .NET (target) applicationc. Once the description of objects is created, object is added to object repository and functional logic is developedd. The process used to develop the Script ensures: i. Report displays total number of controls on form ii. The properties on VB and .NET application (e.g visible\disable), total number of items in List box etc as well as excepted result of functionality are being testede. It is important to note that while the scripts are being created on the VB application and the development of .NET application is in progress, the developed scripts are fully compatible to run on the .NET applicationf. Once .NET application is available, the automated scripts on the source (VB) application after minimal modification can be used for testing on target (.Net) Application

.NET Application under Test

VB Application

Shared Framework Folder with reusable

actions

QTP VB/.NET scripts

Script Execution

Compare Results

Report Errors

Microsoft Visual Basic6.0

®

Microsoft

NET®

.

Figure 3.1



Shared Framework Folder with reusable

actions

VB Application

QTP Tool QTP Scripts

3. Script Execution Phase

a. In script execution phase, automation scripts are executed on VB (Source) application and the results are stored in datasheets that can be later used as a baseline for test verificationb. The same scripts with the data from stored datasheets are then executed on .NET application followed by comparison of results against baselinec. Any deviance from the baseline is analyzed further

Report Errors

VB Application

QTP Tool Script Execution

Base Line Results

QTP VBApp Scripts

Script Execution on Source Application

.NET Application Under Test

QTP Tool Script Execution

Compare Results

QTP .NETApp Scripts

Script Execution on Destination Application

4. Analysis

a. A Root cause analysis of scripts that failed on .NET application is performedb. If script failure is due to functional issues, then it is reported as a defect and sent to the development team for fixes


®

Microsoft

NET®

.


®

Figure 3.2

Figure 3.3



Functional Equivalence Framework for any Migration Project

With this approach, one can create a simple excel based framework for object identification properties and methods to be used by the tool for object identification at runtime.Here is an example of object properties for VB and .NET application

A framework helps achieve better productivity, faster turnaround time and higher accuracy

n automation

Automation Framework for enhanced ROI

ROI for AFET projects can be enhanced through use of frameworks. An automation framework helps achieve better productivity, faster turnaround time and higher accuracy.

For the project being discussed in this paper, Datamatics’ proprietary Automation Regression Testing (DARTS) framework was used:

DARTS© Framework – Key Benefits and features:1. Complete control over the Test Planning, Execution and Test Execution Reporting Cycle2. Technology Specific Component functions to access technology specific features3. Greater control over AUT using built-in feature to check state of the object

Figure 4.0



Better ROI on projects through AFET

4. Reusable Business actions built using component functions, aids to create granular test cases5. Reusable architecture6. Agile framework enables seamless data transfer across business scenarios and better control over execution for the tester7. Recovery scenarios are integrated at the framework level. For example application types like web based, DotNet, VB based etc.

ROI Calculation

For calculating ROI on AFET projects, following factors need to be considered:

1. Cost per test automation resource2. Number of test cases considered for automation3. Automation tool cost4. Time required for creating reusable scripts that work on VB as well as .NET application5. Releases planned per cycle6. Number of regression cycles planned

Complete Application Migration is Planned Over Multiple Releases

DataProduct DetailsSource Product Technology VBMigration Product Technology Vb.NET

DataProduct DetailsSource Product Technology VBMigration Product Technology Vb.NET

Testing Team Size Required For Current ScopeManual 4Automation 2

Release DetailsNo. of Releases Planned 4

Basis of ROI Calculation has been provided below



DataSource Product Technology VBMigration Product Technology Vb.NET

Test Execution DataNo. of Test Cases 500No. of Test Cycles Per Release 2

Manual Testing Effort ParametersTotal Test Cases Available 500Cycle Duration in Working Days 10No. of Working Days 5

No. of Test Cases Increase Per Cycle 50

Working Hours 8Productivity 15

Automation Testing Effort ParametersAvg. Cost Manual/Automation Resource $40 US Per HourNo. of Execution Hours 8Execution Productivity for 1 Cycle 30Execution Productivity for 2 Cycle Onwards 45Automation Tool License Cost $20000 US

Figure 7.0


0

50000

Cycles

100000

200000

150000

300000

250000

Cost

($ U

S)

Cumulative Cost Graph

Cumulative Automation CostCumulative Manual Testing Total Cost


Note: From Release 4 onwards The Cumulative Cost of Automation Testing is lower than the cost of Only Manual Testing

1 2 3 4 5 6 7 8 9 10 11 12

Figure 7.1

Figure 7.2



Conclusion

For projects involving migration of legacy applications to newer technologies, the discussed process for automated functional equivalence testing provide following benefits:

1. Single script can be used to test Functional Equivalence on Source and Target Application2. Script execution is consistent , repeatable and accurate3. Reduction in development and maintenance efforts for automation scripts. The automation strategy and frameworks can be used for any migration project4. Reduced cost due to faster QA cycles and reduced manual testing efforts5. This process enables the flexibility of choosing any automation tool for any migration testing project

Benefits of AFET:single script which is consistent, repeatable and accurate; reduced cost, time and efforts



About the Authors

Sagar M. PatilManager/Architect-Test Automation

At Datamatics Sagar is responsible for providing automation solutions for diverse technologies and platforms. He has 12 years of experience in IT industry and his core expertise lies in Manual Testing, Test Automation, performance testing & Test Management. He has handled multiple complex testing projects using automation tools like QTP, Selenium, TestCompelte, CodedUI, Ranorex which includes framework development. Software Testing is his passion and he likes to address each challenge with an innovation but with simple solution. Sagar is PMP, CSTE, HP Functional test, Six sigma green belt certified professional. His white paper has been awarded in Testride Mumbai conference.

About Datamatics Global Services

Global Information Technology (IT) & Knowledge Process Outsourcing (KPO) organization Delivers smart, next-generation business solutions Trusted partner to several Fortune 500 companies Capabilities built around technology, domain expertise & knowledge of business processes Featured amongst the Global Services 100 List in 2010 & 2011 Decades of global experience having executed projects across 60 countries

2 Alliances with global technology leaders such as Microsoft, IBM & EMC Certified for SEI CMMI Level 3 V1.3, ISO 27001:2005 ISO 9001:2008 SSAE 16 compliant processes Global presence: U.S, UK, Germany, Switzerland, Bosnia, Australia, Singapore & India

To know more, connect with us on [email protected]


mailto:[email protected]

Whitepaper - Automated Functional Equivalence Testing

Technology

Transcript of Whitepaper - Automated Functional Equivalence Testing