Geometric Whitepaper Scriptless Test Automation

8/11/2019 Geometric Whitepaper Scriptless Test Automation

1/11

W

HITE

P

APER

n

vity

improvement

in

software

testing

Version1.0

February,2011

ScriptlessTestAutomatio

Nextgenerationtechniquefor

producti


2/11

2

GettingreadyforPLM

CopyrightNotice

GeometricLimited.Allrightsreserved.

Nopartofthisdocument(whetherinhardcopyorelectronicform)maybereproduced,storedinaretrieva

system,or transmitted, inanyformorbyanymeans,electronic,mechanical,photocopying,recording,o

otherwise, to any third party without the written permission of Geometric Limited. Geometric Limited

reservestherighttochangetheinformationcontainedinthisdocumentwithoutpriornotice.

The names or trademarks or registered trademarks used in this document are the sole property of the

respectiveownersandaregoverned/protectedbytherelevanttrademarkandcopyrightlaws.

ThisdocumentisprovidedbyGeometricLimitedforinformationalpurposesonly,withoutrepresentationo

warrantyofanykind,andGeometricLimitedshallnotbeliableforerrorsoromissionswithrespecttothe

document.The informationcontainedherein isprovidedonanASISbasisandto themaximumexten

permittedbyapplicablelaw,GeometricLimitedherebydisclaimsallotherwarrantiesandconditions,eithe

express, implied or statutory, including but not limited to, any (if any) implied warranties, duties o

conditionsofmerchantability,offitnessforaparticularpurpose,ofaccuracyorcompletenessofresponses

of results, of workmanlike effort, of lack of viruses, and of lack of negligence, all with regard to the

document.

THEREISNOWARRANTYORCONDITIONOFNONINFRINGEMENTOFANYINTELLECTUALPROPERTYRIGHTS

WITH REGARD TO THE DOCUMENT. IN NO EVENT WILL GEOMETRIC LIMITED BE LIABLE TO ANY OTHER

PARTY FOR LOST PROFITS, LOSS OF USE, LOSS OF DATA, OR ANY INCIDENTAL, CONSEQUENTIAL, DIRECT

INDIRECT, OR SPECIAL DAMAGES WHETHER UNDER CONTRACT, TORT, WARRANTY, OR OTHERWISE

ARISINGINANYWAYOUTOFTHISDOCUMENT,WHETHERORNOTSUCHPARTYHADADVANCENOTICEOF

THEPOSSIBILITYOFSUCHDAMAGES.

ConfidentialityNotice

Thisdocumentisdisclosedonlytotherecipientpursuanttoaconfidentialityrelationshipunderwhichthe

recipient has confidentiality obligations defined herein after. This document constitutes confidentia

informationandcontainsproprietaryinformationbelongingtoGeometricLimited,andtherecipient,byits

receiptofthisdocument,acknowledgesthesame.Therecipientshallusetheconfidentialinformationonly

for thepurposedefinedabove forwhichthisdocument issupplied.TherecipientmustobtainGeometric

Limitedswrittenconsentbeforetherecipientdisclosesanyinformationonthecontentsorsubjectmatte

ofthisdocumentorpartthereoftoanythirdpartywhichmayincludeanindividual,firmorcompanyoran

employee or employees of such a firm or company. The recipient acknowledges its obligation to comply

withtheprovisionsofthisconfidentialitynotice.


3/11

3

GettingreadyforPLM

Contents

Scope:GUI

based

Test

Automation ........................................................................ 4

EvolutionofTestAutomation................................................................................. 4

TestAutomationusingthirdpartytesttools.......................................................... 4

ConstituentsofScriptlessTestAutomationtoolorframework............................. 7

Inference ............................................................................................................... 10

Caution.................................................................................................................. 10

AbouttheAuthor .................................................................................................. 10

AboutGeometric .................................................................................................. 11


4/11

4

GettingreadyforPLM

Scope:GUIbasedTestAutomation

GUI

based

Test

automation

is

the

activity

of

creating

automated

test

scripts

that

can

be

runwithouthumanintervention.TheyrunoffthesameUIthattheendUserorCustomerwouldsee.

Thesescriptscancoveralotofapplicationareasinarepeatablemanner.Theycancutdownon

testexecutiontimesignificantlyastheyruncontinuouslyandunattended,evenovernight.

Fromhereon,inthisdocument,TestAutomationwillmeanGUIbasedTestAutomation.

EvolutionofTestAutomation

Historically, Test Automation tools startedoff bygiving a macro recording facility example is

MS Excel. Though the application UI was used to record the automation script, the replay

mechanism depended on synchronous API calls, rather than the UI. The test execution engine

wouldfireacommand,andthenwaitforthecommandtocomplete.

Amacrobasedtoolwasgoodforsuchshort lengthautomation.Combinea logicalsequenceo

suchshortlengthmacros,andyourtestflowisautomated!

Thismechanismcanbeseen insomedesktopbasedCADtools,whereinmacroscanbequickly

recorded,andtherecordedmacroscanbeimmediatelyplayedbackwithlittleornomodification

This approach is the best, even today, but applies to only those applications that have such a

frameworkbuilt in.This isagoodexampleoftestabilitybeingbuilt intoanapplication,rightat

thedesign

stage.

However,notallsoftwareapplicationsthatareinthemarkettodayhavesuchatestframework

builtin.Itsthissegmentofapplicationsthatthethirdpartytesttoolsaddress.

TestAutomationusingthirdpartytesttools

Test automation using third party test tools historically started with test tools that provided a

dumbrecordandreplayfacility.

Though this approachworked well for some desktop based applications, there was a lottobe

desired

in

terms

of

execution

speed

control,

type

of

validation

check

points,

modularity

oscripting,howwelltestinputdatacouldbefedtothescript.

Thiswasthefirstgenerationoftestautomationtools.

Moreover, even though the speed of automation creation was fast, the scripts were very

sensitivetoUIchanges,andchangestoUIwouldeventuallyresultinthrowingawaytheprevious

automationscriptsandcreatingnewones.


5/11

5

GettingreadyforPLM

Figure

1.

Test

Automation

Productivity

Thesecondgenerationoftestautomationtoolsbroughtawhiffoffreshair;theybroughtaloto

featurestothetesttoolthatbroughtupsimilaritiesbetweentesttoolIDEanddevelopersIDE

These tools provided fullfledged scripting, or even objected oriented language support to the

automation engineer. A lot of tool APIs were present to simplify the common windows based

tasks likeapplication launch,fileoperations,generatingelementarytestreports.Someofthese

alsocamewithplatformbasedextensibilitySDK,forexample,for.NetorJava.

Thevarietyofobjectrecognitionaswellascheckpointspossiblewasextensiveandcodingcould

achieve virtually anything that the automation engineer desired be it data driving, keyword

driving,useofvariables,functions,separationofobjectdefinitionfromobjectreferencesacross

thescripts.

However,thisscriptbasedapproachstillneedsalotofinprojectinvestments,intermsoftime

to learn and gain expertise in the scripting language, as well as the time to create a suitable

frameworkthatnotonlyharnessesreusablecode,butalsoeasesthedatadrivingofthescripts

alloftheseeventuallydrawalinebetweenthemanualtesterandtheautomationengineer.

So,eventhoughtheautomationfeasibilityproblemgotsolvedwiththeadventofmorerobust

object recognition technology, the productivity problem still leads to a low ROI from test

automation.

Withthese

thoughts

in

the

background,

we

started

looking

for

test

automation

accelerators.

ThequestforROIfromTestAutomation

Weevaluatedthefollowingapproachesfortheproductivityproblem:


6/11

6

GettingreadyforPLM

Table1.Evaluationofvariousapproaches

# Approach Howithelps Remarks

1 Opensourcetool Direct reduction in investment

needed

Might help overcome

budgetaryconstraints

Investmentineffortmightbe

much larger than that in the

tool

Test Tool support virtually

nonexistent

Usuallytaketimetocatchup

on the latest platform o

technology

2 Bigbang

automation

Benefit of automation made

available much sooner by

deploying a huge automation

team

Couldbeconstrainedbytoo

costs

HighGUIstabilityimportant

3 Prebuilt

framework

Part of the development

alreadydone

Available at low cost as it is

soldtomultiplebuyers

ERP example: SAP

automationframeworks

4

Modelbased Automation

based

on

a

model

of the app, thus the scale of

theoperationissmaller

Being based on a model, it

provides a higher level of

abstraction, thus test design

itself can benefit from this

approach

Many

different

approacheevolving

Waitandwatch

5 1stgeneration

recordand

playback,with

littleor

no

scriptingsupport

Extremely rapid creation of

automation

Poorrobustness

Poormaintainability

Lack

of

tool

APIs

hampe

creation of any suitable

frameworks

Usually unsuitable fo

enterpriseapps

Perhaps for very stable

desktopapps


7/11

7

GettingreadyforPLM

6 2nd

generation

Recordand

playback,with

scriptinglanguage

support

Robustobjectrecognition Maintainability has to be

built in through a suitable

framework

Enables the automation

engineer with a fullfledged

scriptinglanguagesupport

Codingskillsrequired

Noneoftheaboveapproacheslookedpromisingenoughintermsofproductivityofautomation

scriptgeneration.

ButthesecondgenerationtesttoolIDEshavebecomefullfledgedcodeeditorsthatsupportthe

automationengineer innotonlycreationofscriptsbutalsodebuggingandexecution.Andyet

frameworksstillneedtobebuiltbeforeany longtermtestautomationprojectcango livewith

testing.

Atthispoint,westartedthinkingonotherlines.

Years ago, Microsofts Visual Studio and Foundation Classes brought a quantumjump in the

productivity with which Windows GUI could be developed instead of hand coding with the

Win32API,developerscouldnowusewizardstovisuallydesigntheGUIinastudio.

WesoondiscoveredthatScriptlesstoolsbringthesameconcepttotestautomation.

Insteadof

acode

editor,

automation

engineers

now

work

in

agraphical

environment,

where

they

cancomposeautomatedtestsbyselectingobjectsandactionsfromdropdownmenus.Theycan

alsovisuallycreateconditions,iterations,etc.

We have worked with several Scriptless tools and are getting 50% or higher productivity as

comparedtoscriptbasedautomation.Wearerecommendingittoourcustomers,andofcourse

forourowninternalprojects.

Webelieve it isjustifiedtoviewScriptlessautomationtoolsasconstitutingthenextgeneration

oftoolsthatwillgainprominenceintestautomation.

Constituentsof

Scriptless

Test

Automation

tool

or

framework

Scriptlesstestautomationprovidesaveryeasytouse interface insteadoflearningascripting

language,orevenaprogramminglanguage,onegetstoworkonadropdownandExcelkindo

userinterface.


8/11

8

GettingreadyforPLM

All the requirements of a test automation script, like the sequence of keywords, steps within

each keyword, data per step, variables that have a scopeof the full length ofthe keyword, o

evenacrossthesequenceofkeywordsforthetest,andUIobjectdefinitionsareavailableina

veryeasy

to

use

interface

and

no

programming

is

involved!

Allthegoodconceptsoftestautomation likedatadriving,keyworddriving,andmodularityare

allpackagedinveryeasytousesoftware,withouttakingawayanyofthepowerthatascripting

basedtoolprovides.

SomeofthenotablefeaturesofaScriptlessframeworkthataidinautomationcreationare:

Figure1.InputsandActivitiesrequiredtoCreateAutomatedTest

Moreover, they have a clientserver based approach, wherein, automation once created is

readilyavailabletothewholeteam,atthepushofabutton.

These tools not only provide script creation support, but also go beyond they let youdebug

fromvirtuallyanystepwithinthescript.Theyalsoprovidetestmanagementrequiredfortes

automation, like composingtestsuites,runningthetestsuites,reportingthetestrun result. I

oneneedshistoryoftestruns,eventhatisavailable!


9/11

9

GettingreadyforPLM

SomeofthenotablefeaturesofaScriptlessframeworkthataidinTestexecutionorreexecution

are:

Figure2.ActivitiesinvolvedinAutomatedTestExecution

Some ofthe notable features of a Scriptless framework that aid inAutomationResult Analysis

are:

Figure3.CurrentorHistoricalTestRunAnalyses

Wewould liketonotetwosuchtoolsthathavealotofthesefeaturesCertifyfromWorksoft

which is a test automation tool, as well as Qualitia from Zensoft Services, which is a tes

automationframeworkthatridesontopofotherleadingtestautomationtoolssuchasQTP,RFT

WebelieveothervendorsarealsoworkingonprovidingScriptlesscapabilities.

ROIfromthesetools is impressiveastheyareproductivevirtuallyfromdayone,asyoudonot

have to spend time initially on writing a framework. Add to this, the fact that they are time

tested

across

multiple

applications

and

over

considerable

amount

of

time.

Thefactthattheyareeasytousealsomeansthatthere isaverygoodrateofadoptionofthe

toolbytheManualTestercommunity,andthisbenefitstheorganization!


10/11

10

GettingreadyforPLM

Inference

Scriptlessautomationhasthepotentialtobethenextgenerationautomationtechnology.

Itnotonlyaddressestestautomationsproductivityproblem,butalsodoesitinawaythatdoes

notforceyoutodrawalinebetweenthemanualtesterandtheautomationengineer.

Moreover,itlendsacomprehensiveUItoallthetestautomationBestPractices,thusitenables

theOrganizationtoStandardizeafteronehasdonethefirsttestautomationprojectonaJava

basedapplication,onedoesnotneedtostartfromscratchona .Netbasedprojectthesame

testtoolandthesameUIwillapply,asalltheframeworkstuffisalreadyavailable.

Caution

Anoteofcaution,notalltoolsthatinitiallyappeartobeScriptlessaretrulyscriptless,orareas

featurepackedasothers.

AutomationofcustomUIcomponentscouldinsomecasesneedcodingexpertise.

Not all test automation tools support all technologies so check with the tool vendor for any

specifictechnologyrequirements.

Distributedtestexecutionacrossmachines,andaunifiedresultreportingisaworkinprogressin

somecases.

AbouttheAuthor

SudiptoPaulisaTestConsultantatGeometric,andisresponsibleforbuildingtestteams, initia

handholdingandguidanceontestprocessesforiterativeandagileprojects.Hehasover13years

of experience in the information technology field, and has had an allround exposure to the

softwaredevelopmentlifecycle.Hisexperienceincludesrolesoftestconsultant,testautomation

architect&engineer,manualtestlead,requirementsanalyst,designeranddeveloper.

Sudipto'sprimaryareasofinterestincludetestprocessesandtechnicaltesting.Hehasauthored

several

utilities

and

test

automation

frameworks

to

aid

both

developer

side

and

tester

sidetesting.

Sudiptohold aBE (Mechanical Engineering) fromWalchandCollegeofEngineering, Sangli,and

PGCBMfromXLRI,Jamshedpur.


11/11

11

GettingreadyforPLM

AboutGeometric

Geometric (www.geometricglobal.com) is a specialist in the domain of engineering solutions

services and technologies. Its portfolio of Global Engineering services and Digital Technology

solutionsforProductLifecycleManagement(PLM)enablescompaniestoformulate,implement

and execute global engineering and manufacturing strategies aimed at achieving greate

efficienciesintheproductrealizationlifecycle.

Headquartered in Mumbai, India, Geometric was incorporated in 1994 and is listed on

the Bombay and National Stock Exchanges. The company recorded consolidated revenues o

Rupees 5.11 billion (US Dollars 108.1 million) for the year ended March 2010. It employs ove

3600

people

across

10

global

delivery

locations

in

the

US,

Romania,

India,

and

China

Geometric was assessed as CMMI 1.1 Level 5 for its software services and is ISO 9001:2008

certifiedforengineeringoperations.

The

copyright/

trademarks

of

all

products

referenced

herein

are

held

by

their

respective

companies.

Geometric Whitepaper Scriptless Test Automation

Documents

Transcript of Geometric Whitepaper Scriptless Test Automation