Test Automation Approach to Real Life
-
Upload
ramkumnelipudi -
Category
Documents
-
view
222 -
download
0
Transcript of Test Automation Approach to Real Life
-
8/8/2019 Test Automation Approach to Real Life
1/19
Yury Makedonov p. 1 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 1
Test Automation Approaches to Real Life
Copyright 2003 CGI Group Inc. October 1, 2003October 1, 2003
The International Quality Conference, Toronto, Ontario, CanadaThe International Quality Conference, Toronto, Ontario, Canada
Yury Makedonov, CGIYury Makedonov, CGI
Golam Mustofa, CGIGolam Mustofa, CGI
October 01, 2003 Test Automation Presentation for QAI Conference 2
Agenda
Why Automate?
Reality of GUI Test Automation today
What are the challenges?
Test Automation Approaches in CGI
Stand Alone Automation Approach
Fully Integrated Automation approach Summary
Q&A
-
8/8/2019 Test Automation Approach to Real Life
2/19
Yury Makedonov p. 2 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 3
Pr imary Uses Of GUI Test Aut omat ion
Perform regression tests
Volume tests
Reduce human resource requirements
October 01, 2003 Test Automation Presentation for QAI Conference 4
Why Autom ate?
Save Time
Reduce Total $$ (COQ)
Repeatability/More Consistent Coverage
Reliability
Reusability
Portability
-
8/8/2019 Test Automation Approach to Real Life
3/19
Yury Makedonov p. 3 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 5
Real i ty Of GUI Test Aut om at ion Today
By many accounts:
More than 50% of GUI Test Automation
projects fail
Test scripts & tools become shelfware
October 01, 2003 Test Automation Presentation for QAI Conference 6
What Are The Chal lenges?
GUIs are dynamically changing
Application is not stable enough (last minutes changes,
scope creep)
Not enough time to update scripts (poorly designed)
Lack of in-house expertise
Budget constraints
Immature Test Automation methodologies
-
8/8/2019 Test Automation Approach to Real Life
4/19
Yury Makedonov p. 4 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 7
Test Aut om at ion Approaches in CGITest Aut om at ion Approaches in CGI
Test Automation Approaches
Two Approaches Used In CGI
Stand Alone
WinRunnerFully Integrated
WinRunner TestDirector
October 01, 2003 Test Automation Presentation for QAI Conference 8
Standalone WinRunner
Yury Makedonov
-
8/8/2019 Test Automation Approach to Real Life
5/19
Yury Makedonov p. 5 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 9
Stand alone WinRunner
The fundamental principles will be covered in this section
Test automation from point of view of:
Portability of test scripts
Maintainability of test scripts
Test Automation team supporting:
Multiple versions
Of multiple applications
In different environments
In different locations
October 01, 2003 Test Automation Presentation for QAI Conference 10
Application
GUI Map physical object description:
OK{class: push_button,label: OK,vb_name: cmdOK
}
What is a GUI Map?
WinRunner Script logical object name:
button_press ("OK");
OK
-
8/8/2019 Test Automation Approach to Real Life
6/19
Yury Makedonov p. 6 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 11
St ep One Tools eva luat ion
Can a GUI testing tool handle our application?
How time consuming is test automation (Does
it make sense to automate)?
Do we care about a proper organization of test
scripts?
October 01, 2003 Test Automation Presentation for QAI Conference 12
Step Tw o Fi rs t vers ion o f the app l ica t ion :
load("C:\\WR_Scripts\\Lib_UT_Functions");
GUI_load("C:\\WR_Scripts\\ UTGuiMap.gui");
web_browser_invoke(IE,"http://confut.xyz.ont.ca");
UT_Select_Language("English");
UT_Login("a327012", "12345");
-
8/8/2019 Test Automation Approach to Real Life
7/19
Yury Makedonov p. 7 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 13
St ep Three Second vers ion:
New version of theapplication delivered
Cloning of scripts
load("Lib_UT_Functions);
GUI_load(UTGuiMap.gui);
No full path for a GUI map or ascript
October 01, 2003 Test Automation Presentation for QAI Conference 14
St ep Four Several Test ers :
Several testers working with WinRunner Scripts moved onto shared Z: drive
load("Lib_UT_Functions");
GUI_load("CoolGuiMap.gui");
UT_Open_Site( );
UT_Select_Language("English");
UT_Login("a327012", "12345");UT_Select_Portfolio("KEY");
call "Invoice_Create"( );
-
8/8/2019 Test Automation Approach to Real Life
8/19
Yury Makedonov p. 8 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 15
St ep Four Several Test ers :
All modifications are performed on a local copy of ascript.
After debugging script will be copied onto a shareddrive.
Each script has an owner. Owner stores mastercopy of the script on his local workstation.
Simultaneous execution/modification of the same
script/library on different workstations.
October 01, 2003 Test Automation Presentation for QAI Conference 16
St ep Five Several App l ica t ions:
Test automation forseveral applications
Application (version)specific libraries
General libraries (usedby all projects)
Major rework of librariesof functions
-
8/8/2019 Test Automation Approach to Real Life
9/19
Yury Makedonov p. 9 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 17
St ep Six Severa l Env i ronment s,Brow sers, User Roles:
C:\TestData\UTEnvironment.xls file:
URL B rows er U se rID Pa ss word
http://prodaceman1.configurator.com/ut IE a327842 123
We need to execute the same script concurrently:
against different environments, using different browsers,
using different user roles/login IDs.
Store setup parameters in a local file.
Site_Open(flURL, flBrowser, fluserID, flPassword);
October 01, 2003 Test Automation Presentation for QAI Conference 18
Rec omm endat ions for WinRunner pro jec t so f any complex i t y :
Implement the complete hierarchy of folders for WinRunner scripts fromday one: Folder for general libraries
Folders for different Application
Subfolders for different Versions of the application
All scripts for a specific version of an application arestored on the same level of hierarchy
Use path relative to a current test to load an object (Nospecification of a full path, starting with a drive letter, allowed)
Implement a standard script header containing description of ascript + all initialization steps: GUI Load,
Libraries load,
Reading data file, etc.
Implement Script Ownership concept: master copy of anyscript belongs to a single person
MAKE IT A STANDARD!!!
-
8/8/2019 Test Automation Approach to Real Life
10/19
Yury Makedonov p. 10 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 19
Fully Integrated Test Automation - WinRunner &TestDirector
Golam Mustofa
October 01, 2003 Test Automation Presentation for QAI Conference 20
St ep One Know The Tools
Functional Testing Tool
Develops the scripts based on detail Test Cases
Needs to create GUI file and Data Spreadsheet
Needs to connect to the TestDirector
Test Management Tool
WinRunner scripts have to be integrated withTestDirector
TD schedules execution of WinRunner scripts
Manages Defects
-
8/8/2019 Test Automation Approach to Real Life
11/19
Yury Makedonov p. 11 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 21
St ep Tw o Underst and The Overa l lProcess
WinRunner activities
Learn GUIs of the application under testing
Develop / Record scripts
Debug scripts
Import or Save scripts into the TestDirector
TestDirector activities
Locate the Test Data Spreadsheet into a shared drive or attachto the TestDirector
Build the Test Sets
Schedule execution of the Test Sets
Analyse Test Result and create Test Report
Report or Manage defects
October 01, 2003 Test Automation Presentation for QAI Conference 22
St ep Three Create An Ef fec t iveFramework
Create a root folder into the Test Plan Tab of
TD
Design an Initial / Startup scripts
Test Environment setup
Load GUI and function library
Declare variables, data spreadsheet location, etc
It itself can have a data spreadsheet as well
Example
Create sub folders based on components ofthe AUT
Make a folder for compiled module or
functions library Function library for generic functions
Function library for application specificfunctions
-
8/8/2019 Test Automation Approach to Real Life
12/19
Yury Makedonov p. 12 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 23
St ep Three Create An Ef fect iveFramework
Build a td_files folder into the
shared drive Make sub folders named
datatables and gui_filesunder the td_files folder
Have sub folders based oncomponents of the AUT underthe datatables folder
Keep all the GUI files into thegui_files folder orTestDirectors project
October 01, 2003 Test Automation Presentation for QAI Conference 24
Step Four Ut i l ize Data-Dr iven Test Techniques
Single scripts will handle negative and positive boundarytest reading data from the data spreadsheet
Bel l CanadaBel l Canada
Bel l CanadaBel l Canada
Blank dataBlank data
Erro r MessageEr ro r Message
Special charactersSpecial characters
Valid dataValid data
AcceptedAccepted
$!% #$!% #
$!% #$!% #
Erro r MessageEr ro r Message
Have control on the spreadsheet
Skip some rows
Stop in row
Bel l Mobi l i tyBel l Mobi l i t y
Bel l Bel l ExpressVuExpressVu
BCEBCE Nexx ia Nexx ia
BCEBCEAct iMediaAc t iMedia
BCE Medi aBCE Media
Bel l Bel l Emerg is Emerg is
Sympat i coSympat i co
Bel lZ inc.caBel lZ inc.ca
XX
XX
XX
StopStop
Write important data or results into the spreadsheetafter the transactions for future analysis
PassedPassed
PassedPassed
PassedPassed
PassedPassed
PassedPassed
Fai ledFai led
PassedPassed
PassedPassed
PassedPassed
$!% #$!% #
PassedPassed
-
8/8/2019 Test Automation Approach to Real Life
13/19
Yury Makedonov p. 13 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 25
Step Five Locat e Data Table (Spreadsheet )
Attach the data spreadsheet withthe Test Case
Download the spreadsheet on thefly during scripts execution
Read the spreadsheet as input file
Or keep the spreadsheet into theshared drive
TestDirector doesnt allow to save the Data spreadsheet into theProjects database
October 01, 2003 Test Automation Presentation for QAI Conference 26
Step S ix Bui ld Test Sets
Identify all Testing scripts which needs to be into a bundle basedon applications functionality
Create a Test Set
Attach all identified Testing scripts into the Test set
-
8/8/2019 Test Automation Approach to Real Life
14/19
Yury Makedonov p. 14 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 27
Step Seven Sc hedule And Exe cut e Test Sets
From Execution Flow tab click on clock icon
Set the time and date youd like to the Test Set or Individual Test
Press on Run Test Set or Run button
TestDirector will invoke WinRunner and perform all the tests
October 01, 2003 Test Automation Presentation for QAI Conference 28
Step E ight Manage Your Defect s
Analyse the Test Result
Manage any defect through the Defect Tab of TestDirector
-
8/8/2019 Test Automation Approach to Real Life
15/19
Yury Makedonov p. 15 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 29
Summary
Test Automation, when used to perform Regression &Volume testing can reduce cost, human resources andtime
Implement Stand Alone or Fully Integrated methodbased on your environment, goals/priorities, budgetand expertise of resources
Test Automation is worth the investment and effort, ifits implemented properly!!!!
October 01, 2003 Test Automation Presentation for QAI Conference 30
Questions??????
-
8/8/2019 Test Automation Approach to Real Life
16/19
-
8/8/2019 Test Automation Approach to Real Life
17/19
-
8/8/2019 Test Automation Approach to Real Life
18/19
Yury Makedonov p. 18 of 19
International Quality Conference, Toronto, Ontario, Canada October 1, 2003
October 01, 2003 Test Automation Presentation for QAI Conference 35
Appendix B - Design ing In i t ia l / St ar tupsc r ip ts
October 01, 2003 Test Automation Presentation for QAI Conference 36
Appendix B - Sk ip some row s o r in a rowon the dat a spreadsheet
-
8/8/2019 Test Automation Approach to Real Life
19/19
Yury Makedonov p. 19 of 19
I t ti l Q lit C f T t O t i C d O
October 01, 2003 Test Automation Presentation for QAI Conference 37
Appendix B - Loca te Dat a Tab le(Spreadsheet)
October 01, 2003 Test Automation Presentation for QAI Conference 38
Contac t In format ion
Senior Consultant,
Centre for Testing & Quality
CGI Services to BCE
595 Bay Street, Floor 10-B
Toronto, Ontario,
Canada M5G 2C2
(416) 215-3593
Senior Consultant,
Centre for Testing & Quality
CGI Services to BCE
595 Bay Street, Floor 10-B
Toronto, Ontario,
Canada M5G 2C2
(416) 215-7534
Golam MustofaYury Makedonov