The 4 T’s of Test Automation:The 4 T’s of Test Automation:Your Planning Guide to Success
Presented By:David DangQuestcon Technologies
September 18, 2008
The 4 T’s of Test Automation:The 4 T’s of Test Automation:Your Planning Guide to Success
Presented By:David DangQuestcon Technologies
September 18, 2008
Quality - Innovation - VisionQuality - Innovation - Vision
Slide Slide 22
BACKGROUNDBACKGROUND
Test Automation: Saves time and Increases Coverage
Tool Evaluation
Training
Team starts using the tool
Let’s review the typical test automation process . . .
Slide Slide 33
Typical Automation Project OutcomeTypical Automation Project Outcome
Adopted by QA with success at the beginning
Over subsequent release cycles, maintenance time and costs increase
Final outcome, the automation suite becomes a maintenance nightmare and is used by a few users or becomes shelfware
Slide Slide 44
ObjectivesObjectives
The primary objective of this presentation is to demonstrate how proper planning for test automation will yield greater success. Attendees will learn:
Potential benefits of a successful automation project
Key factors to consider in the planning phase of test
automation
Proper alignment of automation tools to technology under
test
Evaluation criteria for manual test cases
Required skill sets for test automation
Timeline impacts of using different automation approaches
Slide Slide 55
While there are multiple considerations that must be addressed during the planning phase of a test automation project, these four factors have the largest impact on the overall success.
Test CasesTest CasesTest CasesTest Cases
Key FactorsKey Factors
TechnologyTechnologyTechnologyTechnology
TalentTalentTalentTalent
TimelineTimelineTimelineTimeline
Slide Slide 66
Create an inventory of the current technology that you must test
PlatformsProgramming LanguagesDatabasesThird-party controls
Determine what technology changes may occur within two yearsExamples of technology under test (TUT):
Platform – Mainframe, Client/Server, and .Net (long-term)Programming Language – Cobol II, VB 6, C ++, and C# (long-term)Database – DB2, Sybase, and MySQL (long-term)Third-party controls – VSFlexGrid and WebServices (long-term)
TechnologyTechnology
Slide Slide 77
TechnologyTechnology
ConsiderationsLevel of support for TUTAmount of customizations required for non-supported TUTEase of useSkill sets required to work with toolCommon languagesTool costTool maintenance costIntegration with other toolsReporting
Slide Slide 88
Matching TUT to the automation tool ensures faster automation script development and decreased maintenance time by:
Reduction in custom codingReduction in object mappingIncreased flexibility in framework designEasier debuggingBetter reporting
Additionally, this approach reduces learning curve of automation toolIncreases the level of testingProvides better cost justification
BenefitsBenefitsBenefitsBenefits
TechnologyTechnology
Slide Slide 99
Test CaseTest Case
Is this a good test case?
Why?
Slide Slide 1010
Test CaseTest Case
Test cases contain detailed levels of information:Test case nameTest case objectivePre-requisitesTest case priorityInput and output dataStep nameStep descriptionExpected result
Determine:Frequency of execution over a yearly periodLevel of testing required for validationComplexity of the test casesManual only steps, e.g. loading a backup tapeAutomation priority based on risk factors
Evaluate Manual Test CasesEvaluate Manual Test CasesEvaluate Manual Test CasesEvaluate Manual Test Cases
Slide Slide 1111
Test CaseTest Case
Slide Slide 1212
Scope identification of the automation effortPrioritization of automation effort based on risk factorsElimination of manual test cases that were out-datedIdentification of manual test cases that provide little to no return on investmentUnderstanding of test case factors in creating an automation framework
BenefitsBenefitsBenefitsBenefits
Test CasesTest Cases
Slide Slide 1313
QA experience in various levels of testing such as smoke, regression, and functionalSolid understanding of business flows, business rules, and data variations in the applicationExperience with multiple data strategies
Creating “new” dataGathering existing dataBuilding data
Experience with test automation conceptsUnderstanding of computer programmingGrasp of application architectureComprehension of Object-oriented programming (OOP)
Required Skill SetsRequired Skill SetsRequired Skill SetsRequired Skill Sets
TalentTalent
Slide Slide 1414
Experience in programming using Java, VB, or C languagesExperience with multiple automation approachesExperience in SQLExperience in writing classes and methodsExperience in writing functions and sub-proceduresExperience in using or interacting with window APIsProject Management
TimelinesDeliverablesResourcesMetricsBudget
Optional Skill SetsOptional Skill SetsOptional Skill SetsOptional Skill Sets
TalentTalent
Slide Slide 1515
Establishes a good baseline for skill sets required to implement test automationCan be used to determine QA members that will work on test automation
Most companies struggle in test automation when they have the QA group perform test automationRecommend creating an automation team
Reduction in trial and error approachAutomation approach designed around skill set level available
BenefitBenefitBenefitBenefit
TalentTalent
Slide Slide 1616
Automation Projects TypicallyAutomation Projects TypicallyMirror SDLCMirror SDLC
Automation Projects TypicallyAutomation Projects TypicallyMirror SDLCMirror SDLC
TimelineTimeline
The same steps that occur in the SDLC should occur in an automation projectMost automation projects only account for Implementation and TestTimeline should take every step in the SDLC into consideration
Slide Slide 1717
Factors that Impact TimelineFactors that Impact TimelineFactors that Impact TimelineFactors that Impact Timeline
TimelineTimeline
Resource availabilityMost companies assign 30 – 50% of QA member time to test automationThe reality is QA members cannot effectively perform both manual and automation testingThis greatly increases the timeline for creating and maintaining test automation suite
Resource skill setsSelected automation toolTesting environmentData strategyManual test case condition
Slide Slide 1818
Factors that Impact TimelineFactors that Impact TimelineFactors that Impact TimelineFactors that Impact Timeline
TimelineTimeline
Automation frameworkThere are five common automation frameworks
Record/PlaybackData-drivenModularAction-based/KeywordDatabase
Automation frameworks are not mutually exclusiveAutomation frameworks can be combined, e.g. Action-based and database approachesAppropriate automation framework dependant on many factors including the company goal for test automation
Slide Slide 1919
TimelineTimeline
Case StudyCase StudyCase StudyCase Study
0200400600800
10001200140016001800
Recor
d/Play
back
Data-
Driven
Mod
ular
Action
-bas
ed/K
eywor
d
Datab
ase
Development
Maintenance
Maintenance/Year @10 Releases/Year
Slide Slide 2020
Sets clear expectations for managementIdentified timeline for automation development life cycle (ADLC)Maintenance time factored into the automation framework designClear understanding of milestones and deliverableEasier measurement of the cost and benefit of test automationROI calculation for break-even point for test automation
BenefitsBenefitsBenefitsBenefits
TimelineTimeline
Slide Slide 2121
The most important aspect of any test automation project is planning. With the right strategy, you will save time and money, and set clear expectations for your organization.
During planning, remember the 4 Ts:Technology
Test Cases
Talent
Timeline
The most important aspect of any test automation project is planning. With the right strategy, you will save time and money, and set clear expectations for your organization.
During planning, remember the 4 Ts:Technology
Test Cases
Talent
Timeline
SummarySummary
Slide Slide 2222
Q & AQ & A
Top Related