Failure and Success in Software Automation by Oleksandr Reminnyi
-
Upload
softserve-inc -
Category
Technology
-
view
303 -
download
0
description
Transcript of Failure and Success in Software Automation by Oleksandr Reminnyi
Failure and Success in Software Automation
Comparing 7 Projects
Oleksandr Reminnyi
Pre word
▪ The only identification of project being successful is the fact that your customer is happy.
PMbook
Understanding the goals
We want to save money and time later on manual testingWe have a corporate direction…But the main is…Automation is a BUZZ word
Average Automation Reality
Automation AND Skill Set
HOKKU
«Let`s automate everything!», the client said. And Mount Fuji trembled.
Customer skills
Should customer be mature in automation?
This is definitely not a prerequisite.
But it can influence…
ХОККУ
Читаю чужі тести і торкається моїх скронь сивина…То мозок проливається
Automation Dev. Team Skills
Does mature mean good automation here?
Not always.
Practices mean more.
Automation AND Time line
Automation and SDLC
When the automation might be successful?
Waterfall OR Agile?
Depends on automation layer;
Post release period!
Implementation OF one TC
Depends on complexityDepends on number of steps
Depends on tested application instrumentation
Average 4 to 20 man hours. Happens 50
Automation Toolset
Automation and Backend
Unit TestsReally? Why?
Component/integration testingGood job!
Service layer automationDefinite place to automate!
Automation And TOOLS
Can tool influence the success of automation?YES!
Automation of Mobility
Can mobile platform automation be buried before birth?
YES!Silk Mobile -
$5000 per dev. per year,
20-30% failure rate only because of the tool issues
MonkeyTalk, Ranorex, etc.Build in the agent, then find issues
Appium maybe light in the end of the tunnel. But later
Consider out of device testing!
Documentation
Can automation exist without regression test Cases suite?
YES!
At least they say it’s successful…
Different types of manual test cases being Automated
- Functional- Regression- Acceptance- …
Benefit is brought at once by
Smoke ones
Automation Metrics
HOKKU
Sacura`s leaves are destined to fall down.Once again we`ll see the Test Report in red.Not only leaves are down.
What is usually Calculated?
ROI? Nothing!
Sometimes:
Failure Rate Suite Run Duration
Given architecture…
SUT UI Other SUT facilities that can be hooked but not needed for automation tool accessServices …
Layer Test scriptsTest scriptsFunctional Libraries
Layer Test scriptsTest scriptsTest scripts
Fr
amew
ork
utiliti
esBa
se c
lass
esRe
porti
ng
Load
ing
confi
gura
tion
Logg
ing
Etc.
Layer Test scriptsTest scriptsUI Maps – facilities to access UI (or other services) through automation tools API
Som
etim
es jo
ined
Layer Automation Tool
DB
Hel
pers
Faci
lities
to h
ook
into
SU
T th
roug
h AF
Mul
ti - l
ayer
ed a
rchi
tect
ure
Gave us possibility to collect More metrics
Something obvious
Questions?
“I see the testing standards that do not tell me how to create a test worthy of running."
DeMarco