7/23/2019 e Babenskas Automated Software Testing
1/36
Automated
Automatedsoftwaresoftware
testingtestingEgidijusBabenskas,IFM-0/2
1
7/23/2019 e Babenskas Automated Software Testing
2/36
Toi!s
0Automated software testing" #$at%#$&%
0Automated 's" Manua(0)ommon t&es of automated testing
met$ods
0Automated software testing framework0*ummar&
2
7/23/2019 e Babenskas Automated Software Testing
3/36
#$at is automated software
testing%A ro!ess w$i!$ !o'ers+0t$e use of software to !ontro( t$eee!ution of tests
0t$e !omarison of a!tua( out!omes toredi!ted out!omes
0t$e setting u of test re!onditions0ot$er test !ontro( and test reortingfun!tions"
7/23/2019 e Babenskas Automated Software Testing
4/36
#$& testing is !riti!a(%
.
7/23/2019 e Babenskas Automated Software Testing
5/36
#$& automated testing%
0A time sa'ings trans(ates dire!t(& into !ostsa'ings
0Imro'es testing rodu!ti'it&
0Imro'es a!!ura!&
0In!reases test !o'erage
0oes w$at manua( testing !annot do
7/23/2019 e Babenskas Automated Software Testing
6/36
Eort and )ost0
et3s assume 4 test !ases0 Eort re5uired to run a(( 4 manua((& 67 10 min
0 Eort re5uired to write unit tests for a(( 4 !ases 67 10 min
0 Eort re5uired to run unit tests for a(( 4 !ases 67 8 1 min
0 9umber of testing iterations 67
0Tota( manua( testing time 67 0 min0Tota( unit testing time 67 10 min
Release Manual Test Auto Test
Manual Test
Cumulative
1 10 10 10
2 10 0 20
3 10 0 30
4 10 0 40
5 10 0 50
4
7/23/2019 e Babenskas Automated Software Testing
7/36
Automated 's" Manua(
testing
09ot a(( tests !an be automated andmost times is di:!u(t to de!ide w$atto automate and w$at to manua((&test"
;
7/23/2019 e Babenskas Automated Software Testing
8/36
Ad'antagesAutomated Testing Manual Testing
< 9eed to run a set of testsreeated(&
< Test !ases $a'e to be runa sma(( number of times
< =e(s erforming
>!omatibi(it& testing> ?ondierent !on@gurations and(atforms
< A((ows t$e tester to
erform more se!i@! tests
7/23/2019 e Babenskas Automated Software Testing
9/36
isad'antages D ot$er
fa!torsAutomated Testing Manual Testing< Its more eensi'e toautomate ?bigger initia(in'estments
< Manua( tests !an be'er& time !onsuming
< ou !annot automate
e'er&t$ing, some testssti(( $a'e to be donemanua((&
< For e'er& re(ease &ou
must rerun t$e same setof tests w$i!$ !an betiresome
Other Factors
< T$e erforman!e of &our test too(s< T$e know(edge (e'e( of &our testing team< T$e !ontinuous growt$ of software to betested
G
7/23/2019 e Babenskas Automated Software Testing
10/36
)ommon t&es of automated
testing met$ods0Monke& testing
0)ature / (a&ba!k
0)ode-based ?Hnit testing
0Inte((igent test automation
10
7/23/2019 e Babenskas Automated Software Testing
11/36
Monke& testing
0andom(& se(e!ting inuts from a (arge rangeof 'a(ues and monitoring if e!etions aret$rown"
0For eam(e, a monke& test !an enterrandom strings into tet boes to ensure$and(ing of a(( ossib(e user inut"
0 It a(ies not on(& for JHI or #EB testing, buta(so for Hnit testing"
11
7/23/2019 e Babenskas Automated Software Testing
12/36
Too(s eam(e
MonkeyK Android testing too(s"
T$e sim(est wa& to use t$e monke&
is wit$ t$e fo((owing !ommand, w$i!$wi(( (aun!$ t$e a(i!ation and send 00seudo-random e'ents to it"
L adb s$e(( monke& -' - &our"a!kage"name00
12
7/23/2019 e Babenskas Automated Software Testing
13/36
)ature / (a&ba!k
0It3s a set of software rograms t$at !atureuser inuts and stores it into a s!rit for(ater re(a&"
eeated testing !an be erformed 5ui!k(&"
oes not re5uire rogramming ski((s"
- #$en t$e JHI !$anges, inut se5uen!es
re'ious(& re!orded ma& no (onger be 'a(id"- i:!u(t to determine (o!ation of bugs"
1
7/23/2019 e Babenskas Automated Software Testing
14/36
Too(s eam(e
Autohotkey - Free ke&board ma!ro rogram"*uorts $otke&s for ke&board, mouse"
1.
7/23/2019 e Babenskas Automated Software Testing
15/36
)ode-based ?Hnit testing
0Indi'idua( units of sour!e !ode are tested todetermine if t$e& @t for use"
0Idea((&, ea!$ test !ase is indeendent fromt$e ot$ers
Bugs !an be found in ear(& de'e(omentstage
Eas& to test boundar& !ases- 9ot ee!ti'e for t$e integrated s&stem
testing
1
7/23/2019 e Babenskas Automated Software Testing
16/36
Hnit test-!ase generation
met$od0andom
0Jeneti!
0Nt$erO
14
7/23/2019 e Babenskas Automated Software Testing
17/36
andom
0Inut 'a(ues generated random(&
0un wit$ generated inuts
0Hser !$e!ks outut 'a(ues ?ass/fai(and sa'es t$em for (ater automati!!$e!king ?Nra!(e"
1;
7/23/2019 e Babenskas Automated Software Testing
18/36
Jeneti!T$is met$ods aroa!$ is to !ombine PM ?PAQAmode( (anguage and geneti! a(gorit$ms" PHnitis used as a test ee!ution (atform"
1C
7/23/2019 e Babenskas Automated Software Testing
19/36
Too(s eam(es
Parasoft Jtest
is anautomated Pa'atesting and stati!!ode ana(&sisrodu!t
TestGenis a !o((e!tion of oen-sour!e
too(s t$at automati!a((& generates unittest !ases"
1G
7/23/2019 e Babenskas Automated Software Testing
20/36
Inte((igent test
automation0A mode(-based met$od for generating tests from
a des!rition of an a(i!ation3s be$a'ior ?HM
Mode(s are sim(e to modif&, so mode(-basedtests are e!onomi!a( to maintain o'er t$e (ife ofa roje!t"
- Most of t$ese met$ods are resear!$ed on(&
t$eoreti!a((&"
20
7/23/2019 e Babenskas Automated Software Testing
21/36
N) HM
Inutsgenerator
HM
N)
Tested
software
Nra!(e
eort
Inuts )onstraints
)onstraintsNututsass /fai(
21
7/23/2019 e Babenskas Automated Software Testing
22/36
Automated testing
framework0A framework is an integrated s&stem
t$at sets t$e ru(es of automation of ase!i@! rodu!t"
0T$is s&stem integrates t$e fun!tion(ibraries, test data sour!es, obje!t detai(sand 'arious reusab(e modu(es"
0T$e framework ro'ides t$e basis of testautomation and sim(i@es t$eautomation eort"
22
7/23/2019 e Babenskas Automated Software Testing
23/36
)ommon t&es of
frameworks0ata-dri'en testing
0Re&word-dri'en testing
0=&brid testing
2
7/23/2019 e Babenskas Automated Software Testing
24/36
ata-dri'en testing
0ata-fo!used automation0Hser de@nes just data sets to run tests wit$
0ata is de@ned in eterna( data sour!e andde-!ou(ed from s!rit
0Jood @t for a sing(e a(i!ation testing,featuring mu(ti-en'ironment, big datasets,and rare(& !$anging Test )ases
+ -
Jood usabi(it& and re-usabi(it&
Sua(it& and !o'erage strong(& deends ont$e automation ski((s of im(ementing erson
Jood test ow !o'erage )ontinuous maintenan!e issues due to a bigamount of t$e !ode
Mu(ti-en'ironmentsuort Manua( 'a(idation is re5uired2.
7/23/2019 e Babenskas Automated Software Testing
25/36
Eam(eata set
Tests!rit
2
7/23/2019 e Babenskas Automated Software Testing
26/36
Re&word-dri'en testing
0Jood @t for a mu(ti-a(i!ation testingfeaturing mu(ti-en'ironment, big datasets, anda big number of s$ort and straig$t Test )ases
0Hsers de-!omose test !ases into reusab(e
a!tion ke&words09on-!oders are ab(e to aut$or automated test
!ases wit$ a!tion ke&words
+ -
Test de'e(oment doesn3tre5uire rogramming ski((s
ue to ke&words (imit doesn3t a((ow!o'ering 'er& !om(i!ated Test )ases
Jood test ow !o'erage Meta-(anguage se!i@! training isre5uired for sta
)oma!t !ode Manua( 'a(idation is re5uired 24
7/23/2019 e Babenskas Automated Software Testing
27/36
Eam(e
2;
7/23/2019 e Babenskas Automated Software Testing
28/36
=&brid testing0
)ombines t$e best of data-dri'en and ke&word-dri'en testing+0Hser de@nes data sets to run tests wit$
0Hser a(so de@nes ow !ontro( t$roug$ a!tion ke&words
0ata is searated from s!rit and stored in eterna(data sour!e wit$ a!tion ke&words
+ -
Test (an / Test *!enario !reation doesn3tre5uire rogramming or training on t$e
Testing Too(
e5uires initia( in'estmentfor design andim(ementation of t$eframework
)oma!t !ode and s!a(ab(e ar!$ite!ture
Bui(t-in stri!t 'a(idation2C
7/23/2019 e Babenskas Automated Software Testing
29/36
Framework !omonents0 Test generation (data and script generators!Too(s mig$t
!reate se!ia(iUed data su!$ as randomiUed emai( messages, orou(ate databases et!"
0 "ystem con#guration!Too(s mig$t reser'e or rerodu!es&stem arameters"
0 Test e$ecution!Too(s mig$t oerate t$e software itse(f, eit$ersimu(ating a user working t$roug$ t$e JHI and using ana(ternati'e testab(e interfa!e"
0 Oracles! An ora!(e is an& me!$anism b& w$i!$ we dete!tfai(ure or su!!ess" Too(s mig$t automati!a((& dete!t !ertainkinds of error !onditions in a rodu!t"
0 Acti%ity recording & co%erage analysis!Too(s mig$t wat!$testing as it $aens and retrose!ti'e(& reort w$at was andwas not tested"
0 Test management!Too(s mig$t re!ord test resu(ts, organiUetest ideas or metri!s"
2G
J ( f k
7/23/2019 e Babenskas Automated Software Testing
30/36
Jenera( frameworkstru!ture
0
)ommon fun!tiona( automated test
7/23/2019 e Babenskas Automated Software Testing
31/36
)ommon fun!tiona( automated testtoo(s
Tool + -
IBM /
ationa(Fun!tiona(
Tester?FT
Bui(t as E!(ise (ug-in wit$ fu((
IE, Pa'a suort and sour!emanagement
*uorts #eb-baseda(i!ations, "9et, Pa'a, *iebe(,*AV, owerBui(der, APAW, AdobeF(e, Adobe F do!uments
Fu(( JHI Nbje!t Ma reositor&
i!ense !ost
= /Mer!ur&Sui!k Testro ?ST
*uorts #eb 2"0, J#T, ojo,HI, *i('er(ig$t, F(e, Aja, PAQA,"9ET a(i!ations
Fu(( JHI Nbje!t Ma reositor&
*eam(ess integration wit$Sua(it&)enter
9o IE?!$anging inre(ease 10
i!ense !ost
Qisua(Basi!s!riting is (imited
*e(enium) D IE
Jood browser suortJood (anguage suort ?PAQA,
ub&, )X)an be easi(& etended as PHnit
suite -
9o JHI Nbje!treositor&
Nn(& web-baseda(i!ation suort
Too( suort is1
T ( (
7/23/2019 e Babenskas Automated Software Testing
32/36
Too(s eam(es
2
7/23/2019 e Babenskas Automated Software Testing
33/36
*ummar&
0Automated testing imro'es rodu!ti'it&,a!!ura!&, in!reases test !o'erage, sa'estime and !ost
09eed to de!ide w$at to automate and w$atto manua((& test
0T$e framework ro'ides t$e basis of test
automation and sim(i@es t$e automationeort"
7/23/2019 e Babenskas Automated Software Testing
34/36
More information0 $tt+//www"s(ides$are"net/(@ngerman/test-automation-best-r
!ati!es-wit$-soa-test-aroa!$0 $tt+//ountia"!s"ute"edu/utjm(/utest"$tm(
0 $tt+//www"s(ides$are"net/&ogindernat$/understanding-of-automation-framework-resentation
0 $tt+//www"ibm"!om/de'e(oerworks/rationa(/(ibrar&/G1"$tm(
0 $tt+//www"satis@!e"!om/resentations/agi(eauto"df
0 $tt+//automation-be&ond"!om/200G/0/0/test-automation-ra!ti!es-re!ord(a&ba!k/
0 $tt+//en"wikiedia"org/wiki/TestYautomation0 $tt+//www"s(ides$are"net/dire!ti/automated-testing-'s-
manua(-testing-G1121
.
http://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.ibm.com/developerworks/rational/library/591.htmlhttp://www.ibm.com/developerworks/rational/library/591.htmlhttp://www.ibm.com/developerworks/rational/library/591.htmlhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.html7/23/2019 e Babenskas Automated Software Testing
35/36
T$ank &ouZT$ank &ouZ
7/23/2019 e Babenskas Automated Software Testing
36/36
Suestion %%%Suestion %%%
0#$at is t$e bene@t of automated testing%
0#$at are t$e !ommon t&es of
automated testing met$ods%0#$at are t$e !ommon t&es of
frameworks%
0#$at !omonents !ome to framework%
0#$at are t$e eam(es of automatedtesting too(s%
4
Top Related