Test Documentation and Test Case Design
-
Upload
rabindra-das -
Category
Documents
-
view
230 -
download
1
Transcript of Test Documentation and Test Case Design
-
8/17/2019 Test Documentation and Test Case Design
1/48
1
Test documentation andTest case design
Iana MourzaQA Lead/Release Lead
VMware, Inc.
2008
-
8/17/2019 Test Documentation and Test Case Design
2/48
2
Testing in Software De elo!"ent
Testing
= process of searching for software
errors
How and when do we start?
-
8/17/2019 Test Documentation and Test Case Design
3/48
-
8/17/2019 Test Documentation and Test Case Design
4/48
4
Software doc("entation
&RD )&rod(ct Re*(ire"ent Doc("ent+S ) (nctional S!ecification+
-I S!ec )-ser Interface S!ecification+Test &lanTest #aseTest S(ite
Tracea ilit$ "atriRis Anal$sis "atri
-
8/17/2019 Test Documentation and Test Case Design
5/48
5
PRD (Product Requirement Document) 1 3at% set of software re*(ire"ents 1 3o% &rod(ct Mar eting, Sales, Tec3nical S(!!ort 1 3en% !lanning stage 1 3$% we need to now w3at t3e !rod(ct is s(!!osed to
do 1 QA role%
4 &artici!ate in re iews4 Anal$5e for co"!leteness4 S!ot a" ig(ities4 6ig3lig3t contradictions4 &ro ide feed ac on feat(res/(sa ilit$
Software doc("entation
-
8/17/2019 Test Documentation and Test Case Design
6/48
6
Software doc("entation
PRD (e amp!e)
-
8/17/2019 Test Documentation and Test Case Design
7/48
7
"# ("unctiona! #pecification) 1 3at% software design doc("ent7 1 3o% ngineering, Arc3itects7 1 3en% )!lanning+/design/)coding+ stage)s+7
1 3$% we need to now 3ow t3e !rod(ct will edesigned7
1 QA role%4 &artici!ate in re iews74 Anal$5e for co"!leteness74 S!ot a" ig(ities74 6ig3lig3t contradictions.
Software doc("entation
-
8/17/2019 Test Documentation and Test Case Design
8/48
8
"# (e amp!e)
Software doc("entation
-
8/17/2019 Test Documentation and Test Case Design
9/48
9
Test P!an 1 3at % a doc("ent descri ing t3e sco!e, a!!roac3,
reso(rces and sc3ed(le of intended testing acti ities7identifies test ite"s, t3e feat(res to e tested, t3e testingtas s, w3o will do eac3 tas and an$ ris s re*(iringcontingenc$ !lanning7
1 3o% QA7 1 3en% )!lanning+/design/coding/testing stage)s+7
Test doc("entation
-
8/17/2019 Test Documentation and Test Case Design
10/48
10
Test P!an (cont$d) 1 3$%4 Di ide res!onsi ilities etween tea"s in ol ed7 if "ore t3an one
QA tea" is in ol ed )ie, "an(al / a(to"ation, or nglis3 /Locali5ation+ 1 res!onsi ilities etween QA tea"s 7
4 &lan for test reso(rces / ti"elines 74 &lan for test co erage74 &lan for 9S / D: / software de!lo$"ent and config(ration "odels
co erage.' QA role%
4 #reate and "aintain t3e doc("ent7
4 Anal$5e for co"!leteness74 6a e it re iewed and signed $ &ro;ect Tea" leads/"anagers.
Test doc("entation
-
8/17/2019 Test Documentation and Test Case Design
11/48
11
Test P!an (e amp!e)
Test doc("entation
-
8/17/2019 Test Documentation and Test Case Design
12/48
12
Test %ase 1 3at% a set of in!(ts, e ec(tion !reconditions and e !ected
o(tco"es de elo!ed for a !artic(lar o ;ecti e, s(c3 ase ercising a !artic(lar !rogra" !at3 or erif$ing co"!liancewit3 a s!ecific re*(ire"ent7
1 3o% QA7
1 3en% )!lanning+/)design+/coding/testing stage)s+7 1 3$%
4 &lan test effort / reso(rces / ti"elines74 &lan / re iew test co erage74 Trac test e ec(tion !rogress74 Trac defects74 Trac software *(alit$ criteria / *(alit$ "etrics74 -nif$ &ass/ ail criteria across all testers74 &lanned/s$ste"atic testing s Ad'6oc.
Test doc("entation
-
8/17/2019 Test Documentation and Test Case Design
13/48
13
Test doc("entationTest %ase (cont$d) 1 i e re*(ired ele"ents of a Test #ase%
4 ID 1 (ni*(e identifier of a test case74 eat(res to e tested / ste!s / in!(t al(es 1 w3at $o(
need to do74 !ected res(lt / o(t!(t al(es 1 w3at $o( are s(!!osed
to get fro" a!!lication74 Act(al res(lt 1 w3at $o( reall$ get fro" a!!lication74 &ass / ail.
-
8/17/2019 Test Documentation and Test Case Design
14/48
14
Test doc("entationTest %ase (cont$d) 1 9!tional ele"ents of a Test #ase %
4 Title 1 er al descri!tion indicati e of testcase o ;ecti e74 egati e class 1 for test e ec(tion !lanning74 Man(al / A(to"ata le / A(to"ated !ara"eter etc 1 for
!lanning !(r!oses74 Test n iron"ent.
-
8/17/2019 Test Documentation and Test Case Design
15/48
15
Test doc("entationTest %ase (cont$d)
1 In!(ts %4 T3ro(g3 t3e -I74 ro" interfacing s$ste"s or de ices74 iles7
4 Data ases74 State74 n iron"ent.
1 9(t!(ts %4 To -I74 To interfacing s$ste"s or de ices74 iles74 Data ases74 State74 Res!onse ti"e.
-
8/17/2019 Test Documentation and Test Case Design
16/48
16
Test doc("entationTest %ase (cont$d) 1 or"at 1 follow co"!an$ standards7 if no
standards 1 c3oose t3e one t3at wor s est for$o(%
4 MS ord doc("ent7
4 MS cel doc("ent74 Me"o'li e !aragra!3s )MS ord, >ote!ad, ord!ad+.
1 #lasses %4 &ositi e and >egati e74 (nctional, >on' (nctional and -I74 I"!licit erifications and e !licit erifications74 S$ste"atic testing and ad'3oc7
-
8/17/2019 Test Documentation and Test Case Design
17/48
-
8/17/2019 Test Documentation and Test Case Design
18/48
18
Test doc("entation
Test %ase (e amp!e)
-
8/17/2019 Test Documentation and Test Case Design
19/48
19
Test doc("entationTest #uite 1 A doc("ent s!ecif$ing a se*(ence of actions for t3e
e ec(tion of "(lti!le test cases7 1 &(r!ose% to !(t t3e test cases into an e ec(ta le order,
alt3o(g3 indi id(al test cases "a$ 3a e an internal set ofste!s or !roced(res7
1 Is t$!icall$ "an(al, if a(to"ated, t$!icall$ referred to as testscri!t )t3o(g3 "an(al !roced(res can also e a t$!e ofscri!t+7
1 M(lti!le Test S(ites need to e organi5ed into so"e
se*(ence 1 t3is defined t3e order in w3ic3 t3e test cases orscri!ts are to e r(n, w3at ti"ing considerations are, w3os3o(ld r(n t3e" etc.
-
8/17/2019 Test Documentation and Test Case Design
20/48
20
Tracea&i!it' matri 1 3at% doc("ent trac ing eac3 software feat(refro" &RD to S to Test docs )Test cases, Tests(ites+7
1 3o% ngineers, QA7 1 3en% )design+/coding/testing stage)s+7 1 3$% we need to "a e s(re eac3 re*(ire"ent is
co ered in S and Test cases7
1 QA role%4 Anal$5e for co"!leteness74 Ma e s(re eac3 feat(re is re!resented74 6ig3lig3t ga!s.
Test doc("entation
-
8/17/2019 Test Documentation and Test Case Design
21/48
21
Tracea&i!it' matri (e amp!e)PRD #ection "# #ection Test case otes
?.?.Validation of(ser logincredentials.
@.?. -serlogin
alidation.
.?.@. -ser login wit3 !ro!ercredentials.
.?.B. -ser login wit3 in alid
(serna"e.
.?. . -ser login wit3 in alid!assword.
?.2.Validation ofcredit cardinfor"ation.
C.2.@. #reditcardinfor"ation
erification.
?0.?.?. Valid credit cardinfor"ation in!(t.
?0.?.2. In alid credit cardn(" er.
?0.?. . In alid credit cardna"e.
E
Test doc("entation
-
8/17/2019 Test Documentation and Test Case Design
22/48
-
8/17/2019 Test Documentation and Test Case Design
23/48
23
Test design
Testing e*e!s 1 +cceptance testing, or"al testing wit3 res!ect to (ser
needs, re*(ire"ents, and (siness !rocesses cond(ctedto deter"ine w3et3er or not a s$ste" satisfies t3eacce!tance criteria and to ena le t3e (ser, c(sto"ers or
ot3er a(t3ori5ed entit$ to deter"ine w3et3er or not toacce!t t3e s$ste". 1 #'stem testing, T3e !rocess of testing an integrated
s$ste" to erif$ t3at it "eets s!ecified re*(ire"ents. 1 Integration testing, Testing !erfor"ed to e !ose defects
in t3e interfaces and in t3e interactions etween integratedco"!onents or s$ste"s.
1 %omponent testing, T3e testing of indi id(al softwareco"!onents.
-
8/17/2019 Test Documentation and Test Case Design
24/48
24
Test design
Testing #trategies 1 De!end on De elo!"ent "odel. 1 Incrementa!, testing "od(les as t3e$ are de elo!ed, eac3
!iece is tested se!aratel$. 9nce all ele"ents are tested,integration/s$ste" testing can e !erfor"ed.
1 Re*(ires additional code to e written, (t allows toeasil$ identif$ t3e so(rce of error
1 -ig -ang % testing is !erfor"ed on f(ll$ integrated s$ste",e er$t3ing is tested wit3 e er$t3ing else.
1 >o e tra code needed, (t errors are 3ard to find.
-
8/17/2019 Test Documentation and Test Case Design
25/48
25
Test design
Test T'pes 1 T3ere are se eral e$ t$!es of tests t3at 3el! i"!ro e t3e
foc(s of t3e testing% 1 (nctional testing% testing s!ecific f(nctions 1 >on'f(nctional testing% testing c3aracteristics of t3e software and
s$ste" 1 Str(ct(ral testing% testing t3e software str(ct(re or arc3itect(re 1 Re'testing )confir"ation+ and regression testing% testing related to
c3anges 1 3ite and lac o testing
1 ac3 test t$!e foc(ses on a !artic(lar test o ;ecti e 1 Test o ;ecti e% A reason or !(r!ose for designing and
e ec(ting a test. 1 Test o ;ect% T3e co"!onent or s$ste" to e tested. 1 Test ite"% T3e indi id(al ele"ent to e tested. T3ere
(s(all$ is one test o ;ect and "an$ test ite"s.
-
8/17/2019 Test Documentation and Test Case Design
26/48
26
Test design
Test T'pes (cont$d) 1 "unctiona! testing % 1 Testing ased on an anal$sis of t3e s!ecification of t3e
f(nctionalit$ of a co"!onent or s$ste". 1 T3e f(nctions are Fw3atF t3e s$ste" does%
1 T3e$ are t$!icall$ defined or descri ed in wor !rod(ctss(c3 as a re*(ire"ents s!ecification, (se cases, or af(nctional s!ecification7
1 T3e$ "a$ e (ndoc("ented7 1 (nctional tests are ased on ot3 e !licit and i"!licit
feat(res and f(nctions7 1 T3e$ "a$ occ(r at all test le els, e.g., tests for
co"!onents "a$ e ased on a co"!onent s!ecification7 1 (nctional testing foc(ses on t3e e ternal e3a ior of t3e
software ) lac ' o testing+.
-
8/17/2019 Test Documentation and Test Case Design
27/48
27
Test design
Test T'pes (cont$d) 1 on."unctiona! testing % 1 oc(ses on F3owF t3e s$ste" wor s7 1 >on'f(nctional tests are t3ose tests re*(ired to "eas(re
c3aracteristics of s$ste"s and software t3at can e*(antified7
1 T3ese *(antifications can ar$ and incl(de ite"s s(c3 as%res!onse ti"es, t3ro(g3!(t, ca!acit$ for !erfor"ancetesting etc.
1 Testing t3e attri (tes of a co"!onent or s$ste" t3at do notrelate to f(nctionalit$, e.g. relia ilit$, efficienc$, (sa ilit$,"aintaina ilit$, co"!ati ilit$ and !orta ilit$.
-
8/17/2019 Test Documentation and Test Case Design
28/48
28
Test design
Test T'pes (cont$d) 1 #tructura! (/hite &o ) testing % 1 Testing ased on an anal$sis of t3e internal str(ct(re of t3e
co"!onent or s$ste" / arc3itect(re of t3e s$ste", as!ectss(c3 as a calling 3ierarc3$, data flow diagra", designs!ecification, etc.7
1 Ma$ "a$ e !erfor"ed at all test le els ' s$ste", s$ste"integration, or acce!tance le els )e.g., to (siness "odelsor "en( str(ct(res+7
1 Str(ct(ral tec3ni*(es are est (sed after s!ecification'ased tec3ni*(es7
1 #an assist in "eas(ring t3e t3oro(g3ness of testing $assessing t3e degree of co erage of a str(ct(re7
1 Tools can e (sed to "eas(re t3e code co erage.
-
8/17/2019 Test Documentation and Test Case Design
29/48
29
Test design
Test T'pes (cont$d) 1 -!ac0 &o testing %
1 T3e !rogra" is treated as lac o 7
1 In!(ts are fed into t3e !rogra", o(t!(ts o ser ed7
1 Searc3 for interesting and c3allenging in!(t co" inationsand conditions 1 t3e$ are "ost li el$ to e !ose an error.
-
8/17/2019 Test Documentation and Test Case Design
30/48
30
Test design
Test T'pes (cont$d) 1 Regression testing (retesting) % 1 Retesting of a !re io(sl$ tested !rogra" following
"odification to ens(re e isting f(nctionalit$ is wor ing!ro!erl$ and new defects/fa(lts 3a e not een introd(cedor (nco ered as a res(lt of t3e c3anges "ade7
1 Tests s3o(ld e designed to e re!eata le 1 t3e$ are to e(sed for retesting7 t3e "ore defects fo(nd, t3e "ore oftent3e tests "a$ 3a e to r(n7
1 (ll regression / &artial regression / >o regression 1 t3ee tent of regression is ased on t3e ris of not f(ndingdefects7
1 A!!lies to f(nctional, non'f(nctional and str(ct(ral testing7 1
-
8/17/2019 Test Documentation and Test Case Design
31/48
31
Test design
#tatic Test Techniques 1 #tatic Testing % 1 Testing of a co"!onent or s$ste" at s!ecification or
i"!le"entation le el wit3o(t e ec(tion of t3e software7 1 >on'e ec(tion ased "et3od for c3ec ing life c$cle
artifacts7 1 Man(al static tec3ni*(es% re iews )ins!ections,
wal t3ro(g3s etc.+, for"al and infor"al7 1 A(to"ated tec3ni*(es% s(!!orting re iews, static anal$sis
tools )co"!ilers+7 1 An$t3ing can e re iewed 1 &RD, S!ecs, Me"os,
&ro!osals, -ser
-
8/17/2019 Test Documentation and Test Case Design
32/48
32
Test designTest %ase optimization 1 9!ti"i5ing Test design and !lanning
"et3odologies%4 :o(ndar$ testing74 *(i alence classes74 Decision ta les74 State transitional diagra"s74 Ris Anal$sis.
-
8/17/2019 Test Documentation and Test Case Design
33/48
33
Test design1qui*a!ence c!ass partitioning ,
1 A lac o test design tec3ni*(e in w3ic3 test cases aredesigned to e ec(te re!resentati es fro" e*(i alence!artitions. In !rinci!le, test cases are designed to co er eac3!artition at least once.
1 #reates t3e "ini"(" n(" er of lac o tests needed to!ro ide "ini"(" test co erage
1 Ste!s% 1 Identif$ e*(i alence classes, t3e in!(t al(es w3ic3 are
treated t3e sa"e $ t3e software%
1 Valid classes% legal in!(t al(es7 1 In alid classes% illegal or (nacce!ta le in!(t al(es7
1 #reate a test case for eac3 e*(i alence class.
-
8/17/2019 Test Documentation and Test Case Design
34/48
34
Test design1qui*a!ence c!ass partitioning (cont$d),
In*a!id 2a!id In*a!id
GH?000 H?000'C0000 HC0000
1qui*a!ence partition (c!ass), 1 A !ortion of an in!(t or o(t!(t do"ain for w3ic3 t3e e3a ior
of a co"!onent or s$ste" is ass("ed to e t3e sa"e,ased on t3e s!ecification.
-
8/17/2019 Test Documentation and Test Case Design
35/48
35
Test design-oundar' *a!ue testing, 1 A lac o test design tec3ni*(e in w3ic3 test cases are
designed ased on o(ndar$ al(es. 1 ac3 in!(t is tested at ot3 ends of its alid range)s+ and
;(st o(tside its alid range)s+. T3is "a es sense for n("ericranges and can e a!!lied to non'n("eric fields as well.
Additional iss(es, s(c3 as field lengt3 for al!3a etic fields,can co"e into !la$ as o(ndaries.
1 :o(ndar$ al(e% An in!(t al(e or o(t!(t al(e, w3ic3 is ont3e edge of an e*(i alence !artition or at t3e s"allestincre"ental distance on eit3er side of an edge, for e a"!let3e "ini"(" or "a i"(" al(e of a range.
-
8/17/2019 Test Documentation and Test Case Design
36/48
36
Test design-oundar' *a!ue testing (cont$d),
1 R(n test cases at t3e o(ndar$ of eac3 in!(t% 1 J(st elow t3e o(ndar$7 1 J(st a o e t3e o(ndar$7
1 T3e foc(s is on one re*(ire"ent at a ti"e7
0 ? ?0 ??
1 #an e co" ined across "(lti!le re*(ire"ents 1 allalid "ini"("s toget3er, all alid "a i"("s toget3er7
1 In alid al(es s3o(ld not e co" ined.
-
8/17/2019 Test Documentation and Test Case Design
37/48
37
Test designDecision ta&!e,
co"!osed of rows and col("ns, se!arated into*(adrants%
#onditions #ondition Alternati es
Actions Action ntries
-
8/17/2019 Test Documentation and Test Case Design
38/48
38
Test designDecision ta&!e,
-
8/17/2019 Test Documentation and Test Case Design
39/48
39
Test design#tate transitiona! diagrams,
Identif$ a finite n(" er of states t3e "odel e ec(tion goest3ro(g3#reate a state transition diagra" s3owing 3ow t3e "odeltransitions fro" one state to t3e ot3er
Assess t3e "odel acc(rac$ $ analyzing t3e conditions(nder w3ic3 a state c3ange occ(rs#tate transition, A transition etween two states of aco"!onent or s$ste".
-
8/17/2019 Test Documentation and Test Case Design
40/48
40
Test design#tate transitiona! diagrams (cont$d),
1 #ircles/elli!ses are s tates 1 Lines re!resent transitions etween states
1 Te t re!resents t3e e*ents t3at ca(se transitions 1 T3e solid circle re!resents an initia! state 1 A solid circle or elli!ses/circles wit3 no e it lines )transitions+ are
fina! states 1 In t3e e a"!le a o e, "ini"al n(" er of test cases to co er eac3
state is two
-
8/17/2019 Test Documentation and Test Case Design
41/48
-
8/17/2019 Test Documentation and Test Case Design
42/48
42
Ris0 +na!'sis (cont$d), 1 /ho, &M, Tec3 S(!!ort, Sales, ngineers, QA7 1 /hen, )design+/coding/testing stage)s+7
1 /h',
1 It 3el!s (s c3oose t3e est test tec3ni*(es 1 It 3el!s (s define t3e e tent of testing to e carried o(t 1 T3e 3ig3er t3e ris , t3e "ore foc(s gi en 1 It allows for t3e !rioriti5ation of t3e testing 1 Atte"!t to find t3e critical defects as earl$ as !ossi le 1
Are t3ere an$ non'testing acti ities t3at can ee"!lo$ed to red(ce ris e.g., !ro ide training toine !erienced !ersonnel
Test design
-
8/17/2019 Test Documentation and Test Case Design
43/48
43
Test designRis0 +na!'sis (sca!e 3.34),
"eature i0e!ihoodof fai!ure(1ng5 6+5
Tech#upport)
Impact offai!ure(PM5 Tech
#upport5#a!es5 6+)
Ris0"actor
?. Login 2 ?0 20
2. #reate
data aserecord
B C B
. Modif$data aserecord
?8
-
8/17/2019 Test Documentation and Test Case Design
44/48
44
Test designRis0 +na!'sis (e amp!e)
-
8/17/2019 Test Documentation and Test Case Design
45/48
45
Test doc("entation%atching an 1rror .7 -ug Report
1 Re!rod(cing an error7 1 Re!orting an error%
1 :(g re!ort 1 "ain ele"ents% 1 ID 7 1 3at is t3e !ro le" )w3at 3a!!ened+7 1 3ere t3e !ro le" occ(rred7 1 Ste!s to re!rod(ce.
1 :(g re!ort 1 str(ct(re% 1 ID7 1 Title/s3ort descri!tion7 1 Long descri!tion )ste!s to re!rod(ce+7 1 &riorit$7 1 Se erit$7 1 &ro;ect ID / "ilestone / release.
-
8/17/2019 Test Documentation and Test Case Design
46/48
46
QNA
-
8/17/2019 Test Documentation and Test Case Design
47/48
47
6o"ewor%hapters 85 39
-
8/17/2019 Test Documentation and Test Case Design
48/48
T3an $o(O
and 6a!!$ TestingO