Test Documentation and Test Case Design

download Test Documentation and Test Case Design

of 48

Transcript of Test Documentation and Test Case Design

  • 8/17/2019 Test Documentation and Test Case Design

    1/48

    1

    Test documentation andTest case design

    Iana MourzaQA Lead/Release Lead

    VMware, Inc.

    2008

  • 8/17/2019 Test Documentation and Test Case Design

    2/48

    2

    Testing in Software De elo!"ent

    Testing

    = process of searching for software

    errors

    How and when do we start?

  • 8/17/2019 Test Documentation and Test Case Design

    3/48

  • 8/17/2019 Test Documentation and Test Case Design

    4/48

    4

    Software doc("entation

    &RD )&rod(ct Re*(ire"ent Doc("ent+S ) (nctional S!ecification+

    -I S!ec )-ser Interface S!ecification+Test &lanTest #aseTest S(ite

    Tracea ilit$ "atriRis Anal$sis "atri

  • 8/17/2019 Test Documentation and Test Case Design

    5/48

    5

    PRD (Product Requirement Document) 1 3at% set of software re*(ire"ents 1 3o% &rod(ct Mar eting, Sales, Tec3nical S(!!ort 1 3en% !lanning stage 1 3$% we need to now w3at t3e !rod(ct is s(!!osed to

    do 1 QA role%

    4 &artici!ate in re iews4 Anal$5e for co"!leteness4 S!ot a" ig(ities4 6ig3lig3t contradictions4 &ro ide feed ac on feat(res/(sa ilit$

    Software doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    6/48

    6

    Software doc("entation

    PRD (e amp!e)

  • 8/17/2019 Test Documentation and Test Case Design

    7/48

    7

    "# ("unctiona! #pecification) 1 3at% software design doc("ent7 1 3o% ngineering, Arc3itects7 1 3en% )!lanning+/design/)coding+ stage)s+7

    1 3$% we need to now 3ow t3e !rod(ct will edesigned7

    1 QA role%4 &artici!ate in re iews74 Anal$5e for co"!leteness74 S!ot a" ig(ities74 6ig3lig3t contradictions.

    Software doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    8/48

    8

    "# (e amp!e)

    Software doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    9/48

    9

    Test P!an 1 3at % a doc("ent descri ing t3e sco!e, a!!roac3,

    reso(rces and sc3ed(le of intended testing acti ities7identifies test ite"s, t3e feat(res to e tested, t3e testingtas s, w3o will do eac3 tas and an$ ris s re*(iringcontingenc$ !lanning7

    1 3o% QA7 1 3en% )!lanning+/design/coding/testing stage)s+7

    Test doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    10/48

    10

    Test P!an (cont$d) 1 3$%4 Di ide res!onsi ilities etween tea"s in ol ed7 if "ore t3an one

    QA tea" is in ol ed )ie, "an(al / a(to"ation, or nglis3 /Locali5ation+ 1 res!onsi ilities etween QA tea"s 7

    4 &lan for test reso(rces / ti"elines 74 &lan for test co erage74 &lan for 9S / D: / software de!lo$"ent and config(ration "odels

    co erage.' QA role%

    4 #reate and "aintain t3e doc("ent7

    4 Anal$5e for co"!leteness74 6a e it re iewed and signed $ &ro;ect Tea" leads/"anagers.

    Test doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    11/48

    11

    Test P!an (e amp!e)

    Test doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    12/48

    12

    Test %ase 1 3at% a set of in!(ts, e ec(tion !reconditions and e !ected

    o(tco"es de elo!ed for a !artic(lar o ;ecti e, s(c3 ase ercising a !artic(lar !rogra" !at3 or erif$ing co"!liancewit3 a s!ecific re*(ire"ent7

    1 3o% QA7

    1 3en% )!lanning+/)design+/coding/testing stage)s+7 1 3$%

    4 &lan test effort / reso(rces / ti"elines74 &lan / re iew test co erage74 Trac test e ec(tion !rogress74 Trac defects74 Trac software *(alit$ criteria / *(alit$ "etrics74 -nif$ &ass/ ail criteria across all testers74 &lanned/s$ste"atic testing s Ad'6oc.

    Test doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    13/48

    13

    Test doc("entationTest %ase (cont$d) 1 i e re*(ired ele"ents of a Test #ase%

    4 ID 1 (ni*(e identifier of a test case74 eat(res to e tested / ste!s / in!(t al(es 1 w3at $o(

    need to do74 !ected res(lt / o(t!(t al(es 1 w3at $o( are s(!!osed

    to get fro" a!!lication74 Act(al res(lt 1 w3at $o( reall$ get fro" a!!lication74 &ass / ail.

  • 8/17/2019 Test Documentation and Test Case Design

    14/48

    14

    Test doc("entationTest %ase (cont$d) 1 9!tional ele"ents of a Test #ase %

    4 Title 1 er al descri!tion indicati e of testcase o ;ecti e74 egati e class 1 for test e ec(tion !lanning74 Man(al / A(to"ata le / A(to"ated !ara"eter etc 1 for

    !lanning !(r!oses74 Test n iron"ent.

  • 8/17/2019 Test Documentation and Test Case Design

    15/48

    15

    Test doc("entationTest %ase (cont$d)

    1 In!(ts %4 T3ro(g3 t3e -I74 ro" interfacing s$ste"s or de ices74 iles7

    4 Data ases74 State74 n iron"ent.

    1 9(t!(ts %4 To -I74 To interfacing s$ste"s or de ices74 iles74 Data ases74 State74 Res!onse ti"e.

  • 8/17/2019 Test Documentation and Test Case Design

    16/48

    16

    Test doc("entationTest %ase (cont$d) 1 or"at 1 follow co"!an$ standards7 if no

    standards 1 c3oose t3e one t3at wor s est for$o(%

    4 MS ord doc("ent7

    4 MS cel doc("ent74 Me"o'li e !aragra!3s )MS ord, >ote!ad, ord!ad+.

    1 #lasses %4 &ositi e and >egati e74 (nctional, >on' (nctional and -I74 I"!licit erifications and e !licit erifications74 S$ste"atic testing and ad'3oc7

  • 8/17/2019 Test Documentation and Test Case Design

    17/48

  • 8/17/2019 Test Documentation and Test Case Design

    18/48

    18

    Test doc("entation

    Test %ase (e amp!e)

  • 8/17/2019 Test Documentation and Test Case Design

    19/48

    19

    Test doc("entationTest #uite 1 A doc("ent s!ecif$ing a se*(ence of actions for t3e

    e ec(tion of "(lti!le test cases7 1 &(r!ose% to !(t t3e test cases into an e ec(ta le order,

    alt3o(g3 indi id(al test cases "a$ 3a e an internal set ofste!s or !roced(res7

    1 Is t$!icall$ "an(al, if a(to"ated, t$!icall$ referred to as testscri!t )t3o(g3 "an(al !roced(res can also e a t$!e ofscri!t+7

    1 M(lti!le Test S(ites need to e organi5ed into so"e

    se*(ence 1 t3is defined t3e order in w3ic3 t3e test cases orscri!ts are to e r(n, w3at ti"ing considerations are, w3os3o(ld r(n t3e" etc.

  • 8/17/2019 Test Documentation and Test Case Design

    20/48

    20

    Tracea&i!it' matri 1 3at% doc("ent trac ing eac3 software feat(refro" &RD to S to Test docs )Test cases, Tests(ites+7

    1 3o% ngineers, QA7 1 3en% )design+/coding/testing stage)s+7 1 3$% we need to "a e s(re eac3 re*(ire"ent is

    co ered in S and Test cases7

    1 QA role%4 Anal$5e for co"!leteness74 Ma e s(re eac3 feat(re is re!resented74 6ig3lig3t ga!s.

    Test doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    21/48

    21

    Tracea&i!it' matri (e amp!e)PRD #ection "# #ection Test case otes

    ?.?.Validation of(ser logincredentials.

    @.?. -serlogin

    alidation.

    .?.@. -ser login wit3 !ro!ercredentials.

    .?.B. -ser login wit3 in alid

    (serna"e.

    .?. . -ser login wit3 in alid!assword.

    ?.2.Validation ofcredit cardinfor"ation.

    C.2.@. #reditcardinfor"ation

    erification.

    ?0.?.?. Valid credit cardinfor"ation in!(t.

    ?0.?.2. In alid credit cardn(" er.

    ?0.?. . In alid credit cardna"e.

    E

    Test doc("entation

  • 8/17/2019 Test Documentation and Test Case Design

    22/48

  • 8/17/2019 Test Documentation and Test Case Design

    23/48

    23

    Test design

    Testing e*e!s 1 +cceptance testing, or"al testing wit3 res!ect to (ser

    needs, re*(ire"ents, and (siness !rocesses cond(ctedto deter"ine w3et3er or not a s$ste" satisfies t3eacce!tance criteria and to ena le t3e (ser, c(sto"ers or

    ot3er a(t3ori5ed entit$ to deter"ine w3et3er or not toacce!t t3e s$ste". 1 #'stem testing, T3e !rocess of testing an integrated

    s$ste" to erif$ t3at it "eets s!ecified re*(ire"ents. 1 Integration testing, Testing !erfor"ed to e !ose defects

    in t3e interfaces and in t3e interactions etween integratedco"!onents or s$ste"s.

    1 %omponent testing, T3e testing of indi id(al softwareco"!onents.

  • 8/17/2019 Test Documentation and Test Case Design

    24/48

    24

    Test design

    Testing #trategies 1 De!end on De elo!"ent "odel. 1 Incrementa!, testing "od(les as t3e$ are de elo!ed, eac3

    !iece is tested se!aratel$. 9nce all ele"ents are tested,integration/s$ste" testing can e !erfor"ed.

    1 Re*(ires additional code to e written, (t allows toeasil$ identif$ t3e so(rce of error

    1 -ig -ang % testing is !erfor"ed on f(ll$ integrated s$ste",e er$t3ing is tested wit3 e er$t3ing else.

    1 >o e tra code needed, (t errors are 3ard to find.

  • 8/17/2019 Test Documentation and Test Case Design

    25/48

    25

    Test design

    Test T'pes 1 T3ere are se eral e$ t$!es of tests t3at 3el! i"!ro e t3e

    foc(s of t3e testing% 1 (nctional testing% testing s!ecific f(nctions 1 >on'f(nctional testing% testing c3aracteristics of t3e software and

    s$ste" 1 Str(ct(ral testing% testing t3e software str(ct(re or arc3itect(re 1 Re'testing )confir"ation+ and regression testing% testing related to

    c3anges 1 3ite and lac o testing

    1 ac3 test t$!e foc(ses on a !artic(lar test o ;ecti e 1 Test o ;ecti e% A reason or !(r!ose for designing and

    e ec(ting a test. 1 Test o ;ect% T3e co"!onent or s$ste" to e tested. 1 Test ite"% T3e indi id(al ele"ent to e tested. T3ere

    (s(all$ is one test o ;ect and "an$ test ite"s.

  • 8/17/2019 Test Documentation and Test Case Design

    26/48

    26

    Test design

    Test T'pes (cont$d) 1 "unctiona! testing % 1 Testing ased on an anal$sis of t3e s!ecification of t3e

    f(nctionalit$ of a co"!onent or s$ste". 1 T3e f(nctions are Fw3atF t3e s$ste" does%

    1 T3e$ are t$!icall$ defined or descri ed in wor !rod(ctss(c3 as a re*(ire"ents s!ecification, (se cases, or af(nctional s!ecification7

    1 T3e$ "a$ e (ndoc("ented7 1 (nctional tests are ased on ot3 e !licit and i"!licit

    feat(res and f(nctions7 1 T3e$ "a$ occ(r at all test le els, e.g., tests for

    co"!onents "a$ e ased on a co"!onent s!ecification7 1 (nctional testing foc(ses on t3e e ternal e3a ior of t3e

    software ) lac ' o testing+.

  • 8/17/2019 Test Documentation and Test Case Design

    27/48

    27

    Test design

    Test T'pes (cont$d) 1 on."unctiona! testing % 1 oc(ses on F3owF t3e s$ste" wor s7 1 >on'f(nctional tests are t3ose tests re*(ired to "eas(re

    c3aracteristics of s$ste"s and software t3at can e*(antified7

    1 T3ese *(antifications can ar$ and incl(de ite"s s(c3 as%res!onse ti"es, t3ro(g3!(t, ca!acit$ for !erfor"ancetesting etc.

    1 Testing t3e attri (tes of a co"!onent or s$ste" t3at do notrelate to f(nctionalit$, e.g. relia ilit$, efficienc$, (sa ilit$,"aintaina ilit$, co"!ati ilit$ and !orta ilit$.

  • 8/17/2019 Test Documentation and Test Case Design

    28/48

    28

    Test design

    Test T'pes (cont$d) 1 #tructura! (/hite &o ) testing % 1 Testing ased on an anal$sis of t3e internal str(ct(re of t3e

    co"!onent or s$ste" / arc3itect(re of t3e s$ste", as!ectss(c3 as a calling 3ierarc3$, data flow diagra", designs!ecification, etc.7

    1 Ma$ "a$ e !erfor"ed at all test le els ' s$ste", s$ste"integration, or acce!tance le els )e.g., to (siness "odelsor "en( str(ct(res+7

    1 Str(ct(ral tec3ni*(es are est (sed after s!ecification'ased tec3ni*(es7

    1 #an assist in "eas(ring t3e t3oro(g3ness of testing $assessing t3e degree of co erage of a str(ct(re7

    1 Tools can e (sed to "eas(re t3e code co erage.

  • 8/17/2019 Test Documentation and Test Case Design

    29/48

    29

    Test design

    Test T'pes (cont$d) 1 -!ac0 &o testing %

    1 T3e !rogra" is treated as lac o 7

    1 In!(ts are fed into t3e !rogra", o(t!(ts o ser ed7

    1 Searc3 for interesting and c3allenging in!(t co" inationsand conditions 1 t3e$ are "ost li el$ to e !ose an error.

  • 8/17/2019 Test Documentation and Test Case Design

    30/48

    30

    Test design

    Test T'pes (cont$d) 1 Regression testing (retesting) % 1 Retesting of a !re io(sl$ tested !rogra" following

    "odification to ens(re e isting f(nctionalit$ is wor ing!ro!erl$ and new defects/fa(lts 3a e not een introd(cedor (nco ered as a res(lt of t3e c3anges "ade7

    1 Tests s3o(ld e designed to e re!eata le 1 t3e$ are to e(sed for retesting7 t3e "ore defects fo(nd, t3e "ore oftent3e tests "a$ 3a e to r(n7

    1 (ll regression / &artial regression / >o regression 1 t3ee tent of regression is ased on t3e ris of not f(ndingdefects7

    1 A!!lies to f(nctional, non'f(nctional and str(ct(ral testing7 1

  • 8/17/2019 Test Documentation and Test Case Design

    31/48

    31

    Test design

    #tatic Test Techniques 1 #tatic Testing % 1 Testing of a co"!onent or s$ste" at s!ecification or

    i"!le"entation le el wit3o(t e ec(tion of t3e software7 1 >on'e ec(tion ased "et3od for c3ec ing life c$cle

    artifacts7 1 Man(al static tec3ni*(es% re iews )ins!ections,

    wal t3ro(g3s etc.+, for"al and infor"al7 1 A(to"ated tec3ni*(es% s(!!orting re iews, static anal$sis

    tools )co"!ilers+7 1 An$t3ing can e re iewed 1 &RD, S!ecs, Me"os,

    &ro!osals, -ser

  • 8/17/2019 Test Documentation and Test Case Design

    32/48

    32

    Test designTest %ase optimization 1 9!ti"i5ing Test design and !lanning

    "et3odologies%4 :o(ndar$ testing74 *(i alence classes74 Decision ta les74 State transitional diagra"s74 Ris Anal$sis.

  • 8/17/2019 Test Documentation and Test Case Design

    33/48

    33

    Test design1qui*a!ence c!ass partitioning ,

    1 A lac o test design tec3ni*(e in w3ic3 test cases aredesigned to e ec(te re!resentati es fro" e*(i alence!artitions. In !rinci!le, test cases are designed to co er eac3!artition at least once.

    1 #reates t3e "ini"(" n(" er of lac o tests needed to!ro ide "ini"(" test co erage

    1 Ste!s% 1 Identif$ e*(i alence classes, t3e in!(t al(es w3ic3 are

    treated t3e sa"e $ t3e software%

    1 Valid classes% legal in!(t al(es7 1 In alid classes% illegal or (nacce!ta le in!(t al(es7

    1 #reate a test case for eac3 e*(i alence class.

  • 8/17/2019 Test Documentation and Test Case Design

    34/48

    34

    Test design1qui*a!ence c!ass partitioning (cont$d),

    In*a!id 2a!id In*a!id

    GH?000 H?000'C0000 HC0000

    1qui*a!ence partition (c!ass), 1 A !ortion of an in!(t or o(t!(t do"ain for w3ic3 t3e e3a ior

    of a co"!onent or s$ste" is ass("ed to e t3e sa"e,ased on t3e s!ecification.

  • 8/17/2019 Test Documentation and Test Case Design

    35/48

    35

    Test design-oundar' *a!ue testing, 1 A lac o test design tec3ni*(e in w3ic3 test cases are

    designed ased on o(ndar$ al(es. 1 ac3 in!(t is tested at ot3 ends of its alid range)s+ and

    ;(st o(tside its alid range)s+. T3is "a es sense for n("ericranges and can e a!!lied to non'n("eric fields as well.

    Additional iss(es, s(c3 as field lengt3 for al!3a etic fields,can co"e into !la$ as o(ndaries.

    1 :o(ndar$ al(e% An in!(t al(e or o(t!(t al(e, w3ic3 is ont3e edge of an e*(i alence !artition or at t3e s"allestincre"ental distance on eit3er side of an edge, for e a"!let3e "ini"(" or "a i"(" al(e of a range.

  • 8/17/2019 Test Documentation and Test Case Design

    36/48

    36

    Test design-oundar' *a!ue testing (cont$d),

    1 R(n test cases at t3e o(ndar$ of eac3 in!(t% 1 J(st elow t3e o(ndar$7 1 J(st a o e t3e o(ndar$7

    1 T3e foc(s is on one re*(ire"ent at a ti"e7

    0 ? ?0 ??

    1 #an e co" ined across "(lti!le re*(ire"ents 1 allalid "ini"("s toget3er, all alid "a i"("s toget3er7

    1 In alid al(es s3o(ld not e co" ined.

  • 8/17/2019 Test Documentation and Test Case Design

    37/48

    37

    Test designDecision ta&!e,

    co"!osed of rows and col("ns, se!arated into*(adrants%

    #onditions #ondition Alternati es

    Actions Action ntries

  • 8/17/2019 Test Documentation and Test Case Design

    38/48

    38

    Test designDecision ta&!e,

  • 8/17/2019 Test Documentation and Test Case Design

    39/48

    39

    Test design#tate transitiona! diagrams,

    Identif$ a finite n(" er of states t3e "odel e ec(tion goest3ro(g3#reate a state transition diagra" s3owing 3ow t3e "odeltransitions fro" one state to t3e ot3er

    Assess t3e "odel acc(rac$ $ analyzing t3e conditions(nder w3ic3 a state c3ange occ(rs#tate transition, A transition etween two states of aco"!onent or s$ste".

  • 8/17/2019 Test Documentation and Test Case Design

    40/48

    40

    Test design#tate transitiona! diagrams (cont$d),

    1 #ircles/elli!ses are s tates 1 Lines re!resent transitions etween states

    1 Te t re!resents t3e e*ents t3at ca(se transitions 1 T3e solid circle re!resents an initia! state 1 A solid circle or elli!ses/circles wit3 no e it lines )transitions+ are

    fina! states 1 In t3e e a"!le a o e, "ini"al n(" er of test cases to co er eac3

    state is two

  • 8/17/2019 Test Documentation and Test Case Design

    41/48

  • 8/17/2019 Test Documentation and Test Case Design

    42/48

    42

    Ris0 +na!'sis (cont$d), 1 /ho, &M, Tec3 S(!!ort, Sales, ngineers, QA7 1 /hen, )design+/coding/testing stage)s+7

    1 /h',

    1 It 3el!s (s c3oose t3e est test tec3ni*(es 1 It 3el!s (s define t3e e tent of testing to e carried o(t 1 T3e 3ig3er t3e ris , t3e "ore foc(s gi en 1 It allows for t3e !rioriti5ation of t3e testing 1 Atte"!t to find t3e critical defects as earl$ as !ossi le 1

    Are t3ere an$ non'testing acti ities t3at can ee"!lo$ed to red(ce ris e.g., !ro ide training toine !erienced !ersonnel

    Test design

  • 8/17/2019 Test Documentation and Test Case Design

    43/48

    43

    Test designRis0 +na!'sis (sca!e 3.34),

    "eature i0e!ihoodof fai!ure(1ng5 6+5

    Tech#upport)

    Impact offai!ure(PM5 Tech

    #upport5#a!es5 6+)

    Ris0"actor

    ?. Login 2 ?0 20

    2. #reate

    data aserecord

    B C B

    . Modif$data aserecord

    ?8

  • 8/17/2019 Test Documentation and Test Case Design

    44/48

    44

    Test designRis0 +na!'sis (e amp!e)

  • 8/17/2019 Test Documentation and Test Case Design

    45/48

    45

    Test doc("entation%atching an 1rror .7 -ug Report

    1 Re!rod(cing an error7 1 Re!orting an error%

    1 :(g re!ort 1 "ain ele"ents% 1 ID 7 1 3at is t3e !ro le" )w3at 3a!!ened+7 1 3ere t3e !ro le" occ(rred7 1 Ste!s to re!rod(ce.

    1 :(g re!ort 1 str(ct(re% 1 ID7 1 Title/s3ort descri!tion7 1 Long descri!tion )ste!s to re!rod(ce+7 1 &riorit$7 1 Se erit$7 1 &ro;ect ID / "ilestone / release.

  • 8/17/2019 Test Documentation and Test Case Design

    46/48

    46

    QNA

  • 8/17/2019 Test Documentation and Test Case Design

    47/48

    47

    6o"ewor%hapters 85 39

  • 8/17/2019 Test Documentation and Test Case Design

    48/48

    T3an $o(O

    and 6a!!$ TestingO