FusionTesting-MaximizingTestExecution.ppt

7/25/2019 FusionTesting-MaximizingTestExecution.ppt

1/21

Fusion Testing

Maximizing Test Execution

By: James Tischart


2/21

About Me Director - Product Delivery SaaS at McAfee 15+ years experience in testing and engineering

Multiple certifications in Agile and Testing

Passionate about Testing as an eng discipline ascience and an art

!ontinue to c"allenge t"e status #uo test ne$approac"es and al$ays strive to i%prove

Support t"e legiti%acy of t"e testing professional int"e broader engineering $orld


3/21

What is F usion Testing?An occurrence that involves the production of a union

&rgani'ation of structured testing

(reedo% of exploratory testing )igors of auto%ated testing !o%bined into one test %et"odology

To achieve

Maxi%i'e code execution *ncreased test coverage )educed test artifact and docu%entation ig"er #uality for users *%proved data for organi'ation


4/21

Fusion

F ocus , start your day $it" 15 %inutes of t"oug"t

Usage , "o$ $ill your users $or $it" t"e syste%

Scope , decide on t"e scope of everyt"ingInitiate , .ust go and explore

O rgani'e , create a plan / be ready to deviate fro% it

Note , eep trac of your exploration to retrace steps


5/21

Fusion Testing GuidanceMaximize test execution with Fusion b !

10 *dentify oals for Testing

20 !"oose t"e rig"t %ix of %et"odologies

30 4tili'e Tests ists to guide exploration

60 Auto%ate t"e )ig"t T"ings

50 Docu%ent at t"e )ig"t Detail


6/21

Fusion Implementation10 uide your tea% t"roug" t"e c"ange

10 !reate a !"ange Message

10 Mentor your tea% %e%bers

20 Train t"e tea% in testing and engineering

30 !reate Metrics to %easure success / failure


7/21

Fusion & Planning"elease Test #lan

Set t"e goals for t"e overall release

Plan t"e test environ%ent

&rgani'e t"e %etrics and perfor%ance

Plan a budget resources and a ti%e fra%e

Plan your auto%ation and tool set

Plan at a "ig" level

Iterative Test #lan 7ery effective in Agile i%ple%entations

8rea up t"e testing into iterations

o$ level details for t"e ti%e fra%e 9eep to t$o $ee s or less Detail t"e testing to acco%plis" Detail t"e perfor%ance tests ist $"at $ill not be tested *dentify ris s to t"e plan Plan $"at )egression Testing *dentify $"at test-lists need to be $ritten

$ail #lan *ndividually created eac" %orning Provides guidance and goals for t"e day

Details t"e execution plan

Test %ists A grouping of test ideas

:o specifics in t"e test list

Priority variations / guide for exploration


8/21


9/21

Identify Defect Traits (or ne$ products use "istorical traits fro% si%ilar pro.ects

Analy'e Severity 1 !usto%er reported incidents

Deter%ine t"e trends to t"ose defects in coding / testing

!onstrain t"e revie$ to a calendar year of defects

:ext analy'e Sev 1 internally found and fixed issues and deter%ine t"e traits

*nfor% t"e developers / testers of t"e traits

!reate test lists or test ideas to cover t"ese traits

)epeat t"e process $it" eac" severity level you use


10/21

Fusion & Automation$etermine our la ers for automation! Server

$atabases' server processes Machine interaction points #rocess interaction with other

la ers

Middleware(A#I la ers

User Interface

Optimize our test artifacts $on)t duplicate test*cases Use automation for

traceabilit +


11/21

Test Results & Metrics

There will alwa s be interest in the test data no matter the test structure+

,! o$ can you accurately assess #uality $"en t"e testing co%binationsexceed t"e particles of t"e universe ?e need to present $"at $as tested $"at "as not been testedand support t"e assess%ent used to %a e t"is prioriti'ation0

@ of Test *deas xecuted based on Priority

@ of Test !ases passedBfailed versus total coverage t"at t"e tests represent

@ of Auto%ated Tests PassedB(ailed by @ of executed ti%es

Perfor%anceB8enc"%ar co%parisons by buildBiterationBrelease Tea% Cuality Satisfaction )ating , get t"e gut feeling of t"e tea%

&pen defects to "ig"lig"t potential issues t"at your custo%ers %ay find

-o to fusiontestin&+blo&spot+com for more details on these metrics

Since ou can)t test ever thin&' here are some ideas of resultsto report!


12/21

Traceability Matrices#ositives Ne&atives

!an trace tests to re#uire%entcoverage

Ti%e consu%ing to create and%aintain

Displays $"at "as been executed &ften out of date and %isleading

S"o$s relations"ip bet$een tests andfeatures

Duplicates infor%ation fro% testcases and re#uire%ents

Provides defect traceability tofeatures

Dedication to fre#uent updatesneeded

$o $on)t

Tie your test lists to re#uire%ents orstories

Trace to t"e test case or test idea

!onstrain your traceability to t"efeature level

Try to trace to t"e functional level

8uild traceability into t"e auto%atedtests

Duplicate infor%ation in %anydocu%ents

Map auto%ation to test lists or testspecs

&ver co%plicate %aintenance of%ultiple sources


13/21

Implementation Challenges


14/21

Challenge: ManagementIdentif the .hallen&es! )eliance on "istorical %etrics

4nderstand t"e current processes and practices

Decision-%a ing ti%elines

:eed traceability to feel confident of data

"espond to the .hallen&es!

Provide better %etrics

S"o$ "o$ ne$ process i%proves #uality

Prove "o$ decision can be %ade faster )evie$ traceability needs and support t"e%


15/21

Challenge: PMOIdentif the .hallen&es! )eliance on "istorical %etrics

Struggle $it" t"e details of testing

:eed predictability for costs and sc"edules

)e#uire i%proved ti%e to %ar et


Debun %isleading %etrics

S"o$ "o$ %ore testing provides better data

Prove reduced costs $it" efficiency gains Trac lost opportunity cost on test artifacts


16/21

Challenge: EngineeringIdentif the .hallen&es! Ta es ti%e a$ay fro% coding

Testing can be tedious

:ot t"eir speciali'ation

)ely on a serial approac" to testing


Test first design increases ne$ coding ti%e

Auto%ated / exploratory tests are less tedious

4nderstanding testing i%proves code $riting ess defects $ill be logged $it" up-front tests


17/21

Challenge: RegulationIdentif the .hallen&es! Docu%entation )e#uire%ents

(or%al or Standards Approval

(ull traceability

)igorous Auto%ation / detailed test results


ean to$ards %ore structured testing

4se exploratory but docu%ent results

Plan for and $or in s"orter iterations Auto%ate %ore docu%ent %ore


18/21

Challenge: OutsourcingIdentif the .hallen&es! )e#uire %ore instruction and detail

ac expertise in t"e product and %ar ets

Multi-cultural co%%unication

(or%al process / docu%ent to support $or


P"ase in (usion and train on exploratory

Provide %ore details in docu%entation

*%ple%ent %ultiple co%%unication pat"s Support and train ne$ processes


19/21

Ensuring Success

Implement in sta&es

10Measure t"e i%pact to custo%ers before / afterE20!ollect %etrics for a pro.ect for test execution ti%e before i%ple%entation30*ntroduce (usion to cover about 25F of testing60*ncrease (usion usage to 5GF to furt"er i%prove execution ti%e

50!ontinue to p"ase in until H5 to IGF of testing is (usion

Monitor

10*ncreases or decreases in execution ti%e20)evie$ docu%entation si'e closely revie$ing t"e test artifacts30Measure t"e a%ount of direct auto%ation60)evie$ $it" t"e tea% t"e a%ount of ti%e spent on %aintenance

50!ontinuously try to i%prove exploration ti%e and increase efficiency

Ad/ust based on the needs of the pro/ect

10&ne pro.ect %ay re#uire %ore precise docu%entation20Anot"er pro.ect %ay re#uire less structure30*dentify t"ese re#uire%ents at t"e start of pro.ects and ad.ust60)educe t"e a%ount of ti%e spent on artifact %anage%ent

50*nfor% people of t"e ti%e i%pacts of data collection


20/21

Five Keys t o Fusion10 $etail , t"e rig"t level of detail at t"e rig"t ti%e

20 #lannin& , consistently plan execute and ad.ust

30 Automate - spend ti%e on specifics in auto%ation

60 "eport , %easure $"at truly represents t"e tea% and custo%er

50 #OM , get %ore people involved to expand t"e breadt" of test


21/21

Conclusion

Rick Craig James Bach

Hung Nyugen Cem Kaner

QA&Test

My family, friends, colleagueswho have supported me!

FusionTesting-MaximizingTestExecution.ppt

Documents

Transcript of FusionTesting-MaximizingTestExecution.ppt