CA Agile Requirements Designer 101 - an introduction to the general premise of Model-Based Testing.

World®’16

Pre-ConLab:ARD101or“SoHowDoesAnyofThisWorkThen?”BenRiley– PrincipalConsultantPresales– CATechnologies

DO5X29L

DEVOPS

2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD

ForInformationalPurposesOnly

©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

TermsofthisPresentation


Abstract

Thislabwillprovideanin-depthoverviewofCAAgileRequirementsDesignerandthefundamentalconceptsthatunderpinitsapproachtosoftwaredesignandtesting.

CAAgileRequirementsDesignerisaflowchart-modellingtoolusedbyBusinessAnalysts,Programmers,TestersandAutomationSpecialiststodefineandautomaticallytestcomplexsystems.

BenRiley

CATechnologiesPrincipalConsultant,Presales


Agenda

2HOURS

SOMELOVELYINFORMATION

HANDSONWITHTHETOOL

LETSSEEWHATYOUGOT….

1

2

3

4


WhatDoWeWanttoAchieveFromThis?

1. Deepunderstandingofthetoolandwhatitcando

2. Afullerunderstandingofwhyitisimportanttoitsusers!

3. LargerawarenessoftypicalDesign-Dev-TestChallenges

4. Let’sPlay…


AFewValuePropositionSlides…

Nottoomany,don’tpanic.


QuicklyandsimplyintroduceagileprocessesintoyourexistingWaterfallLifecycles

AgileRequirementsDesigner


CommonChallengesPart1

§ 56%ofsoftwaredefectscanbetracedbacktoambiguitiesinrequirements1

§ Detectingandresolvingproductiondefectsisover40xmoreexpensivethaninrequirements2

§ Industrystandardformanualtestcasedesigncoverageisbetween10-20%

Stats:1and2- BenderRBT,2009.Others:basedonCATechnologies’audits


CommonChallengesPart2

§ Over-testingofsomefunctionsbyfactorsof40isnotuncommon

§ Changerequeststoexistingsoftwarerequirementscantakeuptoasmuchas60%oftheSDLC

§ IfImakeachangeIhavenoideatheeffectthiswillhaveonupstreamordownstreamsystems


ContinuousDevelopment

§ ContinuousIntegration

§ IterativeDevelopment

§ DevOps

§ IsmostlyOps

§ Itstartswhendevelopmentandtestingarecomplete

§ Itshouldstartwhentheuserhasanidea!


CAAgileRequirementsDesigner

§ Thereareplentyoftoolstomanagerequirements§ Thereareplentyoftoolstomanagetestcases§ Thesecanbeconsideredlogisticalsupport

CAAgileRequirementsDesignerisanacceleratortoimprovetheinputtothesetools§ IsaTestCaseDesignandTestCaseOptimizationtool§ IsaRequirementsDefinitiontool§ Cancalculatecomplexityveryaccurately§ CanidentifychangeacrosstheentireSDLC

CAAgileRequirementsDesignergivesyoumultipleoutputsforoneinput


WhatisCoverage?

TheBusinessthinksit’s:– CodeCovered– NumberofTestCovered– TestsRun– Percentageofusecases– AllPairedCombinations

Itisactually:– DesigningSufficientTestsToVERIFYThatTheDesignandCode

CorrectlyImplementTheRequirements– Didyougettherightanswerfortherightreason?Twoormoredefects

maysometimescanceleachotherout– Observability


ASimpleForm EasytoTest?


TheUserKnowswhattheywantTheAnalystspecifieswhatthatis

TheProgrammerwritesthecode

TheTesterteststheprogram

Thefurtherthevisionsdiverge Thegreatertheproblems

ClarityandVisionDuringDevelopmentBusinessAnalyst Programmer TesterUser


Therearelessbugsandtheproductisdeliveredfaster

Thecloserthevisionmeanstheusergetsaqualityproduct




Verify&Clarify ClearLogic TestCases

Thescopeforambiguityismuchless

Theproductisofhigherqualityandisdeliveredfaster

CAAgileRequirementsDesigner:§ DesignTests- Minimum- Optimal

§ DesignData- Findit- Makeit


TheResultsareOftenPoor


“Everythingshouldbemadeassimpleaspossible,butnotsimpler.”

–AlbertEinstein

Wedo knowthatwhenoureyesareopen,ourvisionaccountsfortwo-thirdsoftheelectricalactivityofthebrain—afull2billionofthe3billionfiringspersecond—whichwasthefindingofneuroanatomistR.S.Fixot inapaperpublishedin1957.


TesterProjectManager/DevelopmentManager

BusinessAnalyst/RequirementsManager TestManager

TheRequirementsareconceivedandcreatedinCAAgileRequirementsDesigner.

Relevantprojectmanagersfilloutdetail.ThiscanbepassedbackupthechaintoBas,etc.

TestManagersensureimplementationissuesaretackled.Alsocanbepassedupthechainforverification.

Testersreceivecomprehensive,logicaltestplansandtestcases.

Requirements-BasedTestingModelofTestPlanDelivery– FullTraceability


WhoWouldUseCAAgileRequirementsDesigner?

§ Icanseeanyprocessflowandpredicthowlongthedevelopmentandtestingprocesswilltake

§ MyUserscanclearlydefineaprocessastheythinkofit§ MyBusinessAnalystcandefinetheprocess,whichwillbeverifiedby

theuser,createlogicflowsforprogrammersandtestcasesfortesters§ MyProgrammerscanverifytheprocesswiththeBusinessAnalystbefore

codingandTesterswithtestcases§ MyTesterscanquicklydefinefunctionallogicflowsandverifythemwith

theProgrammer,aswelldefinethesmallestsetoftestcasestoverifytheprogram

§ MyOutsourcingPartnerscangetstablerequirementsandclearlydefinedworkpackages,whichresultinlowerongoingcosts


§ Generatetestscriptsthatprovidemaximumfunctionalitydirectlyfromrequirements

§ Quicklyandeasilyrespondtochange,andavoiddelaysandbudgetover-runs

§ Fullytestsoftwareinthesmallestnumberofrunsrequired,deliveringqualitysoftwareontimeandunderbudget

AgileRequirementsDesigner


JustaQuickTouroftheGUI…

Afewthingshavechanged…


BacktoBasics

SettingupSimpleFlows


ProcessBlocks

§ Processblocksrepresent‘doingsomething’i.e.notadecision,butsomethinghappening– forexample‘MakeTea’or‘RejectLoan’–theydon’thavetoberesultsthough,anotherexampleprocesscouldbe‘LookupCandidate’

§ Processblocksmayonlyhaveoneoutput,astheyarenotmakingadecision

§ Processblockscanhaveunlimitedinputsfromotherblocks(twodepictedhere)

§ TestData,Automation,andmanyotherusefulpiecesofsupportingdatacanbeattachedtoaprocessblock,aswe’llseelater.


DecisionBlocks

§ Decisionblocksrepresentachoiceordecisionmadeintheflowchart,forexample‘Havewater?’or‘Amountofloan?’

§ Baseduponthesedecisions,decisionoutputmayendupindifferentpartsofthediagram,buttheydoesn’thaveto

§ Adecisioncanhaveasmanyoutputsasrequired,andcanhaveasmanyinputsasrequired

§ Alldecisionblockoptionsinthetoolbarcanbeeditedafterthefacttoresembletheothers,theyarejustdefaults

§ TestData,Automation,andmanyotherusefulpiecesofsupportingdatacanbeattachedtoadecisionblock,unlikeprocessblocksmuchofthisisdoneperoutputaswe’llseelater.


AddingBlocks

§ Tocreatenewblocks,simplydragthemontothecanvas(righthandside)fromthetoolbox.

§ Youcanalsoaddnewblocksbyclickingonasymbolontheleft,andclickingsomewhereonthediagramforittoappear.

§ Formostsetups,thethreeblocksyouwillbeusingthemostarelistedhereontheright:(Start/End,Process,True/False).

§ Anydecisionblockcanbeeditedtohaveasmanyoutputsasnecessary(thinkYellow,Red,BlueratherthanTrue/False)sodon’tworryaboutthatfornow.


EditingBlocks

§ Toeditanexistingblock,double-clickit(orright-clickà Edit)

§ Thenumberoftabscanbeintimidating.Mostofthemareforadvanceuseandyou’llneverhavetoknowwhattheydo.

§ Differentblocktypeshavedifferentsetsoftabs(process,decision,subflow,etc.)


Exercise1.GototheShop?

§ Aneasyonetostart,openMakeTeafromtheexamplesfolder§ RefamiliarizeyourselfwithmanipulatingtheUI,doubleclickon

blockstoopenthemup,dragtomoveblocksandarrows,dragnewitemsfromthepallettocreatenewblocks

§ Whenready,makethefollowingchange:“WhenIwantmilk,butdonothaveit,Imaygototheshop,theshopmayormaynothavemilk– forwhateverreason,ifIdon’tcomebackwithmilk,Iwillstillnotmaketea,ifIdogetmilkIwillmaketea.”tothediagram,adjustingthediagramtotakeyourchangesintoaccount


Exercise1.GototheShop?– TickTock theclock’srunning…10minutes…

10 5 0


Exercise1.GototheShop?Getsomethinglikethis?


Exercise2.SuperCoffee andSuperHealth

We’regoingtocreateanewflow,baseduponafreedrinksschemeforcustomersofSuperCoffee ifthey’vedoneenoughrunningatSuperHealthHere’showtheschemeworks:1. IfIhaverunover10milesintheweek,Iearnafreedrink2. IfIhaverun>0milesintheweek,andIamaGoldmemberofSuperCoffee,Iearn

afreedrink3. IfIhaverun5-10milesintheweek,andIamaHealthPlus memberof

SuperHealth Iearnafreedrink,unlessitisaweekend.Ofcoursebyrunning5-10milesIalsoqualifyforNo.2potentially.

4. Ihaverun0miles,Igetnothing.5. Youcanonlyqualifyforonefreedrink,ifyouearnoneforanyreason,youcan

neverinvalidateit.Howmanypossiblepathsdoyouget?


Exercise2.SuperCoffee andSuperHealth – TickTock…20minutes…

20 15 10

10 5 0


Exercise2.SuperCoffee andSuperHealthGetanythinglikethis?

Ifnot,whynot?Comparewithyourneighbours!


Generation,Storing,Options,Coverage

§ Pathsaregeneratedtospecificcoveragetargets

§ Theoptimiserwillalwayschoosetheminimumnumberofpathstocoverwhatithasbeenaskedtocover

§ Withtheexception,of‘Allpossiblepaths’

§ Therearemanyoptionsforthepathoptimiser,fromfilters,toconstraints.Typicallythedefaultsarefine.

§ Letshavealookthrough…


Coverage

§ OneofthemainfeaturesofCAAgileRequirementsDesigneristheabilitytomeasurecoverageatalltimes,foranyparticularsetofpathsforanyparticularcoverageoption.

§ Typicallycoverageachievedbyhumansispoor.

§ We’regoingtoletthecomputerplaychessforus.

§ Letshavealookatthecoveragewindows…


StoringPaths,etc.

§ Gobacktomaketea,oroneoftheflowsyouhavecreated.

§ Generatetargetingthe‘allnodes’coverageoption,storethepaths,andopenthecoveragewindow,whatisthe%?Openthe‘more’sectiontoseewhat’sactuallybeingcoveredandwhatisnot.Now,finishyourcoverageforAllin/out,usingthe‘addtoexistingpaths’option.

§ Now,we’llhavealookatmanagingstoredpathtypes…


ChangeImpact,theHeartoftheTool

§ TheprincipleofCAAgileRequirementsDesigneris,“Letsmaintaina40-blockdiagram,ratherthanmaintain1000softests/datapoints/scripts”

§ WhenyoumakeachangetothediagraminCAAgileRequirementsDesigner,theImpactanalyserwillallowyoutomaintainthetestcases,andthusthedataandscripts- itwillcalculatewhatpathsnolongermakesense,andgiveyouoptionsastowhatyou’dliketodowiththem,including,repairthem.Foranyonewhohasexperiencedmanuallymaintainingtests,youmightrecognisehavingthetooldothisisamuchbetteridea!


Exercise4.Let’sMakeaChange

1. Openuptheflow‘SongStreaming’2. Cleardownthetestcasesinthestoredpathtype‘testcases’3. Storetheoptimal‘allpairs’pathsforthisflow.Makesurethe“Trytofindpathsin

previousversion”isUnchecked4. Now,implementthechange“WhenIamnotconnectedto3g,orthe3gisnot

fastenough,Ishouldcheck(beforefailingasbefore)ifIamconnectedto2g,IfIam,Ishouldwaitforthebuffer– else,errorasbefore”

5. Gobacktoviewyourstoredpaths,examinewhatthepathimpactanalysersays,deletetheeffectedpaths

6. Nowgoandlookatthecoverage,you’llnoticeit’sdropped!7. GeneratebothA)thenewoptimalsetofpathsandB)justtheadditionalpaths

requiredtofinishthecoverage.


ARealWorldScenario?

§ Therealworldisn’tsoeasy.

§ Goalistohaveafeasiblemodel,notaperfectone.

§ Presentationmatters.

§ Logicallyimplementwhatitcouldmean,somedefensibleinterpretation.


TestDatainCAAgileRequirementsDesigner

§ Howdoesitwork?

§ Ingeneral,theideaistosetdatabasedonwherewegointhediagram,ratherthanworkingitalloutafterwards.

§ Thereasoningisthis:It’seasiertomake40dataassignmentson20blocks,ratherthanmaintainmillionsofdatapointsmanually!

§ Theotheradvantage,asalludedtobefore,isofcoursethatnow,whenwechangeourdiagram,ourdatacanautomaticallyreacttoo!

§ Letmeshowyou…!


Exercise5.Let’sSetUpSomeData

1. Openup‘Maketea’,ateachdecisionpoint,addthenecessarydataassignmentsforeachoutput!

2. Now,generateandstoresomepaths.3. Gotothetestdatatableandexamineallofthedatathatresults.4. Nowthisisallboring,hardcoded(toanextent)data,butwe’llfix

thatlater.5. Playaround,addmorevariablestosomeofthedecisionoutputs,

changetheirvalues,andhavealookattheeffectinthetable.6. Now,changethediagraminsomeway,andreactaswedidbefore,

nowconsideryournewdata,neat,right?


Exercise6.Let’sSetUpaLoan

1. Soletsmakesomething(mildly)morerealistic.2. Setupaflowwherewearegoingtobegeneratingmanydifferent

typesofloan.3. Weneedadecisiontodecidetheproductcode:CT50,CT33,CT48,CT49,

wemustalsodecidetheamount(fornow,wewillhavetheoptions<1,000,1,000-10,000,10,000-100,000,>100,000)

4. Wewillalsoneedadecisiontosetthetermoftheloan,(1yr,3yr,5yr)5. Wewillneedadecisiontosettheinterestrate(<1%,1-5%,>5%)6. Allofthedecisionsuptonowjustflowintothenext,butfromhere,we

needatruefalsedecision‘backdate’,ifitistrue,wemusthaveadecisiontochooseabackdateTime (1days,2-359days,360days)elsedonot.Nowbothpathsgointoa‘calculateresult’block.


Exercise6.Let’sSetUpaLoan

§ Again,generatesomepaths,andhavealookatyourlovelydata!

§ Butthereareissues…howdowedealwithranges?Howdowemakedatareferentiallyintact?Howdowemakeitmoredynamicandrealistic?

§ …Dataexpressions!


TheDataPainterwindow

§ TheDataPainterwindow


Exercise7.Let’sGenerate:

1. Arandomnumberbetween1and20002. ARandomnumberofdigitsbetween3and7inlength,

paddedtotheleftwith0stoalengthof73. Multiplytworandomnumberstogether4. Addarandomnumberofdaystothe~CDATE~01/01/2009if

cdate misbehaves5. Randomlychooseafirstnamefromaseedlist6. A@percval@ofanytwostrings,or,ifyouwanttobefancy,

twofunctions!7. [email protected] (useseedlists)


Let’sGenerate

§ Butwhataboutthiscolumn?


Exercise8.ReferentialandRandom?

1. Usingtheknowledgeofthefunctionswe’vejustbeenusing,goandfixoutloanflow,sothateverythingisparameterised (i.e.therangesallhavearandomexpressionforthembetweenthatrangeetc.)

2. Storepaths,resolvethetestdata3. Now,gobacktoourblock‘calculateresult’4. UsingtheformulaPayment =(Interest rate*amount )/(12*100)+(amount /(

term*12))setupthevariableandexpressionfor‘MonthlyPayment’remembertoreferencethecolumnsforamount,term,interestrateetc.

5. Makesurethisresolvesproperly!6. Examineyourdatainthetable7. Wewon’t,butifwemadeachangetothiskindofflow,thechangeswouldreflect

uptoourdatatoo.


Exercise8.ReferentialandRandom?

1. Here’soneImadeearlier…

2. Testerssometimesspenduptohalftheirtimesearchingorcreatingappropriatedatafortheirtests!Nowwecanagainmaintainafewblocks,ratherthan10000sofdatapoints,andrefreshourdataondemand!Neateh?

3. Thiswillbeapartofourkillerdemo(ofcourse,notinasmuchdetail).


Filters

§ Forthemostpart,filtersareusedtoeasilyselectorseepartsofthediagram

§ Intheverylatestversion,wecanalsoapplyfilterspriortoandpostoptimisation

§ Letsconstructsomefilters!


Exercise9.Filters

1. Constructafiltertoonlyseetheblockswhichhavetestdatadefinedforthem

2. Changetheimportanceofthe‘don’tmaketea’blocktocritical,thencombinethefilterabove(settono)withafilterwhichselectsonlycriticallyimportantblocks

3. Storeafilterforlateruse.


Exercise9.Filters

1. Constructafiltertoonlyseetheblockswhichhavetestdatadefinedforthem

2. Changetheimportanceofthe‘don’tmaketea’blocktocritical,thencombinethefilterabove(settono)withafilterwhichselectsonlycriticallyimportantblocks

3. Storeafilterforlateruse.4. Whywouldwebotherwiththese?


ImportandDeduplication

CAAgileRequirementsDesignercanimporttestcasesfrommanydifferentsources!Including…

§ HPEALM

§ JIRA

§ CAAgileCentral

§ VERSIONONE

§ EXCEL


ImportandDeduplication

§ Sowecanimporttestcases?What’sthepoint?

§ Webringineachtestcaseonastepbystepbasis,typicallythiswillmeanthereduplicatestepssharedacrossmanytests.

§ InCAAgileRequirementsDesigner,youcanhavethetoolsuggestduplicateblockstoyou,suggestionsyoucantheneitheracceptorrejectonacasebycasebasis.

§ Onceyou’vemarkedtheduplicates,thetoolcandeduplicate theblocksforyou,butkeepingeverypathintact– thisleavesyouwithaflowthatithasinferredfromthetestcasesyougiveit!

§ Thisisanimportantaccelerator,butnotperfect!


Exercise10.BringinSomeTestsandDedupe!

§ Inanewflow,gototheimporttab,click‘excel/generaltestcases’– selectthe.docx ‘Doc_procedure_testcases’

§ MaptherelevantfieldstofieldsinCAAgileRequirementsDesigner

§ HitImport!

§ Openthededupewindowunderneathtestfactory

§ Haveaplaywiththesensitivitysettings

§ Markalltheduplicates,thenhitde-dupeinthemainAgD window.

§ Re-layouttheresultingdiagram,makeanyfixes/additions– measurethecoverageofthestoredpaths,canwedoitwithlesstestcases?


Constraints

§ Constraintsallowyoutolayer‘rules’ontopofthediagraminordertoachievespecificfunctionalitywithoutanunreasonablylargeflow

§ Imagineadecisionnearthetopofalargediagram,whichcanbetrueorfalse,butmuchlaterinthediagramwemakethesamedecision– iftheyhavetolineuphowwouldwedothiswithoutconstraints?Wouldwedoublethesizeofthediagram?

§ Cananyonethinkofwhythesemaybenecessary§ Letshavealookatthemforreal…§ Ingeneral,youshouldonlyuseconstraintswhenthesamelogiccannot

(easily)beachievedbyflowlogicalone,andthereisusuallyawaytodothat.


Exercise11.BuildaUsefulConstraint

§ Openour‘loan’flow,createaconstraintsuchthatwhenevertheproductisCT50,thebackdatemustbe360

§ Ifyougeneratepathsnow,youwillnoticetherewillbeless

§ Youcanoptionallyhavethepathexplorerignoreconstraints

§ Feelfreetobuildupmorecomplicatedconstraintsinthe‘pathhints’dialogue.


Extras

§ Versioncomparison

§ Richtextimporter

§ Worddocbuilder

§ Clonescopiesandduplicates

§ Customexcelexport

§ Propertieswindow

§ Flowvalidation

§ Diagramexports

§ Accelerators

§ Shapes

§ Loops


Questions?


RecommendedSessions

SESSION# TITLE DATE/TIME

DO5X23EPre-ConEducation:RequirementsareKing– BetterRequirements=BetterSoftware

11/15/2016at11:00am

DO5X27S What'sNewinCAAgileRequirementsDesigner? 11/17/2016at12:45pm

DO5X40S CaseStudy:WhyRabobankWasInspired 11/17/2016at3:45pm


MustSeeDemos

DataComplianceandTesting

CATDMTheater5

DeliverTestDataFaster

CATDMTheater5

UseModelBasedTesting

CAARDTheater5

AchieveMaximumTestCoverage

CAARDTheater5


Stayconnectedatcommunities.ca.com

Thankyou.

CA Agile Requirements Designer 101 - an introduction to the general premise of Model-Based Testing.

Technology

Transcript of CA Agile Requirements Designer 101 - an introduction to the general premise of Model-Based Testing.