Agility is a Stable Requirement · 2018-11-13 · Bearing in mind that IT/IS is only a fraction of...
32
1 of 32 Agility is a Stable Requirement Graham McLeod inspired.org December 2017 Abstract Change in technology, business and society is ever present and accelerating. It is very unlikely to slow down, thus it is a stable requirement. Our methods of doing strategy, devising future architectures and delivering systems capabilities in support of business processes, capabilities and delivery of services and products therefor need to address this. A great deal of effort has been applied in Agile Methods over the past two decades to accelerate the system development process, i.e. doing things faster. No matter how quickly they deliver, however, these methods often produce something inDlexible. This paper argues for a broader approach, which looks at: the context (much of the change required is outside the system delivery space; the focus (what should we be changing and why?); and three approaches to achieving the change with respect to system deliver: doing less things (de-scope, use packages, libraries, components, frameworks); do things faster (agile methods, automation, generation) and make more Dlexible things (runtime adaptable or domain model driven systems). The last of these is an unconventional approach that holds promise, even if you currently don’t practice, or succeed with, agile methods. Finally, we also address the dilemma of accelerated delivery while dealing with large legacy application landscapes.
Transcript of Agility is a Stable Requirement · 2018-11-13 · Bearing in mind that IT/IS is only a fraction of...
1of32
Agility is a Stable Requirement
Graham McLeod inspired.orgDecember 2017
Abstract
Changeintechnology,businessandsocietyiseverpresentandaccelerating.Itisveryunlikelytoslowdown,thusitisastablerequirement.Ourmethodsofdoingstrategy,devisingfuturearchitecturesanddeliveringsystemscapabilitiesinsupportofbusinessprocesses,capabilitiesanddeliveryofservicesandproductsthereforneedtoaddressthis.AgreatdealofefforthasbeenappliedinAgileMethodsoverthepasttwodecadestoacceleratethesystemdevelopmentprocess,i.e.doingthingsfaster.Nomatterhowquicklytheydeliver,however,thesemethodsoftenproducesomethinginDlexible.Thispaperarguesforabroaderapproach,whichlooksat:thecontext(muchofthechangerequiredisoutsidethesystemdeliveryspace;thefocus(whatshouldwebechangingandwhy?);andthreeapproachestoachievingthechangewithrespecttosystemdeliver:doinglessthings(de-scope,usepackages,libraries,components,frameworks);dothingsfaster(agilemethods,automation,generation)andmakemoreDlexiblethings(runtimeadaptableordomainmodeldrivensystems).Thelastoftheseisanunconventionalapproachthatholdspromise,evenifyoucurrentlydon’tpractice,orsucceedwith,agilemethods.Finally,wealsoaddressthedilemmaofaccelerateddeliverywhiledealingwithlargelegacyapplicationlandscapes.
2of32
Intentionallyblank
3of32
TableofContents
Abstract 1AgilityisaBusinessRequirement 5DeDiningAgility 5Figure1-DimensionsofChangetoAchieveBusinessAgility 5
AgileMethods(Doingthingsfaster) 6Figure2-AgileManifesto 7Figure3-TypicalAgileLifecycleSource:ScrumAlliance 8
Dangersof“Agile” 9AchievingAgilityinSoftwareSolutions 10Figure4-Typical“Waterfall”Lifecycle 10Figure5-ContrastingWaterfallandAgileLifecyclesPMO=ProjectManagementOfDice 11
DeliveringMoreRapidly 11Figure6-TraditionalWaterfallLifecycle 11Figure7-IterativeIncrementalLifecycle 12Figure8-AgileLifecycle 12Figure9-DevOpsLifecycle 13
LeveragingPriorEffort(Doinglessthings) 13Figure10-PartofanoldstonebuiltfoundryinMoratacPark,NorthCarolina 14UseComponentLibrariesandFrameworks 14Figure11-RapidConstructionwithPre-BuiltComponentsSource:https://i.pinimg.com 15Figure12-PrefabricatedStructuredeliveredtosite 16
UseReferenceModelsandPatterns 16Figure13-Ahospitalarchitectureblueprint,orpatternSource:www.hhbc.in 17Figure14-FrameworxfromtheTelemanagementForumprovidesgenericmodelsforTelcos 17
GenerateDetail 18Figure15-3DPrintedHouseSource:https://all3dp.com 18
DomainSpeciDicModelingandGeneration 19Figure16-DomainSpeciDicModelingActivitiesSource:MetaCase 19Figure17-DSMlanguageandgeneratordevelopmentdoesincuranoverhead,butitcanbesmall.Source:MetaCase 20Figure18-DSMdevelopmentmayberecouped,eveninaninitialprojectSource:MetaCase 20Figure19-DSMProjectsvsConventionalDevelopmentSource:MetaCase 21
ReuseandEnsureAdaptability 21DoLessThingsOurselves 22MakemoreFlexibleThings 22Figure20-CapeTownInternationalConventionCentre-AconDigurablefacilitySource:CTICC 23
RuntimeAdaptableSystems 24Figure21-RelativeProductivityofDifferentLanguagesSource:SoftwareProductivityResearch 25Figure22-EVANetmodelerArchitectureSource:inspired.org 26Figure23-RelativeFlexibilityandProductivityofTechnologies 27
AgilityandLegacy 27
......................................................................................................................................................................................................................................................................
............................................................................................................................................................................................................................
.........................................................................................................................................................................................................................................................
......................................................................................
....................................................................................................................................................................................................................................
...........................................................................................................................................
...............................................................................................................................................................................................................................................................................................................................................................
...................................................................................................................................................................................................................................................................................................
...........................................................................................................................
................................................................................................................................
............................................................................................
.........................................................................................................................................
.....................................................................................................................................................................
............................................................................................................................................................................................................................
...................................................................................................................................................
........................................................................................................................................
...............................................
......................................................................................................................................................................................................................................................................................................................................................
.....
..................................................................................................................
...........................................................................................................................................
.....................................................................................................................................
4of32
Figure24-SoftwarevisualizationwithMoosetools 28
Harvest,ReModel,ForwardGenerate 28Figure25-CodeHarvestingandForwardGenerationwithDSM 29
TheLists 30Figure26-Factorsaffectingagility 30
PromotingOrganizationalAgility 30ReferencesandFurtherReading 32Papers 32Presentations 32Videos 32Websites 32
.............................................................................................
...................................................................................................................................................................
........................................................................................................................................................................................................................................................................................
................................................................................................................................................................................................................
............................................................................................................................................................................................................................................................................................................
......................................................................................................................................................................................................................................................................................................................
5of32
AgilityisaBusinessRequirement
Veryfewwoulddisputetheneedforagilityintoday’sbusinessworld,governmentorganizationsandNGOs.Agilityisneededtorespondtoincreasedcompetition,globalisation,governance/regulationandchangesintechnology,customerexpectationsandbusinessmodels.NoonewantstobeaKodakinadigitalphotographyworld,oranEncylopaediaBritannicawhenEncartaorWikipediaobliterateyourbusinessmodel.Giventhattechnology,businessandsocietalchangeiseveraccelerating,itishighlyunlikelythattheneedforagilitywilleverdecrease.ThisleadstothesurprisingconclusionthatAgilityisaStableRequirement.Ifitisastablerequirement,weshouldbeabletodevisewaysofachievingit.Therestofthispaperexaminestheimplicationsofthisrealisationandexploreswaystoachieveagility.
DefiningAgility
Forthepurposesofourdiscussion,wedeDineagilityasTheabilitytosuccessfullyadapttonewcircumstancesoropportunities
inarapidandsustainablemanner
Forabusinessthisincludestheabilitytoadaptoradjust:• BusinessModelandStrategy• OrganizationStructure,CultureandStafDing• BusinessProcesses• ProductsandServices• ChannelsforDelivery• ArrangementswithBusinessPartners• SkillsandTechniques• SupportingSystems,InformationandTechnologyWecanrepresentthedifferentfocusareasinthefollowinggraphic:
Figure1-DimensionsofChangetoAchieveBusinessAgility
6of32
NoticethatIT/ISisonlyoneofmanyaspectsthatrequireagility.Achievingagilityintheotherareasismostlyaconcernofdisciplinesincluding:• Strategy/BusinessArchitecture• ChangeManagement/ProgrammeManagement• OrganizationDesign• ProcessEngineering• ProductandServiceDevelopment• Contracting• Training• EnterpriseArchitecture• ProjectManagementCultureisalsoaveryimportantelementinachievingagiity.AnempoweredworkforceofsmallteamswithinterdisciplinaryskillsworkinginalearningorganizationculturewilldeDinitelyoutperformanautocraticformalone.FortheIT/ISelement,agilityincludesadapting:• Technologiesinuse• Infrastructuresupportingthebusiness• Applicationsystemscapabilities• Informationcapture,storage,processing,sharing,analysisandvisualisation
capabilitiesBearinginmindthatIT/ISisonlyafractionofthefullpicture,wewillzoominonapproachestoachievingagilityinthisdimension.Weplantoaddresstheotherdimensionsinfuturepapers.
AgileMethods(Doingthingsfaster)
ManyorganizationshaveadoptedAgileMethods(e.g.Scrum,DisciplinedAgile,ScaledAgile,ExtremeProgramming,Crystaletcetera)asameansofachievingagility.Theearliermethodsandversionsofmethodsprimarilyfocussedontherapid,incrementaldeliveryofapplicationsystems.Whilerelativelysuccessfulinthisgoal,wecanimmediatelyappreciatefromtheforegoing,thatthisisasmallpartoftheoverallscopementionedabove.TheAgileManifesto,draftedbyagroupofsoftwaredevelopmentluminariesin2001,setoutsomeimportantprinciples.
7of32
Figure2-AgileManifesto
Notethatitemphasisesthepriorityofitemsontheleftoverthoseontheright.Itdoesnot,asmanyadherentsproclaim,eliminatetheitemsontheright.Notetoo,thatitaddressesSoftwareDevelopment.Inourview,amethodshouldtargetSolutionDevelopmentwhichisbroaderthansoftwareonly,inthatitwilladdressotherelementsrequiredtosolvethebusinessproblem,including:• DocumentationsufDicientfortheinstallation,operation,supportandenhancement
ofthedeliveredsystembypartiesotherthanthedevelopers• Proceduresaroundthesystemtoensureitssuccessfuloperationinitsintended
context• Conversionofdatatothesystem• Integrationofthesolutiontothecontextinwhichitmustoperate• IdentiDicationofreusablecomponentsandservicesthatmaybeleveragedinother
efforts• Qualityassurancethroughtestingandothermeans(e.g.inspections)toensurethe
correctness,robustnessandcomplianceofthesolutionAgileMethodstarget,interalia,thefollowinggoals:• Rapiddeliveryofsystemscapabilitytothebusiness• Highlevelsofsponsoranduserengagementindecisionmakingduringthedesign,
buildandtestingactivities• ClariDicationofunclearorunknownrequirementsthroughmock-ups,prototyping,
facilitationanditeration• Gainingexperienceofnewapproachesortechnologiesearlytofacilitatelearning
andeliminaterelianceonuntestedassumptions• Visibilityofprogress
8of32
• CollaborativeteamworkTheyuseavarietyoftechniquesandpracticestofacilitateorachievethesegoals,including:• FeatureRequirementsListsorIssueBacklogs-tocaptureandagreerequirements
andallowgroupingofthesetoscopeaniteration,agreeingprioritieswithsponsorandusers
• Sprints-intenseburstsofactivitywithveryfocussedgoals,includingdeliveryofworkingsoftware
• Standupmeetings-toshareprogress,encouragecollaboration,reducemeetingtimesandkeepadministrationoverheadlow
• Visiblereporting(e.g.Burndowncharts)tomakeprogressandstatusvisibleandsharedandtofocusefforttowardsgoals
• Pairprogramming-toapplymorethanonemindtotheproblemandgeneratebackupskills/providecoaching
• Continuousplanning,integrationand(automated)testing• TestDrivenDevelopment(TDD)wheretestsaredevelopedDirst,andsoftware
subsequentlytopassthetests• Leanconceptswhichincludetheideasofdoingjustenoughtobeusefulandgauging
successbeforecommittingmoreresources,orpivotingtoamoreusefuldirection.Theseandkanbanconceptsalsofavourlimitingworkinprogresstoensurethatfocusishighandproductivityisimprovedbylimitingmulti-tasking
• Eliminationofany“mightbeneeded”requirements-thetermYAGNI(Youaren’tgonnaneedit)hasbeencoinedtosummarisethis,inthebeliefthatusers,analysts,designersandprogrammersarepooratpredictingthefuture,soweshouldnotwastetimeonanyrequirementsthatarenotabsolutelyknowntobeessential
Figure3-TypicalAgileLifecycleSource:ScrumAlliance
9of32
Agilerequirescertainfactorstopermititssuccess,including:• Highleveloftrustbetweenthesponsoranddevelopmentorganization.Afterall,
thereisnocontractedvalue/scopeforthemoneythatwillbespent,asintraditionalcontracting/waterfall.Rather,thesponsorcommitstofundingthedevelopersatacertainrateinthefaiththattheyarecompetent,willbeproductiveandwillfocusontherightissuestodelivervaluetothebusiness.Thiskindoftrusttakestimeandgoodhistorytoestablish
• Agilemethodsarenosubstituteforgoodskills.Infact,successfuluseofAgilewilloftenrequirehigherskills,sinceteammembersmustexercisemuchmoreindividualjudgementanddiscretion.Agileshouldthusnotbeattemptedwithateamofjuniorswithlittleexperience.Theycancertainlybeincludedinateamwithmoreexperience,buthavingexperienceonhandisapre-requisiteforsuccess
• Agilereliesalotonpeople.Continuityofstaffandfulltimecommitmenttotheprojectarenecessarytoensurethatthetacitknowledgeinplayisnotlostordiluted
Dangersof“Agile”
Agileissometimesusedasanexcuseforvariousills.CommonissuesarisingfromtheabuseormisunderstandingofAgileinclude:• Lackofarchitecture-assumingthatwecan“builditaswego”,resultinginpoorly
architectedsystemsthatdonotadheretogoodprinciplesandwillbedifDiculttomaintaininfutureorwhichdonotDitintothecontextwheretheymustoperate.Infact,betteragilemethodsdocaterforarchitecturalrequirements,e.g.bydoinganarchitecture“spike”beforecommencingdetaildevelopment
• Lackofdocumentation-AlthoughAgilefavourspeopleandinteractionoveradministrationanddocumentation,itdoesnotsayweshouldnotdocument.Rememberthatweneedtoproduceaninstallable,operable,maintainablesolution
• Rehashingrequirementsthatwealreadyknow.Yes,AgileisagreatwaytoreDineunclearrequirements,ordiscoverunknownones.Thatdoesnotmeanthatallourrequirementsareunknownoruniquetothisproject.Wecancertainlysaveahugeamountoftimeandeffortbysimplydocumentingourknownrequirementsorusingestablishedframeworks,referencemodelsandpatterns
• Under-deliveringonscope.Agileprojectswilloftencutscopetomeetadeadline.Thatcanbeagoodtechnique,providedthatwedonotdoittothedetrimentofthebusinessrequirementsandpriorities.WehaveseenAgileprojectsthatreducedthescopebeyonda“minimumviableproduct(MVP)”todeliversomethingnotusefultothebusiness
• Focusonfunctionalityattheexpenseofotherrequirements,suchasintegrationtoothersystems,performance,robustnessorscalability.Wemustpayadequateattentiontonon-functionalrequirementsaswell.Failuretodosocanultimatelydoomtheprojectandsolution
NewerAgileMethods(suchasScaledAgile(SaFe)DisciplinedAgile(DaD))areaddressingissuesbeyondjustsoftware,andincreasingly,issuessuchasarchitectureandDittoorganizationalobjectives.Theseareencouragingdevelopments.InthenextsectionwewilllookatwaystoachieveagilityandwhereAgileMethodsDitinthis.
10of32
AchievingAgilityinSoCwareSoluEons
Theprimaryapproachfollowedtoimproveagilityinsoftwareistotrytoreducethetimebetweenidentifyinganeedanddeliveringasolution.Thetraditional“waterfall”lifecyclehadaseriesofsteps,moreorlessinsequencewitha“Dlow”fromearliertolaterstages.Discoveryofnewrequirementsordisprovingassumptionslaterinthelifecycle,oftenrequired“cyclingback”toanearlierstageandameasureofreworktocontinue.Theoveralllifecyclecouldtakeawhiletocomplete,oftentothefrustrationofbusinessunderpressureforrapidcalendartimedelivery.
Figure4-Typical“Waterfall”Lifecycle
Iterativeandincrementalformswereevolvedtoimprovethedelivery.Intheseapproaches,asubsetofrequirementswouldbeselected,developedanddelivered,beforecyclingbacktotacklemorerequirements.Thishadtheadvantageofearlierdeliveryofpriorityitemsandbeingabletoapplylearningfromearlieriterationstolaterones.Theseapproachesworkedwell,providedthattherequirementswerefairlywellunderstood.However,manyclassesofproblemsarose(especiallyasInformationSystemsandTechnologyexpandedintonewtypesofapplicationsneverbeforecontemplated)wheretherequirementswerenotunderstoodattheoutset.Inthesecasesamorefacilitative/prototyping/experimentalstyleemergedthatledto“Agile”methods,whichassumethatrequirementsarenotunderstoodandwillonlybecomeclearduringtheproject.Agileemphasiseshighsponsor/userinvolvement,teamwork,communication,deliveryandfeedbackinrapiditerations.Scopemaybeadjustedineachiterationinconsultationwiththesponsorandlearningfromearlieriterationsisfedintosubsequentones.Eachiterationhaselementsofrequirements,design,development,testingandmaydeployworkingsoftware.Wecontrastthewaterfallandagilelifecyclesbelow:
11of32
Figure5-ContrastingWaterfallandAgileLifecyclesPMO=ProjectManagementOfQice
DeliveringMoreRapidly
AlotofmethodemphasisovertheyearshasbeenonreducingthetimetodeliveryfromidentiDicationofabusinessrequirement.Wecanseethisprogressionasfollows:
Figure6-TraditionalWaterfallLifecycle
Inthetraditionalwaterfallcycle,deploymentoccursquitelateinthecycleandalongtimeafterrequirementsareDirstdiscussed.Thiscanbeaproblemforbusinessiftherequirementisurgent,e.g.complyingwithasuddenlegislativechange.Otherproblemsthatcanarisearethattherequirementchangesorbecomesobsoletebeforethedeliveryofthesoftwarecapability.Thiscanleadtoagreatdealofwastedeffortandinvestment.Iftherequirementsarepoorlyunderstood,orunclear,theriskofdeliveringsomethingthatisnotoptimalisveryhigh.Fixingitcanalsoincuralongdelayandhighcostaswewaitforanothercycle.Toaddresstheseconcerns,theiterativeincrementalapproachbrokerequirementsupintopriority(anddependency)subsets,eachofwhichcouldbetakenthroughthelifecycleinashorterperiod.Thisisillustratedbelow.
12of32
Figure7-IterativeIncrementalLifecycle
Notethatthephasesarestillthere,buttheyareshortersinceeachistacklingasmallerscope.Thereisearlierandmorefrequentdelivery.AsintheDigure,thereisalsoopportunitytooverlapthedevelopmentifmultipleteamsareavailable.Agilemethodsrecognisethatrequirementsareoftenunclearormayevolveduringthelifecycle,sotheyaccommodaterequirementschangebothwithinandbetweeniterations,Seebelow.
Figure8-AgileLifecycle
Byallowingthis,theyovercometheproblemofwastedeffortonincorrectorobsoleterequirements.Aswementionedpreviouslythough,thereareanumberofpreconditionstogettingAgileright,aswellasdangersfromitsmisapplication.ThecombinationofAgileMethods,Virtualisedinfrastructure,IntegrationbetweenAgileDevelopment,AutomatedTestingandDeploymenthasledtotheDevOpsmovement,whereDevelopmentandOperationsarebroughtmuchclosertogether.
13of32
Figure9-DevOpsLifecycle
Therequirementstodeploymentcycleisshortenedbyintegratedmethods,highcollaborationacrossfunctionalareas,andadvancedintegratedtoolchainsthatcantaketheoutputfromdevelopmenttools,feedittotestsuitesandtheitemsthatpasstestingtooperationsinaseamlessprocess.Continuousdeploymentisanextremeformofthiswhereeachfeatureorchangecanbedeployedimmediatelyitistestedina“minirelease”.Thiswilloftenalsorequiretheadoptionofserviceorientedapproachessothatdeploymentofcomponentscanoccurwithoutneedingacompletemonolithicproducttobebuiltorlinked.Inadvancedshops,suchasFacebookorAirBnb,therecouldbeseveraldozen“pushes”intoproductiononadailybasis.Google,forexample,maydeployseveraldifferentchangescenariosinparallelandmonitorwhicharemoresuccessfulinproduction,beforedecommissioningthelesssuccessfulcandidates.
LeveragingPriorEffort(Doinglessthings)
Theslowestwaytobuildsomethingisdoingitfromrawmaterials,uniquelyforthecurrentrequirement.Ananalogyherewouldbebuildingastonestructurewherestonesareindividuallyprepared.
14of32
Figure10-PartofanoldstonebuiltfoundryinMoratacPark,NorthCarolina
Thinkabouttheeffortinvolvedinconstruction:• Findingandtransportingstone• Cuttingstonetosizeandshapebyhammerandchisel• Fittingeachuniquestonecarefullywithitsneighbours• Cuttinglumber,planingitintoplanks,makingdoorframesanddoors• Obtainingbrassforhinges,cutting,beatingandDilingitintoshapetoformhinges,
handlesetc.Thisiscertainlynotarapidprocess.Itrequireshighskillsandagreatdealofeffort.Changingoradaptingtheresultingstructurewillnotbeeasyorquick.InITterms,thisisanalogoustodevelopingasystemfromsourcecodeonly,withoutusinganypre-existinglibrariesorinfrastructuresoftware.Letslookatsomeoptionstoachievegreateragility.
UseComponentLibrariesandFrameworks
Ifwehaveaccesstopre-madecomponents,wecanfocusmoreonconstructionandlessonthebasiccomponentmaking.ConsidertheDigurebelow.Herewehavebricks,cement,tiles,timbertrusses,doorframesetc.deliveredtosite.Theprimaryefforthereisincombiningtheseintouniquecombinationstomeettheclient’srequirements.Effortismuchlowerandtimetobuildmuchless.Wecanprobablyalsogetbywithlowerlevelsofskill.
15of32
Figure11-RapidConstructionwithPre-BuiltComponentsSource:https://i.pinimg.com
IntheITenvironmenttodaywehaveverylargelibrariesofcomponentsdesignedtoworktogetherandtoprovideforthebuildingofnearlyallcommonsolutionsandcapabilities.Frameworksarelargecomponent(alsoclassandresource)librarieswhichprovideintegratedandcompatiblesupportaswellasarchitecturalguidanceonhowtolayerandcombinetheelementssuccessfully.ExamplesincludeMicrosoft.Net,Java2EnterpriseEdition(J2EE)andApple’sCoreandApplicationServices.Eachoftheseprovidesdeveloperswithaplethoraofcapabilitieswhichcanbeaccessedusingcommonapplicationprogrammingparadigmsandfacilities.Theyinsulatedevelopersfromagreatmanytechnicaldetails.Takingpre-fabricationtothenextlevel,wecanmakeuseofcompletepre-fabbuildings,deliveredtositeandjustconnectedtotheinfrastructure(e.g.water,electricityandsewage).InITterms,thisequatestotheuseofpackagesforatleastpartofthesolution.Theideaisthatweusecommoditydesignsandconstructionsforgenericrequirements,andonlyadaptordevelopbespokeelementswherenostandardonesareavailablethatwilldothejob.Packagescanreducetimetomarketandinvestmentrequireddramatically,iftheysuitrequirementswellandarenotheavilymodiDied.Themorewemodifythem,themorewelosetheadvantagesweseekandslipbackintodevelopment,sometimeswithrequirementsforrareandexpensiveskillsinesoterictools.
16of32
Figure12-PrefabricatedStructuredeliveredtosite
UseReferenceModelsandPa:erns
Componentsreducetheefforttobuildthesolution.Wherewehavetodesignthesolutionorelementsofit,wecansavetimebyusingreferencemodelsandpatterns.Patternsareatadesignorarchitecturelevel,ratherthanthephysicalcomponentlevel.Theycaptureknowledgeofprovenapproachesthatworkandthatcanbeeasilyadaptedtoouruniqueneeds.Anexamplewouldbethepatternusedforaclinic,wheretherewouldbeareceptionarea,assessmentarea,consultationrooms,treatmentarea,dispensingfacility,testareaandablutionfacilitiesforstaffandpatients.Thepatterncouldbescaledupordowndependinguponthebudgetandthenumberofclientsitisexpectedtoserve.
17of32
Figure13-Ahospitalarchitectureblueprint,orpatternSource:www.hhbc.in
AnITequivalentwouldbetheModelViewController(MVC)patterningraphicaluserinterfacesystems,whichprovidesarchitecturalguidanceinhowtosplitresponsibilitiesbetweenlayersofthesoftware.Thiscanbeadaptedfordesktop,webormobileuse.Wherepatternsaddressparticulardomainknowledge,theyaretypicallyreferredtoasreferencemodels.ExamplesinindustryincludetheBAINreferencemodelsforservicesinBanking,theACCORDmodelsinInsurance,theARTSmodelsinRetailandtheFrameWorxmodelsforprocess,dataandapplicationsinTelecommunications.
Figure14-FrameworxfromtheTelemanagementForumprovidesgenericmodelsforTelcos
Thesecanembodyhundredsofyearsofanalysiseffortandcansaveagreatdealofworkduringanalysis,architectureanddesign.Theyalsotypicallyfacilitateinteroperabilityofoursolutionswithbusinesspartnersandpackagesobtainedfromindustry.Theirusecanreduceriskandtimetodelivery,aswellascost.Thesemainlyreducetimeinanalysisanddesign,butcanalsoshortenbuild,testandintegrationtimebyusingprovensolutionsandmodels.
18of32
GenerateDetail
Developmentnormallyincludestheactivitiesofrequirementselicitation,design,buildingandtesting.Requirementsanddesignarebestdoneasmodelstoensuretheircompleteness,rigourandaccuratecommunicationbetweenparties.Thebuildandtestactivitiesareoftenthemostlabourintensive.Softwaregenerationapproacheseliminatealotofthiseffortbygeneratingthecodefrommodelscapturingtherequirementsanddesign.Traditionally,thishasbeentoutedasamajorboontoproductivity,buthasseldomdeliveredinpractice.ApproachesinthisspaceincludeModelDrivenDesign(MDD)fromtheObjectManagementGroup(OMG).Problemsinachievinghigherproductivitywerefoundtobemostlyduetothemodelsbeingsolution/technologyorientedandgenericratherthanrequirement/domainorientedandspeciDic.Ananalogyherewouldbemodelingahouseintermsoffoundations,brickwalls,doorandwindowframes,raftersandrooDingsheets.Thesearegenericcomponentsatafairlylowlevel.Specifyingthemodelatthisleveldoesnotsavealotofeffortoverbuildingit.However,ifwecouldspecifythehouseatthelevelofaconceptualarchitecture(styleandDloorplan/roomsonly)andgeneratetheDinishedhousefromthatautomatically(e.g.by3DPrinting)thenwecouldseemajorgainsinproductivityandrapiddevelopment.
Figure15-3DPrintedHouseSource:https://all3dp.com
19of32
ThesystemsequivalentofthisisknownasDomainSpeciDicModeling(DSM).Requirementsaremodeledusingdomainconcepts,suchasCustomer,Product,Order,Branchandassociatedprocessesandevents.Codegenerationproducesworkingsoftwarerespectingissuessuchasarchitecture,standardscompliance,securityandothercontextualrequirementsandtargetingthetechnologyplatformofchoice.Therearemajorsavingsintimeandcost,sincetheefforttocreatethearchitectureandgeneratorsisexpendedonlyonceandthegeneratedcodeiserrorfreesinceitfollowspreviouslytestedpatterns.
DomainSpecificModelingandGeneraEon
Figure16-DomainSpeciQicModelingActivitiesSource:MetaCase
Inindustrialprojects,DSMhasprovenhighlyproductive.Itisusedextensivelyintheautomotiveindustry,forexample,torapidlygeneratenewsoftwarefornewgenerationsofvehicles.Anewmodelcouldbelaunchedeverytwoyearsorsorequiringmillionsoflinesofcustomcodetomanagefuel,emissions,powerdelivery,transmission,traction,braking,entertainment,comfortsystems,lighting,navigationandamyriadotherthingsreliably.Codingthisintheconventionalwayintherequiredtimeandwiththerequisitereliabilityandqualityissimplynotfeasible.Otherindustrieswhichrelyheavilyuponthesetechniquesincludetelecommunications(e.g.developmentofswitches),cellular(developmentofhandsetsoftware)andotherelectronicsmanufacture,wheredevicesareincreasinglydrivenbysoftware.DSMisnotrestrictedtotheseDields,buthasalsobeenusedeffectivelyinbanking/Dinance,insurance/assurance,agriculture,plantautomationandotherapplications.
20of32
DSM Solution Development Time
Man days
Figure17-DSMlanguageandgeneratordevelopmentdoesincuranoverhead,butitcanbesmall.Source:MetaCase
ThereisanoverheadincreationofthedomainspeciDiclanguageandassociatedgeneratorsfortheDirstproject(s),butthesavingsdownstreampaybackhandsomely.
Figure18-DSMdevelopmentmayberecouped,eveninaninitialprojectSource:MetaCase
21of32
Results of the studies � Laboratory study
– Measuring time: at least 750% faster – Asking opinions: results (scale 1-5, 5 best):
� Pilot – Measuring time:
>900% faster
Figure19-DSMProjectsvsConventionalDevelopmentSource:MetaCase
Overallproductivity,qualityandmatchtodomainrequirementscanbeimprovedsubstantially,aswellastimetomarket.
ReuseandEnsureAdaptability
Anotherwaytoreduceeffort,andthustime,todelivery,istoreusewhatwehavedonepreviously.Thiscanoccurintheformofcomponents,frameworks,patternsandservicesasdetailedpreviously.Itcanalsooccurintheformofreusingcode,whichwehaveensurediswellstructuredenoughtobeintelligible,welldocumentedenoughtochangesafelyandwellstoredinasafeplacesothatitiseasilyDindable.Afurtherrequirementforreuseisthatthecode,components,frameworksetc.areofgoodquality.Wedonotwanttoreuseandproliferateproblemsandbugs!Achievingreuseisnoteasyandrequirespremeditation,discipline,skillsandtoolsupport.Someareasweneedtopayattentiontoinclude:• Architecture,sothatcomponentsarecoherentandfocussedonachievingone
purposewell• Design,sothatelementsarewellconceived,havecleaninterfacesandcaninter-
operate• Standards,sothatthingsarecompatibleandcompliant• Serviceorientationandloosecoupling,sothatitisrelativelyeasytoswap
componentsinorout,ortoreusethingsindifferentcontexts• Reliability,toensurethatwecansafelyusethingswithoutexperiencingproblems• Performance,sothatelementscanbeusedindifferentscaleapplications• Security,toensurethatwedonotexposetheorganizationtounduerisk
22of32
• Documentationandknowledgemanagement,sothatcomponentsareeasilydiscoverableandpotentialuserscanassesstheirsuitabilityquicklyandeasily
DoLessThingsOurselves
Anotherwaytoacceleratedeliveryistodolessthingsourselves.Wehavepreviouslydiscussedsavingeffortbyusingcomponents,libraries,frameworksandreferencemodels.Wecanalsoachieveitbygettingsomeoneelsetodosomeofthework.Thismaynotsaveeffortorcost,butitcanalleviateresourceconstraintsandreducecalendartimetodelivery.Apre-requisiteforthisisthatwehaveagoodideaofrequirementsandarchitectureatahighlevel,sothatwecanapportiontheworkandresponsibility.AfurtherrequirementisthatwehaveaproductiveandtrustedoutsourcepartnerwhowillDitintoourworkapproachanddeliverproductsandcomponentscompatiblewithourenvironment.Unfortunately,Dindingasuitablepartnerandbuildingthenecessarycollaborativeworkmethodsmaytakealotofeffortandtime.Itabitlike:“Godgrantmepatience,buthurry!”.
MakemoreFlexibleThings
ADinalwayinwhichwecanbecomemoreagile,isdifferenttothoseproposedabove.Essentiallythepreviousapproachesconcentratedmostlyonbeingabletodelivertheendproductmorequickly.ThereisanorthogonalapproachwhichinsteadshiftsthefocustodeliveringaproductwhichisitselfmoreDlexible,obviatingtheneedfornewdelivery.Thisharksbacktoourtitleof“AgileisaStableRequirement”.Aphysicalexampleofthiswouldbethesituationwherewebuildaconferencecentre.Weknowupfrontthatthefacilitymustbeabletoadapttoaccommodatetheneedsofawidevarietyofevents,rangingfromtradeshows,tomusicconcerts,agriculturalshows,sportsevents,businessconferencesandindustrycoursesandtutorials.Weneedtobeabletoaccommodatetheeventsinshortorderandinanysequenceofvariety.Thisrequiresthatthefacilitywedesignandbuildmust,itself,beabletoadaptveryquickly.ThiscanbeaccommodatedbyhavinglargecoveredspaceswhicharereconDigurableviamovablewallsintodifferentspaces,whilecateringinmorepermanentwaysfortheneedswhicharecommonacrossallfunctions,suchascatering,parking,ablutionsetc.AgreatexampleofthisistheCapeTownInternationalConferenceCentre(CTICC).ItcanbeconDiguredinlessthan24hourstocaterforalltheabovementionedtypesofevents.Thisismorerapidthananyconstructionmethodcouldpossiblyhopetoachieve.ThereconDigurationisachievablebecausethefacilitywasdesignedwiththislevelofadaptabilityinmind.Thefocusisonanadaptableresultingproduct,ratherthanonthebuildprocess.
23of32
Figure20-CapeTownInternationalConventionCentre-AconQigurablefacilitySource:CTICC
InITterms,thiscanbeachievedintwomainways:• RuntimeadaptablesystemswhicharedrivenbyuserconDigurablemetamodels,
businessrules,outputparametersandformulae.Theycanbeadaptedbyuserswhoareknowledgableaboutthebusinessdomainwithoutneedingtoknowtheunderlyingtechnology
• ModelDrivenDevelopment,especiallyDSMasdiscussedabove.HerethesystemspeciDicationismaintainedasasetofintegratedmodels.Whenchangesarerequired,thenecessarychangesaremadeatthebusinessdomainmodellevelandnewcodeisgenerated,potentiallydirectlytoproduction
TheformerstyleallowsgreatDlexibilityandcancaterformanychangestobusinessrequirementswithoutneedingtechnicalsystemchanges.AlimitationisthatthesystemwilloftenbelessefDicientthanalessDlexiblesystem,sothisapproachmaynotbeappropriatewhereveryhighvolumesmustbeprocessedorresponsetimesareverycritical.ThesecondstyleisalmostasDlexible,butdoesstillrequirethegenerationanddeploymentsteps,althoughthesecanbeautomated.Itisabletoaddresshighdemandsforvolumesandcriticalperformance.Ineffect,theabovedebunkstheYouAren’tGonnaNeedIt(YAGNI)principleadvocatedinAgilemethodstoreducescope.WereplaceitwithYAGNI,butmeaningYouAREGoingtoNeedIt,the“it”beingagility.Thegoals,howeverarenotincompatible.RememberthatAgileevolvedtoassistintherapiddeliveryofbusinesscapabilities.YAGNIwasappliedtoreducescope,sothatwecouldreduceefforttodelivermorerapidly.So,ifwenowturnthataround,areweincreasingscopeandeffort?No,butthereisatrick.Ifwesimplyaddedmorefunctionforthingsthatwemightneed,then,yes,scopewould
24of32
increaseandthatwouldbebad.BUTifweworkatahigherlevelofabstractionandidentifythekindsofthingsthatwewillneednowandinthefuture,wecanoftenreducethescopeofthesystemfurther,thussavingeffortandtimeintheoriginaldevelopmentwhilealsoequippingthesolutionwiththeabilitytoadaptinproductionusewithoutmoredevelopment.So,thisisawin-win.Thecaveatisthatwemayrequirehigherskillsinconceiving,designingandbuildingtheoriginalsolution.
RunEmeAdaptableSystems
AnexampleofsuchasystemfamiliartomostofusisMicrosoftExcel(orotherspreadsheets),whichprovidesarangeofgenericfacilitiesthathelpuserstailorthesystemtotheirneedsviaformulas,headingsandothercapabilities,suchaschartdeDinitions.Thegenericcapabilitiesanticipatethefactthatuserswillhaveaverybroadrangeofrequirementswhichdifferinthedetails,buthaveunderlyingsimilarities(e.g.theneedtoworkwithnumbers,manipulatethemwithformulas,organisethemintotablesandsorttheminvariousways.Usersareabletousetheapplicationtomeettheirneedsinshortorderwithoutrecoursetodeveloperstomakechanges,orthedelayandcostofsoftwaredevelopment.SpreadsheetshaveconsistentlymeasuredveryhighproductivitylevelswhenempiricaldataisanalyzedfordeliveryofgivenrequirementsinspeciDiclanguageenvironments.WitnessthefollowingstatisticsfromCapersJones:
LANGUAGE
1st Generation default
2nd Generation default
3rd Generation default
4th Generation default
5th Generation default
ABAP/4
Access
ANSI BASIC
ANSI COBOL 74
ANSI COBOL 85
ANSI SQL
Assembly (Basic)
Assembly (Macro)
C
C++
CICS
LEVEL
1.00
3.00
4.00
16.00
70.00
20.00
8.50
5.00
3.00
3.50
25.00
1.00
1.50
2.50
6.00
7.00
AVERAGE SOURCE STATEMENTS PER FUNCTION POINT
320
107
80
20
5
16
38
64
107
91
13
320
213
128
53
46
25of32
Figure21-RelativeProductivityofDifferentLanguagesSource:SoftwareProductivityResearch
LanguageLevelisarelativeproductivityindicatorforcomputerlanguages.Alanguageratedas12wouldhalvetheprogramming(build)phaseofaprojectrelativetoalanguagerated6.Ifthelanguageisalsoaccessibletodomain/businesspersonnelratherthanprofessionalcomputersystemdevelopers,itmayalsoreduceeffort,timeandcostintheanalysisanddesignphasesaswell.AnotherexampleofaruntimeadaptablesystemistheEVANetmodelerenterprisemodelingandknowledgerepositorytoolsetfromInspired.org.ThisallowsruntimedeDinitionofametamodeldescribingtheconcepts,propertiesoftheseandrelationshipsbetweenthem,ofrelevancetoauser,group,domainorenterprise.Itthenusesthisinformationtomodifyuserinterfaces,reports,visualizationtoolsandotheraspectsdynamicallyatruntime.
COBOL
Common LISP
Crystal Reports
DELPHI
EXCEL 5
FORTRAN 77
Haskell
HTML 3.0
JAVA
Machine language
Object-Oriented default
Objective-C
PASCAL
PERL
PowerBuilder
Reuse default
RPG III
SMALLTALK
Spreadsheet default
SQL
Visual Basic 5
3.00
5.00
16.00
11.00
57.00
3.00
8.50
22.00
6.00
0.50
11.00
12.00
3.50
15.00
20.00
60.00
5.75
15.00
50.00
25.00
11.00
107
64
20
29
6
107
38
15
53
640
29
27
91
21
16
5
56
21
6
13
29
26of32
Figure22-EVANetmodelerArchitectureSource:inspired.org
Inoneinstancewemetwithaclientinthebankingindustryandtheirindustryconsultantstodeterminerequirementsinsupportofastrategy,architectureand“re-baselining”projecttobeconductedacrossthreecontinents.Wedrewtheseasametamodelonawhiteboardduringalongworkshopday.IntheeveningwecapturedthemetamodeltoconDigurethetooltoaddresstheproblemandsetupaninstanceonaserveravailableviatheInternet.Thefollowingdaywedemonstratedthetoolsupportfortheproblemsbeingtackledtotheconsultantsandgottheirusercredentials.Wecapturedtheseintothetoolthateveningandthefollowingdaythetoolwasbeingusedtocaptureinformationfromliveworkshopsonthreecontinents.Thekeytothisrapiddeploymentwasagainthefactthatthetoolsetisdesignedtocaterformanyandvariedrequirementsbyfactoringoutwhatiscommontothemandsupportingthatinawaythatuserscantailortotheirneeds.Technically,thetoolhaspatternsforbusinesslogicanduserinterfaceinvariousstyles,andinjectsthestructurestowhichtheseshouldapplyintothecodegeneratedfromthese,whichisservedtoabrowserviathenetwork.Moredetailcanbefoundin[McLeod,2001].
27of32
Figure23-RelativeFlexibilityandProductivityofTechnologies
Agility andLegacyAchievingagilitygoingforwardistoughenough.Butmostsiteshavealargecollectionoflegacyapplicationswhichrepresentasubstantialasset(oratleastexpenditure)andunderpinoperations.Oftenthesearepoorlydocumentedandhave“unravelled”tosomeextentthroughpoormaintenance,sothattheiroriginalarchitecturesanddesignhasbecomeobscured.Howdoweleveragetheseandmakesuchanenvironmentmoreagile?Wecanveryseldomaffordtothrowitawayandstartfromscratch.Evenifthiswasaffordableeconomically,itwouldprobablynotbepracticalfromatimeperspective.WemayalsoDindthatthedomainknowledgeisnotavailableintheenvironmentandusercommunity,butisembeddedintheoldapplications.Conventionalmaintenancereliesalotonreadingcode.Thisisadauntingprospectwhenweconsiderthatalegacyapplication(ofwhichtheremightbescoresorhundreds)caneasilyrepresentseveralmillionlinesofcode.Surveysshowthatmaintenanceprogramersspendasmuchas80%oftheirtimereadingcode.Thisactivityhasn’tchangedmuchsincethe1950’s,withtheexceptionthatthevolumeofcodehasincreasedexponentially.Thisisapoorwaytounderstandanexistingsystem.Itdoesnotscale.Fortunately,thereisanewmovementadvocatedbyTudorGirbaandcolleagues,knownasHumaneAssessment.Theideaistounderstandsystemsrapidlywithoutbeingcrueltoprogrammers.Wealreadyapplysophisticatedtoolstounderstandourbusinessdata
28of32
andtogeneratevisualisationsandinsightsfromvastdatacollections(“bigdata”).Heretheprogrammersareliketheshoemakerschildrenwhoarebarefoot.Canwenotdobetterandusebettertoolsandtechniquestosolveourownproblemsaswellasthoseofusers?Indeedwecan.Techniquesknownassoftwarevisualizationhavebeendevelopedoveraperiodoftwentyyearsandthetoolssupportingthese,includingtheMooseanalysisplatformanditsaddonsandextensionshavebecomeverysophisticated.
Figure24-SoftwarevisualizationwithMoosetools
Essentially,wecanscanexistingsystems(code)toidentifyrelevantdata(e.g.ClassesordeDinitionsrepresentingconceptsanddatastructures;Functions,ModulesandMethodsrepresentingfunctionsoractionswhichoperateonthedatastructures;andtherelationshipsbetweenthesee.g.Whichcodedependsuponwhichotherpartsandwhichcodeuseswhatdata).Thedatacanthenbevisualizedusingavarietyofavailabletechniquestoidentifystructure,problemsandopportunitiesforimprovement.Thiscanfacilitaterapidmaintenancewithlesseffortandrisk.
Harvest,ReModel,ForwardGenerate
Takingitfurther,wecansupplementtheanalysiswithsomehumanexpertisetoderivedomainknowledge,semanticinformationandothervaluableassetsfromtheexistingsoftware.Thesecanbeusedtocreatedomainframeworksandreferencemodelsforforwardengineering.
29of32
WecanthencoupletherecoveredmodelswithDomainSpeciDicModelingtechniquestocreateapowerfulApplicationRenewalmethod.
HarvestKnowledge
AugmentKnowledge
GenerateSolution
LegacySourceCode
DomainSpecificModels
TechnicalArchitect
DSMDriven
Application
NewSolutionCode
NewSolutionRuntime
TargetIDE
ReferenceModel
DomainExpert
ContinuousBusinessSolution
Improvement
ContinuousTechnicalSolution
Improvement
High PerformanceSolution Required
Low PerformanceSolution Required
UserCommunity
Figure25-CodeHarvestingandForwardGenerationwithDSM
Inspirediscurrentlydevelopingandpioneeringthesetechniquesandisseekingcommercialpartners/customerstocollaborateontheseefforts.WeareengagedwiththeauthorsofcodeharvestingandDSMtechniquesandtooling.Ifyouareinterested,pleasecontactthewriter.
30of32
TheLists
Inthissectionwesummarisesomeofthecriticaldimensionsthatpromoteorinhibitagility.Inmanycasesthesewillhaveincrementaleffectswhenusedtogether.Inourculture,training,methods,managementpracticesandprojectsweshouldtryanddoasmanyofthePromoteitemsaspossible,whileavoidingasmanyoftheInhibititemsaspossible.
Figure26-Factorsaffectingagility
PromoEngOrganizaEonalAgility
Theforgoingdiscussionmayseemdaunting.Afterall,therearemanydimensionsandmanyassociateddisciplines,skills,cultures,techniquesandtoolstomasteroradopt.Wheredowestart?Itcanbeusefultotakealeafoutofthebookofcontinuousimprovement,orthemoreformaldisciplineofSixSigma,whileavoidingtherigorousstatisticalsideofthelatter.Effectivelywefollowasimplecycle:• EstablishIntent-Decidewhatwewanttoachieve(e.g.OrganizationalAgility)and
whatthatwillrequire,viadecomposition(e.g.AgileFriendlyCulture;AbilitytoRapidlyDeliverSystemsCapabilityinProduction;AbilitytoFlexBusinessModeletc.)
• Identifythebiggestbottleneckstoachievement.Whatareourcurrentworstperformingareas?Maybeitisaveryautocraticculture,maybeitisaninabilitytoaccuratelydeDinerequirements,maybeitisaqualityprobleminthedeliveredproduct.ThisisbestdonebylookingatspeciDicperformancemeasuresand
Promote AgilityHigh SkillsUser InvolvementFrequent Short MeetingsEmpowered StaffHigh Level of AbstractionDomain Specific ModelingIntegrated MethodsModel OrientedSmall TeamsAutomationHigh QualityGood InfrastructureLearning CultureColocated TeamsVisual Work, ProgressCollaboration. MentoringInformal with high trustDevelopers have Domain KnowledgeUsers have IT KnowledgeContinuity of StaffReuseTeam of TeamsRuntime adaptable productVirtual infrastructure
Inhibit AgilityLow SkillsUser Dis-engagementDelayed Large MeetingsCentralised or Remote Decision MakingLow Level of AbstractionGeneric ModelingNon-Integrated MethodsDocument OrientedLarge TeamsManual WorkLow QualityPoor InfrastructureDo What You are Told CultureDistributed TeamsHidden Work, ProgressPunitive, AuthoritarianFormal with low trustDevelopers have only IT KnowledgeUsers have only Domain KnowledgeDisrupted or Part Time Allocation of StaffBuilding AnewCommand and ControlDesign time adaptationTraditional infrastructure
31of32
associatedbenchmarks,comparedtoourperformance.E.g.Ifcompetitorscandeliveranewservicetothemarketin3monthsandwetake10.Findthetop3-5areaswhichareunderourcontrolandamenabletochange.Focusjustonthoseforthenextwhile
• Improvetheunderperformingareasbyfacilitatingculturalchange,improvingmethods,skills,techniquesandtools,buildingtrustandwhateverotherfocussedmeanswilladdresstheissue
• Iterate-oncechangeisachievedinanareaandveriDiedbynewmeasurement,selectthenextmostproblematicarea(s)forattentionandkeepdoingit
Inthiswaywecanbeassuredthatwearealwaysmakingprogress.ItmayseemslowatDirst,butitwillbeincrementalaschangescombinetheirimprovementsinamultiplicativeway.Wecanacceleratechangebytheapplicationofmoreeffort,resourcesormoney,providedthattheyarealwaysfocussedondoingthingsbetter,notjustfaster.
32of32
ReferencesandFurtherReading
PapersMcLeod,Graham.“Pamela:AProto-PatternforRapidlyDelivered,RuntimeExtensibleSystems.”EvaluationofModellingMethodsinSystemsAnalysisandDesign(EMMSAD)(2001)availablehere
PresentaEons
SoftwareVisualization,TudorGirba
Videos
AgileVisualisationinMondrianSoftwareEnvironmentalismIntroductiontoDomainSpeciDicModelling20DomainSpeciDicModellingExamples
Websites
http://www.inspired.orghttp://www.moosetechnology.orghttp://www.metacase.com
