TLA Specification Requirementstlacommunity.com/wp-content/uploads/2018/05/TLA-Specification... ·...

FLUENTFastLearningfromUnlabeledEpisodesofNext-GenerationTailoring

TLASpecificationRequirementsContract:W911QY-16-C-0019

Preparedfor:

Dr.SaeSchatz,DirectorAdvancedDistributedLearning(ADL)Initiative

Preparedby:

SoarTech

3600GreenCt.,Ste.600,AnnArbor,MI48105 Phone:734-327-8000•Fax:734-913-8537•Web:www.soartech.com

TLASpecificationRequirementsVersion:0.1.1 Page2of16

TABLEOFCONTENTSTableofContents.........................................................................................................................................2

RevisionHistory............................................................................................................................................3

1 Introduction.........................................................................................................................................3

2 TLASpecificationRequirementsSummary...........................................................................................3

3 Design-BasedResearchRequirementsAnalysis...................................................................................5

3.1 UseCaseInclusiveandForwardThinking....................................................................................5

3.1.1 ManyUsers...........................................................................................................................6

3.1.2 ManyDeploymentPlatforms...............................................................................................6

3.1.3 ManyProtocols.....................................................................................................................6

3.1.4 ManyComponents...............................................................................................................7

3.2 LongevityTechniques...................................................................................................................7

3.3 EncourageCommunityCollaboration..........................................................................................8

3.3.1 BuildOnExistingWork.........................................................................................................8

3.3.2 EasytoUse...........................................................................................................................9

3.4 DataCentric................................................................................................................................11

3.4.1 LotsofData........................................................................................................................11

3.4.2 PlugandPlay......................................................................................................................12

3.4.3 ProducersandConsumers..................................................................................................12

3.4.4 Extensible...........................................................................................................................13

3.4.5 Real-timeDataProcessingandStoredDataHistory..........................................................14

3.4.6 DistributedDataModels....................................................................................................14

3.4.7 PerformanceMinded.........................................................................................................15

3.4.8 AutomationFriendly...........................................................................................................15

3.4.9 Security...............................................................................................................................16


RevisionHistory

1 INTRODUCTIONTheTLAisanarchitecturetoenablelearningthatisdescribedbyspecifications.SeetheTLADesignDocument.docxforanintroductiontotheTLAarchitecture.ManytermsanddefinitionsareprovidedintheTLADesignDocument.docxaswell,sotheauthorssuggestthatreadersreferencethatdocumentasneeded.TLAspecificationsareformalizeddocumentationthatenablesmultipleproviderstocontributeindividualcomponentstoaTLAecosysteminsuchawaythatallcomponentsinteroperateseamlessly.Specificationsareaimedatlong-termgoals,and,thus,typicallyincludeenoughflexibilitytosupportmanyinstantiationsthatsupportdifferentusecases.TLAdevelopmentisusingadesign-basedresearchapproach;iterativeanalysis,design,development,andimplementationcycleswillshepherdtheinitialseedofanideathroughtomaturation.ThisdocumentdescribestherequirementsthattheTLAspecificationsneedtoaddress,aselicitedbythedesign-basedresearchapproach.TheinitialsetofrequirementsisanoutcomeofanalysisoftheYear1experimentalprototypedemonstratedatFortBragginApril2017.

2 TLASPECIFICATIONREQUIREMENTSSUMMARYThissectionbrieflysummarizesthehigh-levelTLAspecificationrequirementsidentifiedduringYear1oftheTLAdesign-basedresearch.Thesummaryinthissectionispresentedupfrontforreadersdesiringaquickoverview.Foradeeperunderstanding,pleasecontinueontosection3.Foreaseofreference,eachsummarytableinthissectioncorrespondstoamajortopicinsection3.Pleasenotethattheorderinwhichtherequirementsaresummarizedinthissectionreflectstheorderofdiscussionintheanalysissection;orderingheredoesnotimplyrelativeimportance.

SummaryofRelatedRequirements:UseCasesandForwardThinkingDecoupledfromspecificusecasesSupportmanytypesofuserswithoutmodificationeachtimeanewusertypeisidentifiedSupportmanytypesofdeploymentplatformswithoutmodificationeachtimeanewplatformtypeisintroducedSupportmanytypesoftransportprotocolsSupportmanytypesofsecurityprotocolsSupportinclusionofmorethanonetransportprotocolinasingleTLAinstantiation

Name Date ReasonforChange Version

SoarTech 7/18/2017 InitialDraft 0.1.0

SoarTech 7/26/2017 SummaryTable 0.1.1


SupportinclusionofmorethanonesecurityprotocolinasingleTLAinstantiationDecoupledatadefinitionfromtransportandsecurityprotocoldescriptionDecoupledfromspecific,instantiated,componenttypesSupportaddingnewcomponenttypeswithoutspecificationmodificationEasytomaintainSubdividedintofamiliesofrelatedspecificationstofacilitatemodularityandlongtermmaintenanceEasytoextendandrefineCompatiblewitha"provingground"workflowthatfacilitatesimprovementCompatiblewithbestpracticesthatbridgethegapbetweenlonglivedspecificationsandshort-livedimplementationdetails

SummaryofRelatedRequirements:EncourageCommunityCollaborationLeverageexistingstandardsandspecificationswheneverpossibletomeetrequirementsSupportcombiningexistingspecifications(includingpartsofexistingspecifications)Supportrecordingprovenanceforincluded,pre-existing,specificationsSupportcompatibilitywithmanyexistingspecificationsSupportcompatibilitywithoverlappingexistingspecificationthroughequivalencymappingorothersimilarmeansEncouragecommunitycontributionintheworkflowforextensionsandimprovementsEasytounderstandEasyfordeveloperstouseHumanreadableMachineinterpretableSupportautomatedcomponentdiscoverySupportcomputationalresourcesresidinginthecloud,oronatraditionalorganizationallyownedserverSupportnegotiationofresourcelimitswhennewcomponentsare"pluggedin"SupportcostbasedresourcethrottlingSupportcommunicationofcomponentQoScapabilitiesandneedsSupportfaulttoleranceSupportrespectinglearnerprivacySupportdataremoval

SummaryofRelatedRequirements:DataCentricSupportadatacentricapproachUseasmallnumberofeasilyunderstoodlogicaldatamodelsUseanon-componentcentricdataorganizationparadigmsothataddingusecasesdoesnotresultinanexplosionoflogicaldatamodelsSupportlogicaldatamodelsthatarecapableofsharinglargequantitiesofdataacrossmanyfunctionalareasSupportsharingdataaboutscienceoflearningconceptsandsoftwareassetsinadditiontodataaboutthelearnerexperiencesSupportutilizationofmanyscienceoflearningunderlyingtheoriesSupportutilizationofmanycontentsequencingstrategies


Supportaproducer/consumerdataflowmodelClearlydescribedataconcept:Whatdataisshared?Clearlydescribedatastructure:Howistheshareddatastructured?ClearlydescribeconceptualAPI:Whatcanbesaidaboutthedatabetweenproducersandconsumers?Decoupledataconcept,datamodelandconceptualAPIfromspecificusecasespecificassumptionsSupportaddingnewdatatypestosupportnewusecaseswithoutrequiringaspecificationmodificationeachtimeanewpieceofdataisidentifiedSupportConceptualAPIflexibleenoughtonottorequireaspecificationmodificationeachtimeanewusecaseisintroducedSupportdeveloperfriendlyAPIforrequestingfiltereddatafromdataproducersSupportstoringdatahistorytosupportnon-real-timedataanalysisSupportbothreal-timedataandnon-real-timedataexchangesSupportmetadatafordataproducerstoadvertisethepartsofthelogicaldatamodelstheyproduceSupportmetadatafordataproducerstoadvertisewhethertheykeepahistoricalrecordofthedatatheyproduceforconsumerstoretrieveatalatertimeSupportadistributeddatamodel,wherethedatacanresideindifferentphysicallocations,butbeaccessedbyalldataconsumersasthoughitwereasingle,globallyshareddatamodelSupportdistributingthelogicaldatamodelsuchthatdataproducersthatcanproviderhistoryarenotrequiredtotransferthedatatoaphysicallycentralizeddatastoragelocationSupportmultiplemanytypesofscaleouttechniqueswithoutrequiringaspecificationmodificationSupportmanytypesofloadbalancingtechniqueswithoutrequiringaspecificationmodificationSupportdatafilteringthatoptimizesnetworkbandwidthuseSupportprogrammaticallyaddingandmodifyingmetadatainawaymakesiteasytoautomatemetadatapopulationSupportmanymetadatapopulationautomationtechniquesDecoupledatadefinitionfromdescriptionsofsecuringatrestdata

3 DESIGN-BASEDRESEARCHREQUIREMENTSANALYSISThissectionprovidesmoredetaileddescriptionsoftherequirementsintroducedinsection2.ItdescribeshoweachrequirementwasidentifiedduringYear1ofTLAdesign-basedresearch.

3.1 USECASEINCLUSIVEANDFORWARDTHINKINGItisn'tenoughtoproducespecificationsthatmeettoday'srequirements;TLAspecificationsshouldberelevantformanyyearstocome.BecausethedesiredlifespanoftheTLAislong(aimedat2025),itisnecessarytoconsiderhowtostayrelevantinthefaceofemerginginstructionalmethodologiesandlearnerneeds.ToensuretheTLAlifespanisnumberedinyearsordecadesratherthanmonths,oneofthecoredesignrequirementsisthatitmustbedesignedsuchthatnewandunexpectedusecasesareencouragedandwelcomed.AssumingthatthenumberofusecaseisnotfixedmeansthattheTLAmustbedesignedtobeflexibleandadaptable.HowdoesthisimpactspecificationsdescribingtheTLA?Themostsignificantimpactisthatassumptionscoupledtospecificusecasescannotbebakedintothespecification.AbstractionsthatareapplicabletomanyusecasesmustbeelicitedandutilizedsothattheTLAspecificationsdonotneedtochangeeachtimeanewusecaseisintroduced.Inthissection,wewillexploresomeofthedimensionsthatrequireabstractiontomaintainflexibilityacrossusecases.


3.1.1 ManyUsersIfthereisnotafixedusecase,thenitfollowsthattherecanbemanydifferentusers.SomeofthetypesofusersthatYear1'sresearchhasidentifiedinclude:

• Learners• ContentCurators• Researchers• Instructors• Network/IAAdministrators

WhatdoesthismeanfortheTLAspecifications?Sincethereisnosingleuser-facingapplicationthatwillinterfacewiththeTLA,itnotpossibletoenumerateallofthewaysthattheuser-facingapplication(s)willwanttousethedatathattheTLAmakesavailable.Therefore,thespecificationmustbedesignedsuchthatdifferentuser-facingapplicationswithdifferentgoalscanbeconstructedwithoutaspecificationchangewhenanewuser-facingapplicationisneeded.Examplesofsuchapplicationsincludelearningactivitiesandinstructordashboards.

3.1.2 ManyDeploymentPlatformsTomeettheneedsofavarietyofusecases,theTLAspecificationsshouldbeinclusiveinregardtodeploymentplatforms.Weliveinaworldofconstantlychanginghardwareanddevices,andtheconsumerdevicemarketchangesespeciallyrapidly.Tobeeffectiveovertime,theTLAspecificationsmustbecapableofsupportingtheenduser'sdesiredhardwaredeliverydevices.Todaythatmightmeanlaptops,tablets,andcellphones.ButoverthelifetimeoftheTLA,itmustbeextensibletoincludenewplatformsastheyemerge,fordeliverydevicesinsmartcarsorsmarthomeappliances.ToviewananimateddepictionofhowausermightinteractwiththeTLAinthefuture,followthislink:

http://goanimate.com/videos/0EoEFEKlWNxY?utm_source=linkshareandutm_medium=linkshareandutm_campaign=usercontent.

Whenanewdeliverydeviceemerges,itshouldnotrequireaTLAspecificationmodificationtobeginusingitwithaTLAinstantiation.

3.1.3 ManyProtocolsSpecificationsaredesignedtohelpdeveloperscommunicate.Thisoftenrequiresagreatdealofspecificity.Whilespecificityfacilitatesnear-termcommunication,itcanimpedelong-termmaintenance.Ifausecaseistightlydefined,thenassumptionscanbemadethatsimplifyhowtoaddressthelevelofspecificitythatisrequired.Thisisparticularlyprevalentinregardtotransportprotocolsandsecurityprotocols.Forexample,atightlyconstrainedusecasemightbeabletomakethesimplifyingassumption"allsoftwarecommunicationnecessarycanbeaccomplishedusingREST."Whenthisisthecase,thespecificationsforaddressingtheneedsofsuchusecasesareoftentightlycoupledtoaspecifictransportprotocol,suchasREST.Thisconflatesthedescriptionofwhatthedataisandwhatcanbesaidaboutitwiththemethodusedtotransportitacrossthewire.Similarly,securityprotocols,orhowtosecurethedatabeingtransported,canbeconflatedwithwhatthedataisandwhatcanbesaidaboutit.Thisworksreasonablywellforsystemswhoseusecasesarenotexpectedtovaryovertime.However,sincetheTLAisdesignedtowelcomenewandunexpectedusecases,thesekindsofsimplifyingassumptionsdonotsatisfylong-termrequirements.Asanillustration,let'sconsiderthechoiceofRESTasthetransportprotocolfortheYear1experimentalprototypeandinitialdraftoftheTLAspecifications.RESTis


fundamentallyapullprotocol;whenacomponentwantsinformation,itasksforitdirectly.TheFortBraggexperimentrevealedthatinadditiontopull,pushingisalsoneededtosatisfythemeta-adaptationusecase.Someofthecomponentvendorswouldliketoregisterinterestinaneventhappeningandreceiveanotificationifthateventtakesplace(push).Inparticular,theFortBraggexperimentelicitedtheneedtomixpushandpullprotocolsinthesameimplementation.Tomeettheflexibilityneedsthatdifferentusecasesdemand,thetransportandsecurityprotocolsmustbedecoupledfromthedescriptionofwhatthedataisandwhatcanbesaidaboutthedataintheTLAspecifications.

3.1.4 ManyComponentsDesigninganarchitecturethatcanbeadaptedtodifferentusecasesmeansthatitmustprovidebuildingblocksthatcanbecomposedintomanydifferentinstantiations,whereeachinstantiationistargetedataspecificsetofusecases.Ifthespecificationiscomponent-centric,thenitwouldneedtobemodifiedeachtimeanewtypeofcomponentwasdiscoveredtobenecessarytomeettheneedsofaspecificusecase.InsteadspecificationsshouldsupportflexiblecomponentdescriptionssothatnewcomponenttypesthatconformtotheTLAspecificationscanbeaddedasnecessaryinindividualinstantiations.BasedontheYear1experimentalprototypingandanalysisofavarietyofpotentialfutureusecases,itappearsthatalltypesofcomponentscanbecategorizedas

• TLAConformantApps–User-facingtoolsthatuseTLAdatatomeetanenduserneed• Processors–Softwarecomponentsthatintakedata,transformitinsomeway,andoutput

anothertypeofdata• DataStores–Softwarecomponentsthatstore,butdonotmodifydata

TheTLAspecificationshould,therefore,provideameansforeachofthesegeneralcategoriesofcomponentstoutilizeTLAdata,withoutcouplingthespecificationtoaspecifictypeofcomponentsuchasan"EvidenceMapper."Thatis,theTLAspecificationcannotbetightlycoupledtospecificcomponentnames.

3.2 LONGEVITYTECHNIQUESToachievethevisionofalong-standingTLAspecificationthatiseasytomaintain,severalknowntechniquescanbeemployed.Asdiscussedintheprecedingsections,identifyingabstractionsanddecouplingthespecificationfromnear-termimplementationchoicescanbeverybeneficial.However,inpractice,thespecificationstillmustallowdeveloperstocommunicatewellenoughtoachievefunctionalimplementations.Fortunately,twospecificstrategiesmakethispossible.

1. BestPracticesguidescanbepublishedthatfillinpracticaldetailsthatindicatethebest-knownwaytoimplementaspecificationusingtoday'stechnology.Astechnologieschange,ifthespecificationwaswell-thought-out,itwillnotneedtobemodifiedforeachnewtechnologythatemerges.Instead,thebestpracticescanbeupdatedtodescribehowtoutilizenewlyemergedtechnologiestoimplementtheexistingspecification.

2. Specificationscanbedividedintofamiliesofrelatedspecifications.Implementationsmaythenutilizethesubsetofspecificationsthatbestmeetstheirneeds.Forexample,isolatingthedescriptionofwhatthedataisinonespecificationandprovidingseparatespecificationsthatdescribehowtouseindividualtransportprotocolswiththedatainareferenceddata


specificationcanallowimplementerstoselectthedatatheyneedwithoutanimpliedtransportprotocolselection.

Lastly,tosucceedatlong-termviability,theTLAspecificationmusthaveameansofinteractingsuccessfullywiththecommunitytoenablerefinementovertime.Todoso,aworkflowmustbedevelopedthatsupportsa"provingground"foridentificationofnewideas,evaluationoftheirmerit,andpromotionofbest-of-breedideasfromtheprovinggroundintotheformalspecifications.

3.3 ENCOURAGECOMMUNITYCOLLABORATIONThegoaloftheTLAistocreateanecosystemthatfosterscollaboration.Insteadofbuildingtrainingsolutionsbycontractingasinglesolutionprovidertobuildthefullproduct,ADLenvisionsanenvironmentwheremanyvendorscanplugincomponentsthatreflecttheirparticularareasofexpertise,therebycreatingalearningsystemthatsurpasseswhatcanbecreatedbyasingleprovider.Tobringthisvisiontofruition,theTLAspecificationsmustbecommunityoriented.Theymustbepubliclyavailableformanycollaboratorstouseandcontributeto.Inthissection,wewillexploresomeofthedimensionsrequiredtoencouragecommunitycollaboration.

3.3.1 BuildOnExistingWorkTheTLAspecificationsshouldnotstartfromscratch.Leveragingexistingstandardsandspecificationswheneverpossibletomeetrequirementsisaprimarygoal.Doingsowillallowustotakeadvantageofyears’worthofworkthecommunityhasputintodevelopingthem.InYear1'sexperimentalprototype,existingstandardswerenotincorporated.Thiswasadeliberate,andtemporary,choice.TherequirementsatthestartofYear1wereinsufficientlydefinedtobematchedtoexistingspecifications.AllowingfreeexplorationofideasduringYear1wasdonemindfully,withtheendgoalofelicitingrequirements.DuringthecreationoftheinitialexperimentalprototypeanditsdemonstrationatFortBragginApril2017,manyrefinementsweremadetoourinitialunderstandingofTLArequirements.ThefirstdraftoftheTLAAPISpecifications(version0.1)wasproducedasabasiccaptureofwhatwasnecessaryatanengineeringleveltoproduceafullyfunctionalTLAinstantiation.ThesepreliminaryAPIshavebeenusedbytherequirementsanalysisprocessasinputandhavebeenveryhelpfulinidentifyingrequirementsthatmayotherwisehavebeenoverlooked.Inputhasalsobeengatheredinotherways,suchasduringthe2016/2017Delphistudy.Wehavenowgatheredenoughinformationtoallowustostepback,analyzewhatwelearned,andwritedowntherequirementsthatwillneedtobeformalizedintospecifications(thisdocument).ThenextstepistoperformagapanalysiswhereweevaluatetheTLAspecificationrequirementsweelicitedoverthecourseofthefirstyearagainstthewidevarietyofexistingspecificationsoutinthecommunity.Agapanalysiswillhelpustounderstandwhichexistingspecificationswecanleverage,andwheretheTLAhasuniquerequirementsnotyetmetbyanyexistingspecification.

Tofosteracommunitycollaboration,theTLAspecificationshouldbeassembledbycombiningexistingstandardsthatmatchTLArequirements.Doingthiseffectivelywillbeachallenge,butthereareseveralgoalsthatshouldguidetheendresult.Whilesomespecificationsmaybeappropriatetoincludewholesale,therewillbeothersthatonlypartiallyoverlaptheTLA'srequirements.Inthecaseofpartialoverlap,amethodologyforincorporatingthepartialmatchmustbeidentified(suchasMetadataApplicationProfiles).Wedonotwanttoinventanewstandardthatoverlapspartofanexistingone.Whenexistingstandardsareincludedwhollyorpartially,provenancemustbetrackedsotheoriginof


theinclusioncanbetraced.Next,therewillbeplaceswheremorethanoneexistingstandardmeetsaTLArequirement.Wheretherearealreadyexisting,overlappingspecifications,thegoalfortheTLAspecificationshouldbetobuildonourspiritofinclusivity.Ratherthandown-selectingtoasinglematch,everyeffortshouldbemadetomaketheTLAcompatiblewithallmatches.Thismayentailproducinganequivalencymappingthatfacilitatesconvertingterminology(andunderlyingprinciples)fromonespecificationlanguagetoanother.Lastly,astheTLAisforgingnewgroundinlearningsystems,therewillundoubtedlybegapswhereTLArequirementsarenotmetbyanyexistingstandard.Asdiscussedinsection3.2,aprovinggroundwillbeneededwhereemergingideascanbeexplored,evaluated,andpromotedintocommunity-curatedspecifications.ItisimportanttoensurethatnewideasforgedintheTLAdevelopmentlifecyclegivebacktothecommunity,anditisequallyimportantthatthecommunityhasaviableavenueforcontributingtoTLArequirementsgatheringandspecificationdevelopment.

3.3.2 EasytoUseItwillbecriticalforgainingmomentuminthecommunitythattheTLAiseasytounderstandandeasytouse.Ofcourse,whenwesay"use"wethenhavetoidentifytheuser.Inthiscase,theuserisnotthelearner,butthepersonnelnecessaryforcreatingaTLAinstantiation.Thereareseveralusertypesthatneedtobeconsideredincluding:

• Organizationalleveldecisionmakers• Instructorsandcoursecreators• Developers• Network/IAAdministrators

InstantiationofaTLAinstancerequirescollaborationamongalltypesofusers,andtypicallythecontributorswillnotallbepartofthesamecompany/organization.

Therearetwoverybasicprinciplesthatareagoodfirststep.First,theTLAspecificationsshouldbehumanreadableandclearlywrittentobemeaningfultoawidereadingaudience.Thespecificationmustclearlydescribewhatthevaluepropositionisforparticipatingproviders.Itmustalsoclearlydescribehowandwhentousethedifferentpartsofthearchitecture.InYear1,valuablefeedbackwasgatheredfromtheexperimentalprototypedevelopmentteamindicatingthatforActivitydevelopersitwasdifficulttounderstandhowtheoutputcomponentstheydevelopedinfluencedthebehaviorofothercomponentsintheTLA(thatconsumedtheoutputtheyproduced).TheTLASpecificationconstitutesacontractthatfacilitatescomponent-to-componentcommunicationinsoftware,sowewillneedtoimprovethedescriptionofhowinputsandoutputsfunctiontogether.Tomakeiteasytomaketheleapfromadeveloperunderstandingthespecificationtoasoftwaresystemusingthespecification,thespecificationformatshouldalsobemachineinterpretable.Ifacomputercandirectly"read"thespecification,thenthedeveloperhaslessworktodo;thereisnointermediarystepofconvertingahuman-readableformatintoamachine-interpretableformat.

ForaTLAecosystemtobesuccessful,manyActivityproviderswillneedtocontributeindividualactivitiessothatcollectivelytherearemanyavailableforlearnerstobenefitfrom.Imaginehowboringtheinternetwouldbeifindividualsfromallovertheworlddidn'tfeelmotivatedtoaddcontentfortheworldtosee.TheprincipleisthesamefortheTLA:thespecificationshouldmakeiteasyforthosewhohavecontenttoshare,toshareit.Anditisnotonlylearner-facingcontentwewanttoencourageproviderstoshare.Wewantsmartpeopleandorganizationswhohavetheabilitytobuild


computationalcomponentsthatcanresideinsidetheTLACloudtobeabletocontributethoseofferingswithequalease.ThekeytakeawayfromYear1'sexperimentalprototypewasthat"pluggingin"anewcomponentintoaTLAinstantiationwastoohard/manual/time-consuming.Thisbearssurprisingsimilaritytothe"tyrannyofnumbers"problemfacedduringtheinventionofthemicrochip(seehttps://en.wikipedia.org/wiki/Tyranny_of_numbers).Thebasicpremisewasthatitwaspossibletoenvisionahardwaresystemthatwasmorecomplexthanitwascost-effectivetowiretogetherbyhand.Theeventualsolutionwasautomationleadingtomicrochipsasweknowthemtoday.Weareatapointinhistorywherethisisnowtrueofsoftware:wecanenvisionsystemssignificantlymorecomplexthanitiscost-effectivetobuild,duetothemanualnatureofconnectingsoftwarecomponentstogetherbyhandtomeettheneedsofindividualusecases.TheTLA'sgoalsputitintothiscategory:wewantanecosystemwhereitiseasyto"plugin"componentsandreconfigureaninstantiationtomeettheneedsofnewlydiscoveredusecases.Tomakeiteasyfordeveloperstodothis,newadvancesinautomationwillbeneededtosolvesomeaspectsoftheproblem.Forexample,coursecreatorsareaccustomedtomanuallysequencingcontent,butiftheTLAsucceedstherewillbemillionsofpiecesofavailablecontentforcoursecreatorstosiftthrough.Howcantheydosoeffectively?Theansweriscertainlysomeformofautomation,butwhatkindisnotyetclear.Forareassuchasthis,theTLAspecificationsmustbeflexibleenoughtoallowtheinsertionofsuchananswerwhenitisdiscoveredinfutureexperimentalprototyping.However,someaspectsofsimplifying"pluggingin"alreadyhaveindustry-acceptedsolutions.Onesuchexampleisautomatedcomponentdiscovery.Automatedcomponentdiscoveryallowscomponentsthat"plugin"toregisterthemselveswhenthey"plugin"andthendescribethemselvestoothercomponentsthatarealreadypluggedin.TheTLAspecificationneedstoincludesupportforautomatedcomponentdiscoverytomakeiteasyfordeveloperstocontributecomponentstotheTLAecosystem,andtolocateandutilizecomponentsprovidedbyothercontributors.

Instantiatingasoftwaresystemrequiresresources,computingpower,diskspace,andnetworkconnectivity,allwhichmustbepurchased.TomakeitaneasydecisionfordecisionmakersandITpersonneltoendorsetheTLA,theTLAspecificationsmusthelpalleviatetheircostconcerns.OurSpring2017demonstrationatFortBraggutilizedAmazonWebServices(AWS)foracquiringourcomputingresources.Wesawfirst-handhowsuccessatenticingparticipantstocontributenewandunexpectedcomponents(inthiscasereal-timedashboards),couldleadtoanunexpectedspikeintheusageofAWSresources,whichhasadirectmonetaryimpact.Tofacilitatethebalancebetweencreativityandcost,theTLAspecificationmustencourageimplementationsthataregooddenizens,respectingthecostofcomputingandstorageresources.Thespecificationshouldmakeiteasyforcomponentstopublishtheircomputingresourceneedsandnegotiateforareasonableallowancewhentheyare"pluggedin."Additionally,theTLAspecificationmustallowtheimplementerthechoiceofplacingtheircomputationalresourcesinthecloud(e.g.,AWS)iftheydesire.However,thisshouldnotbemandated;theTLAspecificationshouldalsobecompatiblewithimplementationsthatutilizeamoretraditionalserver-basedinstallationwithinasingleorganization.Inaddition,theTLAspecificationmustenablecost-based,real-timethrottlingsothatITAdministratorscanthrottlecomponentsthatareusingmorethantheirfairshareofresourcestopreventcostoverruns.

BecausetheTLAisintendedtobeinstantiatedandleftrunningforlongperiodsoftime,potentiallymanyyears,performanceandmaintenanceneedsmustbeconsidered.ForasmoothlearnerexperiencetheTLAshouldalwaysappearresponsive.Tomakethiseasytoachieve,theTLAspecificationneedstoincludetheabilityforcomponentstopublishtheirqualityofservice(QoS)requirementsforinputthey


require,aswellastheQoSlevelstheycansupportfortheoutputtheyproduce.Byallowingcommunicationoftheseimportantaspectsinthespecification,itispossibletoassembleinstantiationsthatmeetbothreal-timeandnon-real-timeusecases.Similarly,componentscanfailovertime.Foracontinuous,smoothend-userexperience,theTLAspecificationmustenablefaulttolerance.Forexample,hotswappingmustbesupportedsothatafailedcomponentcanberecognizedandreplacedwithoutcompromisingsystemavailability.

AnotherramificationofTLAinstantiationsthathavelifespansnumberedinyearsisthatlargequantitiesofdatawillbeaccumulated.Thisisverydesirablefromaresearcherperspective.However,therearepracticalmatterstoconsiderregardingdatalifespan.Asdiscussedabove,throttlingmaybenecessarytopreventcostoverrunsinstoragebylimitingdataaccumulation.Additionally,theTLAspecificationneedstofacilitatedataremoval.Thismightbenecessarytolimithistoricaldatatoalife-spanthatisshorterthanthelifespanoftheTLAitself.Protectionofpersonalprivacyisanotherexampleofwhendatadeletionmaybeneeded(seePS4TLA:PrivacySupportfortheTotalLearningArchitectureSpecificationDocument,version0.1formoreinformationonprivacy).AnindividuallearnermaywishtoremovetheirprofileentirelyandleavetheTLAecosystemforreasonsofpersonalprivacy.Tosupportthis,theTLAspecificationwouldneedtoallowdeletionofdatathatisassociatedwiththatindividuallearner.

3.4 DATACENTRICThegoaloftheTLAistoimprovetheexperienceforthelearner,and,thus,improveoveralllearning.OneofthekeyoutcomesofYear1'sdesign-basedresearchwasrealizingtheextenttowhichthisboilsdowntofacilitatingdataexchanges.Ratherthankeepingdatarelevanttoindividuallearnerslockedawayinindividualsystemswhereitisonlyaccessiblefromwithinthesystemthatcreatedit,theTLAaimstofacilitatesharingdataacrosslearningsystems.Byopeningupcommunicationtoallowdatafromoneprovidertobeaccessedbyanotherprovider,amorecompletepictureoflearningemergesthatenablessmartcomponentstoproducebetterbehaviorbecausetheyhavemoreinformationaboutthelearnertoworkwith.Inthissection,wewillexploresomeofthedimensionsrequiredtoencouragedatasharing.

3.4.1 LotsofDataItbecameapparentoverthecourseofYear1'sdesign-basedresearchthattorealizeitsfullpotential,theTLAneedstoincreasetheamountofdatathatTLAparticipantscanshare.AtthebeginningofprototypinginYear1,weputforthaninitial,minimalsetofdatatosharewiththehypothesisthatitwassufficientfortheApril2017FortBraggprototype.Whatwelearnedoverthecourseofconstructingtheprototypewasthatmanymorekindsofdataweredesiredbyparticipantdevelopersthanweinitiallyanticipated.Andfurther,someofthedatawasnotjustdesired,itwasrequiredtomaketheprototypefunctional.SomeofthetypesofadditionaldataincludedmorexAPIverbs,morelearnerprofilefields,morelearnercontext,andmoredetaileddescriptionsofactivitiesandcompetencyalignments.Thetakeawaylessonwasthemoredatawecanshare,thebetter!

AsweexploredthedataexchangesnecessarytomaketheApril2017experimentalprototypefunctional,welearnedthatthedatasharedbytheTLAecosystemcoversmanyfunctionalareas.ItwasdifficultinthepreliminaryTLAspecifications(version0.1)toconveyhowandwhentouseeachtypeofshareddata,inpartbecauseitwasunclearhowtocategorizethedata.Thepreliminarycategorizationattemptwascomponentcentric,withsetsofdataorganizedbythecomponentsthatwouldstorethe


shareddata.Thisapproachprovedsub-optimalfortworeasons.First,therewerealreadytoomanydatamodels;itwasconfusingtotrytokeepthemallstraight.Second,acomponent-centriccategorizationdoesnotscalewellasusecasesareadded.Asdescribedinsection3.1.4,differentusecasesrequiredifferentsetsofinstantiatedcomponents.Therefore,categorizingthedatamodelsaroundause-casecentricaspectwillresultinanexplosionofdatamodelsasmoreusecasesareadded.AnalyzingthetypesofdatathatYear1haselicitedasvaluablehasrevealedthatasimplercategorizationispossible.AllTLAdatacanbedividedintothreecategorizationsthatareapplicableacrossusecases:

1. Learner: Datathatisaboutaspecific,individuallearner2. ScienceofLearning:Datadescribinghowhumanslearn3. Asset:Datadescribingthecharacteristicsofspecificsoftwarecomponents

Therefore,tobeeffective,theTLAspecificationsshouldhaveasmallnumberofdatamodelsthatareeasytounderstand,yetcapableofsharinglargequantitiesofdatasuchasthethreepresentedhere.Toscaleeffectively,theTLAspecificationsshoulduseanon-componentcentricdataorganizationscheme,sothatitisnotnecessarytoaddnewdatamodelseachtimeanewusecaseisidentified.

3.4.2 PlugandPlayAsanticipated,desireformanymoretypesofdatadescribingindividuallearnerswaselicitedthroughoutYear1,rangingfromdemographicsdatatocertificationstopreferences.Whileweanticipateddataaboutthelearnertobeaprimaryareaforexpansion,itwassurprisinghowmuchdataaboutsoftwareisneeded!Forexample,Recommendersoftwarecomponents(henceforthcalledRecommenders)needtoknowabouttheavailableActivitiesinordertomakegoodrecommendations.Itisnotenoughtoknowwhatalearnerhasexperiencedalreadyandwhattheyneedtoimprovein.Tomakeagoodrecommendation,theRecommenderalsoneedstobeabletoconsumedataaboutpotentialActivitiesthelearnercouldperform.Similarly,foraRecommendertopresentaremediationorchallengerecommendation,itmusthaveaccesstorelationshipsbetweencompetenciesandvariousscienceoflearningapproachestosequencingcontent.Assetandscienceoflearningmetadataiscriticalforenablingsmartsoftwarecomponentstoperformtheiralgorithmseffectively.Inadditiontoenablingsmartcomponents,theassetandscienceoflearningmetadataalsohelpsresearchersandcoursecurators.Withthisdataavailable,thesoftwarearchitecturecanallowmultipleunderlyinglearningtheoriestobe"pluggedin"sothatresearcherstestandrefinethem.Similarly,italsosupports"pluggingin"multipletheoriesforsequencingalearner'strajectorythroughavailablecontentsothatcoursecuratorscanchoosewhatbestsuitstheireducationalagendas.Therefore,theTLAspecificationneedstoprovidealocationforspecifyingmetadataaboutsoftwareandscienceoflearningmethodologiestoenable"pluggingin"softwarecomponentsthatprovidedifferingapproachestolearning.

3.4.3 ProducersandConsumersIndependentofthespecifictransportprotocolthatisemployed,ourrequirementsanalysishasrevealedthatallTLAdatasharingcanbedescribedusingaproducer/consumerdataflowmodel.Alltypesofdataeitherusedintheexperimentalprototype,orrequestedforfutureinclusion,havetwopropertiesincommon.First,eachdatumistheoutputofsomecomponent,ordataproducer,intheTLAecosystem.Second,eachdatumproducedcanbevaluableasinputtoanothercomponent,ordataconsumer.Thedataproducerdoesnotnecessarilyneedtoknowwhatthedataconsumerwilldowiththedata.However,forthedataconsumertomakeeffectiveuseoftheproduceddata,someamountofmetadatadescribinghowtoproperlyinterprettheproduceddataishelpful.Forexample,let'sconsideranActivity


thatproducesa"passed"xAPIstatement.Withoutknowingwhichcompetencythe"passed"statementisassociatedwith,itdifficultforanEvidenceMappertoknowhowtointerpretthemeaningof"passed"inrelationtocompetencies.ButiftheActivityalsoprovidesmetadataindicatingthecompetencythatthe"passed"statementisalignedwith,thenanEvidenceMappercanmakeaninferenceaboutthelearner'smasterylevelintheassociatedcompetency.WhiletheEvidenceMapperasthedataconsumerneedsthecompetencymapping,theActivity,ordataproducer,doesnotneedtoknowhowthedatatheypublishwillbeused.Thisishighlydesirable,asitfosterscreativity,encouragingnewdataconsumerstofindnewandnovelwaystousethelearningdatatobenefitthelearner.Tofacilitateaproducer/consumerdataflow,theTLAspecificationsshouldclearlydescribe:

• DataConcept:Whatdataisshared?• DataStructure:Howistheshareddatastructured?• ConceptualAPI:Whatcanbesaidaboutthedatabetweenproducersandconsumers?

ThetermConceptualAPIisusedtodistinguishthelogicaldataexchangethattakesplacefromtheover-the-wiretransportprotocolusedtosendthedata.TheconceptualAPIisconcernedonlywithwhatcanbesaid,nothowtosenditoverthewire.

3.4.4 ExtensibleBecausetheTLAisdesignedtoencouragenewusecases(asdiscussedinsection3.1),itisanticipatedthatthedatathatwillbeexchangedusingtheTLAwillgrowovertime.Howdoesthisimpactthespecification?ItmeanstheTLAspecificationsmustprovideamechanismforaddingnewdatarequiredbynewusecases,withoutrequiringaspecificationmodificationeachtimeanewpieceofdataisdesired.Additionally,wehaveestablishedthatTLAcommunicationbetweenproducersandconsumersiscenteredonexchangingdata.Sonaturally,ifthedataitselfisextensible,thenwhatcanbesaidaboutthedatamustalsobeextensible.Toaccomplishthis,theTLAspecificationshouldspecifydataexchangesinageneralwaythatwillbeflexibleenoughtofacilitateapplyingtheTLAtonewandunexpectedusecaseswithoutrequiringaspecificationmodificationeachtimeanewusecaseisintroduced.

APIspecifications,ingeneral,describedataexchangesandmanipulation.ExistingparadigmsforhowtostructureAPIspecificationsrangefromCRUD(create/read/update/delete)wheretheinterfaceisstrictlylimitedtosimpledatastorageoperationsthatdonotvarywiththedatatypeorusecase,toextensiveAPIsmadebyindividualvendorsthataretightlycoupledtobothusecasesanddatasuchastheWindowsAPI(https://msdn.microsoft.com/en-us/library/windows/desktop/ff818516(v=vs.85).aspx).BecausetheTLAmustaccommodatenewandunexpectedusecases,theTLAspecificationscannotincludeuse-case-specificassumptions.However,thedesign-basedresearchofYear1indicatesthatmorethansimpleCRUDoperationsaredesired.Forexample,let'sconsidertheuseofNuxeointhe2017FortBraggexperiment.Nuxeowasusedasastand-inActivityIndexbecauseitalreadyexistedandcouldberepurposedtoapproximateActivityIndexbehaviorwithoutalargecodinginvestment(valuableasNuxeoisnotintendedtobealong-termsolutionforanActivityIndex).OneofthekeydevelopercomplaintsaboutNuxeoinYear1wasitsclunkyqueryAPI.Thedatawasavailable,anditwaspossibletoretrieve,buttheAPIforretrievingitwassufficientlydifficulttousethatdevelopersavoidedusingit,insteadhand-synchronizingduplicateddatasetsinternally.Therefore,tobeeffective,theTLAspecificationsneedtoprovideadeveloper-friendlyAPIfordataconsumerstousetoacquireandfilterdataaproducercanprovide.Inparticular,somedataconsumerswillwantallthedatathataparticular


producercanprovide,whereasotherdataconsumerswillbemoreselectiveandwillwanttoexchangeonlysmallamountsofdata.

3.4.5 Real-timeDataProcessingandStoredDataHistoryAttheonsetofYear1,itwasnotclearhowmuchofthedataexchangedbetweenproducersandconsumersshouldbestored.Woulditbebettertoprocessthedataontheflyinthelivesystemandnotstoreitatall?The2017FortBraggexperimentwasinstrumentalinidentifyingananswer.Whilethetechnologydoesexisttoprocessbigdatainreal-time,thisdoesnotfullymeettheneedsofourpreliminaryusecase(meta-adaptation;seeTLAUseCases.docxformoreinformation).Duringtheexperiment,severaldatadisplaydashboardswerecreatedthatcoulddisplaylearnerprogressinformationindifferentwaysinreal-time.Thedashboardswerekeptupandrunningliveduringtheexperimentforlongperiodsoftime.Inadditiontotheanticipateduseofseeingalivesnapshotof"rightnow"inthesystem,timelinestylechartsthatthedashboardscreatedalsodisplayedavisualhistory.Thevisualhistoryaspectwasofjustasmuchinteresttoinstructors,learners,andstakeholdersasthemomenttomomentsnapshot.However,becausesomeofthedashboardprototypeswerenotstoringanyhistoryaftertheyexited,assoonasthedashboardwasclosed,thehistoricalviewwasgone.Similarly,ifthedashboardwasnotupandrunningwhenaninterestingeventoccurredinthesystem,thateventcouldnotbevisualizedafterwardifthedataabouttheeventwasnotstored.Thus,oneoutcomeoftheexperimentwasthatTLAspecificationmustsupportstoringdata,orhistory,tosupportofflinedataanalysisneeds.

Withinthesinglemeta-adaptationusecase,theexperimentalprototypealsodemonstratedthattheTLAspecificationneedstosupportreal-timedataexchangesaswellasnon-real-timedataexchange.Forexample,ifthelearnercompletesanActivity,thenthecompletioncouldtriggeraninferenceaboutthelearner'smasteryestimateinarelatedcompetency.Themasteryestimateupdatemight,inturn,causeaRecommendertosuggestthenextmostappropriateActivity.Ideally,thelearnershouldnothavetowaitanoticeableamountoftimeforthissequenceofeventstooccur.TheirperceptionshouldbethattheycompletedoneActivity,andimmediatelyhavearecommendationforthenext.Therefore,theTLAspecificationofwhatthedataisandhowdataproducersandconsumersexchangeitmustbecompatiblewithbothofflineandreal-timeprocessingneeds.

3.4.6 DistributedDataModelsWemustalsoconsiderwherethedatathatisstoredshouldreside.Wedescribedthreelogicaldatamodels(learner,scienceoflearning,asset)insection3.4.1.However,thelogicalcategorizationdoesnotnecessarilyneedtomapone-to-onewithabackingdatabase.Let'sconsidertwoelementsofthelearnerdatamodel:themasterestimatedescribingthelearner'sabilityinaspecificcompetency,andthelearninggoalsthatdescribewhichcompetenciesthelearnerneedstolearn.Logically,botharepartsofthelearnerdatamodel.However,physically,thesedifferentdataelementsmightresideindifferentcomponentsinstantiatedindifferentphysicallocations.TheTLAspecificationshouldsupportadistributeddatamodel,wherethedatacanresideindifferentphysicallocations,butbeaccessedbyalldataconsumersasthoughitwereasingle,globallyshareddatamodel.Thatis,whilethegoalsandmasteryestimatemightbeprovidedbydifferentcomponentsindifferentlocations,theybothcanbeconsideredassub-sectionsofthelogicallearnerdatamodel.ArrangingtheTLAspecificationinsuchawayallowsdifferentproviderstoaddnewaspectstothelearnerdatamodelwithoutrequiringthedatatobetransferredtoaphysicallycentralizeddatastoragelocation.Thisishighlydesirableasitwill


encouragebettercommunitycollaboration.Toachievethisgoal,theTLAspecificationneedstobeabletosupportdescriptivemetadatasothatdataproducerscanspecifywhatpartsofthelogicaldatamodelstheyproduce,anddecidewhetherornottheywillstorethedatatheyproduceforlaterretrieval.

3.4.7 PerformanceMindedScalabilitydemandsthatthespecificationapproachconsiderperformanceaswell.Wehaveconsidereddividingthelogicaldatamodelsphysicallytoallowcollaborationbetweenperformers.Giventhathistoricaldataisdesired,andwillgrowovertime,itislogicalthatsupportingscale-outNAStechniqueswillalsobenecessarytoprovidethestorageneededfortheexpandinghistoricalrecords.Theamountofdatastored,therateofdataaccumulation,andthedemandfordataretrievalfromdatastoresallimpacttheprocessingpowerandstoragespacerequiredtomeetthedemand.Asthedemandgoesup,loadbalancerswillneedtobeintroducedtoensureconsistentperformance.TheTLAspecificationofthedatashouldnotrestrictorprohibitscaleoutandloadbalancingtechniques,norshoulditmandatethem.Insteaditshouldseektobecompatiblewiththelateststateofthearttechniquesinloadbalancinganddatastoragetechniqueswithoutrequiringaspecificationmodificationtoachieveperformancegoals.

Inadditiontothestoragespaceandprocessingpowerrequiredtomanagethestoreddata,networkresourcesmustalsobeconsidered.Asanexample,somedataconsumersmightwishtoaskanLRSforallxAPIstatementsdescribingtheexperiencesofaspecificlearner,whereasothersmightwanttoknowonlythelastxAPIstatementaboutaspecificlearner.ImagineiftheonlywaytoretrieveinformationfromanLRSistoaskforallxAPIstatementsthatexist.Overtime,asthenumberofstoredxAPIstatementsgrows,thesizeoftheresponseto"givemeallxAPIstatements"growsaccordingly.Certainly,thedataconsumercouldretrievethemall,thenfilterthemastheylike.Butdoingthishasaperformanceimpact.Ifthedataconsumerisdoingthefiltering,thenallthedatatobefilteredmustfirstbetransmittedacrossthenetworkfromthedataproducertothedataconsumer.Amongotherperformanceconcerns,thisunnecessarilywastesnetworkbandwidth.Toalleviatethisproblem,theTLAspecificationshoulddescribewhatcanbesaidindataexchangesaboutthedatainawaythatispowerfulenoughtoenablefilteringtohappenatthedataproducer.Thiswillmakeitpossiblefordataconsumerstoaskmoreprecisequestionsaboutthedatasuchas"givemethelastxAPIstatementaboutaspecificlearner"sothattheyarerespectfulofsharednetworkresources.

Additionalperformanceaspectssuchasfaulttoleranceandthrottlinghavealreadybeencoveredinsection3.3.2.TheTLAspecificationofthedatamodelsandtheirAPIsmustbecompatiblewiththeperformancecharacteristicsthatsectiondescribes.

3.4.8 AutomationFriendlyDuringthe2017FortBraggexperiment,oneofthepainpointsencounteredwastheneedtomanuallyentermetadataabouteachActivity,whichincludedmanualentryofthecompetenciesassociatedwiththeactivity.Manuallyenteringthisinformationissubjecttohumanerror,andisverytimeconsuming.Whileitwastractabletoperforminthecontextofasmallexperiment,amanualsolutiondoesnotscaleverywell.Theobservationfromthedesign-basedresearchinYear1isthatifonlyamanualsolutionisavailable,itisverylikelythatthemetadataishighlylikelytobeincomplete.ActivitieswithinsufficientmetadatacannotbeusedeffectivelybyotherTLAparticipants,sincetheremaynotbeenoughmachine-readabledatadescribingthemtodeterminehowtoutilizethem.Toaddressthisdifficulty,theTLAspecificationsneedtoprovideprogrammaticmeansforaddingandmodifyingmetadatainawaythat


makesiteasytoautomatedataentry.Automationcanrangefromsimplyshiftingthedataentrytoadevelopertask(e.g.,allowingsmartcontenttoprogrammaticallyreporttheirowncharacteristics)allthewaytoplugginginsmartalgorithmstopopulatemetadata.TheTLAspecificationshouldbecompatiblewiththiswiderangeofautomationtechniques.

3.4.9 SecuritySimilartothediscussionoftransportsecurityinsection3.1.3,securityofthedataatrestmustalsobeconsidered.Thatis,storedhistory,asdiscussedinsection3.4.5,needstobestoredsafely.Aswithtransportsecurity,theat-restsecurityshouldbedecoupledfromthedescriptionofwhatthedataisintheTLAspecification.DoingsoallowsnewsecuritytechniquestobeincorporatedastheyemergewithoutrequiringachangetotheTLAspecification.

TLA Specification Requirementstlacommunity.com/wp-content/uploads/2018/05/TLA-Specification... ·...

Documents

Transcript of TLA Specification Requirementstlacommunity.com/wp-content/uploads/2018/05/TLA-Specification... ·...