Post on 28-Jan-2020
FLUENTFastLearningfromUnlabeledEpisodesofNext-GenerationTailoring
TLASpecificationRequirementsContract:W911QY-16-C-0019
Preparedfor:
Dr.SaeSchatz,DirectorAdvancedDistributedLearning(ADL)Initiative
Preparedby:
SoarTech
3600GreenCt.,Ste.600,AnnArbor,MI48105 Phone:734-327-8000•Fax:734-913-8537•Web:www.soartech.com
TLASpecificationRequirementsVersion:0.1.1 Page2of16
TABLEOFCONTENTSTableofContents.........................................................................................................................................2
RevisionHistory............................................................................................................................................3
1 Introduction.........................................................................................................................................3
2 TLASpecificationRequirementsSummary...........................................................................................3
3 Design-BasedResearchRequirementsAnalysis...................................................................................5
3.1 UseCaseInclusiveandForwardThinking....................................................................................5
3.1.1 ManyUsers...........................................................................................................................6
3.1.2 ManyDeploymentPlatforms...............................................................................................6
3.1.3 ManyProtocols.....................................................................................................................6
3.1.4 ManyComponents...............................................................................................................7
3.2 LongevityTechniques...................................................................................................................7
3.3 EncourageCommunityCollaboration..........................................................................................8
3.3.1 BuildOnExistingWork.........................................................................................................8
3.3.2 EasytoUse...........................................................................................................................9
3.4 DataCentric................................................................................................................................11
3.4.1 LotsofData........................................................................................................................11
3.4.2 PlugandPlay......................................................................................................................12
3.4.3 ProducersandConsumers..................................................................................................12
3.4.4 Extensible...........................................................................................................................13
3.4.5 Real-timeDataProcessingandStoredDataHistory..........................................................14
3.4.6 DistributedDataModels....................................................................................................14
3.4.7 PerformanceMinded.........................................................................................................15
3.4.8 AutomationFriendly...........................................................................................................15
3.4.9 Security...............................................................................................................................16
TLASpecificationRequirementsVersion:0.1.1 Page3of16
RevisionHistory
1 INTRODUCTIONTheTLAisanarchitecturetoenablelearningthatisdescribedbyspecifications.SeetheTLADesignDocument.docxforanintroductiontotheTLAarchitecture.ManytermsanddefinitionsareprovidedintheTLADesignDocument.docxaswell,sotheauthorssuggestthatreadersreferencethatdocumentasneeded.TLAspecificationsareformalizeddocumentationthatenablesmultipleproviderstocontributeindividualcomponentstoaTLAecosysteminsuchawaythatallcomponentsinteroperateseamlessly.Specificationsareaimedatlong-termgoals,and,thus,typicallyincludeenoughflexibilitytosupportmanyinstantiationsthatsupportdifferentusecases.TLAdevelopmentisusingadesign-basedresearchapproach;iterativeanalysis,design,development,andimplementationcycleswillshepherdtheinitialseedofanideathroughtomaturation.ThisdocumentdescribestherequirementsthattheTLAspecificationsneedtoaddress,aselicitedbythedesign-basedresearchapproach.TheinitialsetofrequirementsisanoutcomeofanalysisoftheYear1experimentalprototypedemonstratedatFortBragginApril2017.
2 TLASPECIFICATIONREQUIREMENTSSUMMARYThissectionbrieflysummarizesthehigh-levelTLAspecificationrequirementsidentifiedduringYear1oftheTLAdesign-basedresearch.Thesummaryinthissectionispresentedupfrontforreadersdesiringaquickoverview.Foradeeperunderstanding,pleasecontinueontosection3.Foreaseofreference,eachsummarytableinthissectioncorrespondstoamajortopicinsection3.Pleasenotethattheorderinwhichtherequirementsaresummarizedinthissectionreflectstheorderofdiscussionintheanalysissection;orderingheredoesnotimplyrelativeimportance.
SummaryofRelatedRequirements:UseCasesandForwardThinkingDecoupledfromspecificusecasesSupportmanytypesofuserswithoutmodificationeachtimeanewusertypeisidentifiedSupportmanytypesofdeploymentplatformswithoutmodificationeachtimeanewplatformtypeisintroducedSupportmanytypesoftransportprotocolsSupportmanytypesofsecurityprotocolsSupportinclusionofmorethanonetransportprotocolinasingleTLAinstantiation
Name Date ReasonforChange Version
SoarTech 7/18/2017 InitialDraft 0.1.0
SoarTech 7/26/2017 SummaryTable 0.1.1
TLASpecificationRequirementsVersion:0.1.1 Page4of16
SupportinclusionofmorethanonesecurityprotocolinasingleTLAinstantiationDecoupledatadefinitionfromtransportandsecurityprotocoldescriptionDecoupledfromspecific,instantiated,componenttypesSupportaddingnewcomponenttypeswithoutspecificationmodificationEasytomaintainSubdividedintofamiliesofrelatedspecificationstofacilitatemodularityandlongtermmaintenanceEasytoextendandrefineCompatiblewitha"provingground"workflowthatfacilitatesimprovementCompatiblewithbestpracticesthatbridgethegapbetweenlonglivedspecificationsandshort-livedimplementationdetails
SummaryofRelatedRequirements:EncourageCommunityCollaborationLeverageexistingstandardsandspecificationswheneverpossibletomeetrequirementsSupportcombiningexistingspecifications(includingpartsofexistingspecifications)Supportrecordingprovenanceforincluded,pre-existing,specificationsSupportcompatibilitywithmanyexistingspecificationsSupportcompatibilitywithoverlappingexistingspecificationthroughequivalencymappingorothersimilarmeansEncouragecommunitycontributionintheworkflowforextensionsandimprovementsEasytounderstandEasyfordeveloperstouseHumanreadableMachineinterpretableSupportautomatedcomponentdiscoverySupportcomputationalresourcesresidinginthecloud,oronatraditionalorganizationallyownedserverSupportnegotiationofresourcelimitswhennewcomponentsare"pluggedin"SupportcostbasedresourcethrottlingSupportcommunicationofcomponentQoScapabilitiesandneedsSupportfaulttoleranceSupportrespectinglearnerprivacySupportdataremoval
SummaryofRelatedRequirements:DataCentricSupportadatacentricapproachUseasmallnumberofeasilyunderstoodlogicaldatamodelsUseanon-componentcentricdataorganizationparadigmsothataddingusecasesdoesnotresultinanexplosionoflogicaldatamodelsSupportlogicaldatamodelsthatarecapableofsharinglargequantitiesofdataacrossmanyfunctionalareasSupportsharingdataaboutscienceoflearningconceptsandsoftwareassetsinadditiontodataaboutthelearnerexperiencesSupportutilizationofmanyscienceoflearningunderlyingtheoriesSupportutilizationofmanycontentsequencingstrategies
TLASpecificationRequirementsVersion:0.1.1 Page5of16
Supportaproducer/consumerdataflowmodelClearlydescribedataconcept:Whatdataisshared?Clearlydescribedatastructure:Howistheshareddatastructured?ClearlydescribeconceptualAPI:Whatcanbesaidaboutthedatabetweenproducersandconsumers?Decoupledataconcept,datamodelandconceptualAPIfromspecificusecasespecificassumptionsSupportaddingnewdatatypestosupportnewusecaseswithoutrequiringaspecificationmodificationeachtimeanewpieceofdataisidentifiedSupportConceptualAPIflexibleenoughtonottorequireaspecificationmodificationeachtimeanewusecaseisintroducedSupportdeveloperfriendlyAPIforrequestingfiltereddatafromdataproducersSupportstoringdatahistorytosupportnon-real-timedataanalysisSupportbothreal-timedataandnon-real-timedataexchangesSupportmetadatafordataproducerstoadvertisethepartsofthelogicaldatamodelstheyproduceSupportmetadatafordataproducerstoadvertisewhethertheykeepahistoricalrecordofthedatatheyproduceforconsumerstoretrieveatalatertimeSupportadistributeddatamodel,wherethedatacanresideindifferentphysicallocations,butbeaccessedbyalldataconsumersasthoughitwereasingle,globallyshareddatamodelSupportdistributingthelogicaldatamodelsuchthatdataproducersthatcanproviderhistoryarenotrequiredtotransferthedatatoaphysicallycentralizeddatastoragelocationSupportmultiplemanytypesofscaleouttechniqueswithoutrequiringaspecificationmodificationSupportmanytypesofloadbalancingtechniqueswithoutrequiringaspecificationmodificationSupportdatafilteringthatoptimizesnetworkbandwidthuseSupportprogrammaticallyaddingandmodifyingmetadatainawaymakesiteasytoautomatemetadatapopulationSupportmanymetadatapopulationautomationtechniquesDecoupledatadefinitionfromdescriptionsofsecuringatrestdata
3 DESIGN-BASEDRESEARCHREQUIREMENTSANALYSISThissectionprovidesmoredetaileddescriptionsoftherequirementsintroducedinsection2.ItdescribeshoweachrequirementwasidentifiedduringYear1ofTLAdesign-basedresearch.
3.1 USECASEINCLUSIVEANDFORWARDTHINKINGItisn'tenoughtoproducespecificationsthatmeettoday'srequirements;TLAspecificationsshouldberelevantformanyyearstocome.BecausethedesiredlifespanoftheTLAislong(aimedat2025),itisnecessarytoconsiderhowtostayrelevantinthefaceofemerginginstructionalmethodologiesandlearnerneeds.ToensuretheTLAlifespanisnumberedinyearsordecadesratherthanmonths,oneofthecoredesignrequirementsisthatitmustbedesignedsuchthatnewandunexpectedusecasesareencouragedandwelcomed.AssumingthatthenumberofusecaseisnotfixedmeansthattheTLAmustbedesignedtobeflexibleandadaptable.HowdoesthisimpactspecificationsdescribingtheTLA?Themostsignificantimpactisthatassumptionscoupledtospecificusecasescannotbebakedintothespecification.AbstractionsthatareapplicabletomanyusecasesmustbeelicitedandutilizedsothattheTLAspecificationsdonotneedtochangeeachtimeanewusecaseisintroduced.Inthissection,wewillexploresomeofthedimensionsthatrequireabstractiontomaintainflexibilityacrossusecases.
TLASpecificationRequirementsVersion:0.1.1 Page6of16
3.1.1 ManyUsersIfthereisnotafixedusecase,thenitfollowsthattherecanbemanydifferentusers.SomeofthetypesofusersthatYear1'sresearchhasidentifiedinclude:
• Learners• ContentCurators• Researchers• Instructors• Network/IAAdministrators
WhatdoesthismeanfortheTLAspecifications?Sincethereisnosingleuser-facingapplicationthatwillinterfacewiththeTLA,itnotpossibletoenumerateallofthewaysthattheuser-facingapplication(s)willwanttousethedatathattheTLAmakesavailable.Therefore,thespecificationmustbedesignedsuchthatdifferentuser-facingapplicationswithdifferentgoalscanbeconstructedwithoutaspecificationchangewhenanewuser-facingapplicationisneeded.Examplesofsuchapplicationsincludelearningactivitiesandinstructordashboards.
3.1.2 ManyDeploymentPlatformsTomeettheneedsofavarietyofusecases,theTLAspecificationsshouldbeinclusiveinregardtodeploymentplatforms.Weliveinaworldofconstantlychanginghardwareanddevices,andtheconsumerdevicemarketchangesespeciallyrapidly.Tobeeffectiveovertime,theTLAspecificationsmustbecapableofsupportingtheenduser'sdesiredhardwaredeliverydevices.Todaythatmightmeanlaptops,tablets,andcellphones.ButoverthelifetimeoftheTLA,itmustbeextensibletoincludenewplatformsastheyemerge,fordeliverydevicesinsmartcarsorsmarthomeappliances.ToviewananimateddepictionofhowausermightinteractwiththeTLAinthefuture,followthislink:
http://goanimate.com/videos/0EoEFEKlWNxY?utm_source=linkshareandutm_medium=linkshareandutm_campaign=usercontent.
Whenanewdeliverydeviceemerges,itshouldnotrequireaTLAspecificationmodificationtobeginusingitwithaTLAinstantiation.
3.1.3 ManyProtocolsSpecificationsaredesignedtohelpdeveloperscommunicate.Thisoftenrequiresagreatdealofspecificity.Whilespecificityfacilitatesnear-termcommunication,itcanimpedelong-termmaintenance.Ifausecaseistightlydefined,thenassumptionscanbemadethatsimplifyhowtoaddressthelevelofspecificitythatisrequired.Thisisparticularlyprevalentinregardtotransportprotocolsandsecurityprotocols.Forexample,atightlyconstrainedusecasemightbeabletomakethesimplifyingassumption"allsoftwarecommunicationnecessarycanbeaccomplishedusingREST."Whenthisisthecase,thespecificationsforaddressingtheneedsofsuchusecasesareoftentightlycoupledtoaspecifictransportprotocol,suchasREST.Thisconflatesthedescriptionofwhatthedataisandwhatcanbesaidaboutitwiththemethodusedtotransportitacrossthewire.Similarly,securityprotocols,orhowtosecurethedatabeingtransported,canbeconflatedwithwhatthedataisandwhatcanbesaidaboutit.Thisworksreasonablywellforsystemswhoseusecasesarenotexpectedtovaryovertime.However,sincetheTLAisdesignedtowelcomenewandunexpectedusecases,thesekindsofsimplifyingassumptionsdonotsatisfylong-termrequirements.Asanillustration,let'sconsiderthechoiceofRESTasthetransportprotocolfortheYear1experimentalprototypeandinitialdraftoftheTLAspecifications.RESTis
TLASpecificationRequirementsVersion:0.1.1 Page7of16
fundamentallyapullprotocol;whenacomponentwantsinformation,itasksforitdirectly.TheFortBraggexperimentrevealedthatinadditiontopull,pushingisalsoneededtosatisfythemeta-adaptationusecase.Someofthecomponentvendorswouldliketoregisterinterestinaneventhappeningandreceiveanotificationifthateventtakesplace(push).Inparticular,theFortBraggexperimentelicitedtheneedtomixpushandpullprotocolsinthesameimplementation.Tomeettheflexibilityneedsthatdifferentusecasesdemand,thetransportandsecurityprotocolsmustbedecoupledfromthedescriptionofwhatthedataisandwhatcanbesaidaboutthedataintheTLAspecifications.
3.1.4 ManyComponentsDesigninganarchitecturethatcanbeadaptedtodifferentusecasesmeansthatitmustprovidebuildingblocksthatcanbecomposedintomanydifferentinstantiations,whereeachinstantiationistargetedataspecificsetofusecases.Ifthespecificationiscomponent-centric,thenitwouldneedtobemodifiedeachtimeanewtypeofcomponentwasdiscoveredtobenecessarytomeettheneedsofaspecificusecase.InsteadspecificationsshouldsupportflexiblecomponentdescriptionssothatnewcomponenttypesthatconformtotheTLAspecificationscanbeaddedasnecessaryinindividualinstantiations.BasedontheYear1experimentalprototypingandanalysisofavarietyofpotentialfutureusecases,itappearsthatalltypesofcomponentscanbecategorizedas
• TLAConformantApps–User-facingtoolsthatuseTLAdatatomeetanenduserneed• Processors–Softwarecomponentsthatintakedata,transformitinsomeway,andoutput
anothertypeofdata• DataStores–Softwarecomponentsthatstore,butdonotmodifydata
TheTLAspecificationshould,therefore,provideameansforeachofthesegeneralcategoriesofcomponentstoutilizeTLAdata,withoutcouplingthespecificationtoaspecifictypeofcomponentsuchasan"EvidenceMapper."Thatis,theTLAspecificationcannotbetightlycoupledtospecificcomponentnames.
3.2 LONGEVITYTECHNIQUESToachievethevisionofalong-standingTLAspecificationthatiseasytomaintain,severalknowntechniquescanbeemployed.Asdiscussedintheprecedingsections,identifyingabstractionsanddecouplingthespecificationfromnear-termimplementationchoicescanbeverybeneficial.However,inpractice,thespecificationstillmustallowdeveloperstocommunicatewellenoughtoachievefunctionalimplementations.Fortunately,twospecificstrategiesmakethispossible.
1. BestPracticesguidescanbepublishedthatfillinpracticaldetailsthatindicatethebest-knownwaytoimplementaspecificationusingtoday'stechnology.Astechnologieschange,ifthespecificationwaswell-thought-out,itwillnotneedtobemodifiedforeachnewtechnologythatemerges.Instead,thebestpracticescanbeupdatedtodescribehowtoutilizenewlyemergedtechnologiestoimplementtheexistingspecification.
2. Specificationscanbedividedintofamiliesofrelatedspecifications.Implementationsmaythenutilizethesubsetofspecificationsthatbestmeetstheirneeds.Forexample,isolatingthedescriptionofwhatthedataisinonespecificationandprovidingseparatespecificationsthatdescribehowtouseindividualtransportprotocolswiththedatainareferenceddata
TLASpecificationRequirementsVersion:0.1.1 Page8of16
specificationcanallowimplementerstoselectthedatatheyneedwithoutanimpliedtransportprotocolselection.
Lastly,tosucceedatlong-termviability,theTLAspecificationmusthaveameansofinteractingsuccessfullywiththecommunitytoenablerefinementovertime.Todoso,aworkflowmustbedevelopedthatsupportsa"provingground"foridentificationofnewideas,evaluationoftheirmerit,andpromotionofbest-of-breedideasfromtheprovinggroundintotheformalspecifications.
3.3 ENCOURAGECOMMUNITYCOLLABORATIONThegoaloftheTLAistocreateanecosystemthatfosterscollaboration.Insteadofbuildingtrainingsolutionsbycontractingasinglesolutionprovidertobuildthefullproduct,ADLenvisionsanenvironmentwheremanyvendorscanplugincomponentsthatreflecttheirparticularareasofexpertise,therebycreatingalearningsystemthatsurpasseswhatcanbecreatedbyasingleprovider.Tobringthisvisiontofruition,theTLAspecificationsmustbecommunityoriented.Theymustbepubliclyavailableformanycollaboratorstouseandcontributeto.Inthissection,wewillexploresomeofthedimensionsrequiredtoencouragecommunitycollaboration.
3.3.1 BuildOnExistingWorkTheTLAspecificationsshouldnotstartfromscratch.Leveragingexistingstandardsandspecificationswheneverpossibletomeetrequirementsisaprimarygoal.Doingsowillallowustotakeadvantageofyears’worthofworkthecommunityhasputintodevelopingthem.InYear1'sexperimentalprototype,existingstandardswerenotincorporated.Thiswasadeliberate,andtemporary,choice.TherequirementsatthestartofYear1wereinsufficientlydefinedtobematchedtoexistingspecifications.AllowingfreeexplorationofideasduringYear1wasdonemindfully,withtheendgoalofelicitingrequirements.DuringthecreationoftheinitialexperimentalprototypeanditsdemonstrationatFortBragginApril2017,manyrefinementsweremadetoourinitialunderstandingofTLArequirements.ThefirstdraftoftheTLAAPISpecifications(version0.1)wasproducedasabasiccaptureofwhatwasnecessaryatanengineeringleveltoproduceafullyfunctionalTLAinstantiation.ThesepreliminaryAPIshavebeenusedbytherequirementsanalysisprocessasinputandhavebeenveryhelpfulinidentifyingrequirementsthatmayotherwisehavebeenoverlooked.Inputhasalsobeengatheredinotherways,suchasduringthe2016/2017Delphistudy.Wehavenowgatheredenoughinformationtoallowustostepback,analyzewhatwelearned,andwritedowntherequirementsthatwillneedtobeformalizedintospecifications(thisdocument).ThenextstepistoperformagapanalysiswhereweevaluatetheTLAspecificationrequirementsweelicitedoverthecourseofthefirstyearagainstthewidevarietyofexistingspecificationsoutinthecommunity.Agapanalysiswillhelpustounderstandwhichexistingspecificationswecanleverage,andwheretheTLAhasuniquerequirementsnotyetmetbyanyexistingspecification.
Tofosteracommunitycollaboration,theTLAspecificationshouldbeassembledbycombiningexistingstandardsthatmatchTLArequirements.Doingthiseffectivelywillbeachallenge,butthereareseveralgoalsthatshouldguidetheendresult.Whilesomespecificationsmaybeappropriatetoincludewholesale,therewillbeothersthatonlypartiallyoverlaptheTLA'srequirements.Inthecaseofpartialoverlap,amethodologyforincorporatingthepartialmatchmustbeidentified(suchasMetadataApplicationProfiles).Wedonotwanttoinventanewstandardthatoverlapspartofanexistingone.Whenexistingstandardsareincludedwhollyorpartially,provenancemustbetrackedsotheoriginof
TLASpecificationRequirementsVersion:0.1.1 Page9of16
theinclusioncanbetraced.Next,therewillbeplaceswheremorethanoneexistingstandardmeetsaTLArequirement.Wheretherearealreadyexisting,overlappingspecifications,thegoalfortheTLAspecificationshouldbetobuildonourspiritofinclusivity.Ratherthandown-selectingtoasinglematch,everyeffortshouldbemadetomaketheTLAcompatiblewithallmatches.Thismayentailproducinganequivalencymappingthatfacilitatesconvertingterminology(andunderlyingprinciples)fromonespecificationlanguagetoanother.Lastly,astheTLAisforgingnewgroundinlearningsystems,therewillundoubtedlybegapswhereTLArequirementsarenotmetbyanyexistingstandard.Asdiscussedinsection3.2,aprovinggroundwillbeneededwhereemergingideascanbeexplored,evaluated,andpromotedintocommunity-curatedspecifications.ItisimportanttoensurethatnewideasforgedintheTLAdevelopmentlifecyclegivebacktothecommunity,anditisequallyimportantthatthecommunityhasaviableavenueforcontributingtoTLArequirementsgatheringandspecificationdevelopment.
3.3.2 EasytoUseItwillbecriticalforgainingmomentuminthecommunitythattheTLAiseasytounderstandandeasytouse.Ofcourse,whenwesay"use"wethenhavetoidentifytheuser.Inthiscase,theuserisnotthelearner,butthepersonnelnecessaryforcreatingaTLAinstantiation.Thereareseveralusertypesthatneedtobeconsideredincluding:
• Organizationalleveldecisionmakers• Instructorsandcoursecreators• Developers• Network/IAAdministrators
InstantiationofaTLAinstancerequirescollaborationamongalltypesofusers,andtypicallythecontributorswillnotallbepartofthesamecompany/organization.
Therearetwoverybasicprinciplesthatareagoodfirststep.First,theTLAspecificationsshouldbehumanreadableandclearlywrittentobemeaningfultoawidereadingaudience.Thespecificationmustclearlydescribewhatthevaluepropositionisforparticipatingproviders.Itmustalsoclearlydescribehowandwhentousethedifferentpartsofthearchitecture.InYear1,valuablefeedbackwasgatheredfromtheexperimentalprototypedevelopmentteamindicatingthatforActivitydevelopersitwasdifficulttounderstandhowtheoutputcomponentstheydevelopedinfluencedthebehaviorofothercomponentsintheTLA(thatconsumedtheoutputtheyproduced).TheTLASpecificationconstitutesacontractthatfacilitatescomponent-to-componentcommunicationinsoftware,sowewillneedtoimprovethedescriptionofhowinputsandoutputsfunctiontogether.Tomakeiteasytomaketheleapfromadeveloperunderstandingthespecificationtoasoftwaresystemusingthespecification,thespecificationformatshouldalsobemachineinterpretable.Ifacomputercandirectly"read"thespecification,thenthedeveloperhaslessworktodo;thereisnointermediarystepofconvertingahuman-readableformatintoamachine-interpretableformat.
ForaTLAecosystemtobesuccessful,manyActivityproviderswillneedtocontributeindividualactivitiessothatcollectivelytherearemanyavailableforlearnerstobenefitfrom.Imaginehowboringtheinternetwouldbeifindividualsfromallovertheworlddidn'tfeelmotivatedtoaddcontentfortheworldtosee.TheprincipleisthesamefortheTLA:thespecificationshouldmakeiteasyforthosewhohavecontenttoshare,toshareit.Anditisnotonlylearner-facingcontentwewanttoencourageproviderstoshare.Wewantsmartpeopleandorganizationswhohavetheabilitytobuild
TLASpecificationRequirementsVersion:0.1.1 Page10of16
computationalcomponentsthatcanresideinsidetheTLACloudtobeabletocontributethoseofferingswithequalease.ThekeytakeawayfromYear1'sexperimentalprototypewasthat"pluggingin"anewcomponentintoaTLAinstantiationwastoohard/manual/time-consuming.Thisbearssurprisingsimilaritytothe"tyrannyofnumbers"problemfacedduringtheinventionofthemicrochip(seehttps://en.wikipedia.org/wiki/Tyranny_of_numbers).Thebasicpremisewasthatitwaspossibletoenvisionahardwaresystemthatwasmorecomplexthanitwascost-effectivetowiretogetherbyhand.Theeventualsolutionwasautomationleadingtomicrochipsasweknowthemtoday.Weareatapointinhistorywherethisisnowtrueofsoftware:wecanenvisionsystemssignificantlymorecomplexthanitiscost-effectivetobuild,duetothemanualnatureofconnectingsoftwarecomponentstogetherbyhandtomeettheneedsofindividualusecases.TheTLA'sgoalsputitintothiscategory:wewantanecosystemwhereitiseasyto"plugin"componentsandreconfigureaninstantiationtomeettheneedsofnewlydiscoveredusecases.Tomakeiteasyfordeveloperstodothis,newadvancesinautomationwillbeneededtosolvesomeaspectsoftheproblem.Forexample,coursecreatorsareaccustomedtomanuallysequencingcontent,butiftheTLAsucceedstherewillbemillionsofpiecesofavailablecontentforcoursecreatorstosiftthrough.Howcantheydosoeffectively?Theansweriscertainlysomeformofautomation,butwhatkindisnotyetclear.Forareassuchasthis,theTLAspecificationsmustbeflexibleenoughtoallowtheinsertionofsuchananswerwhenitisdiscoveredinfutureexperimentalprototyping.However,someaspectsofsimplifying"pluggingin"alreadyhaveindustry-acceptedsolutions.Onesuchexampleisautomatedcomponentdiscovery.Automatedcomponentdiscoveryallowscomponentsthat"plugin"toregisterthemselveswhenthey"plugin"andthendescribethemselvestoothercomponentsthatarealreadypluggedin.TheTLAspecificationneedstoincludesupportforautomatedcomponentdiscoverytomakeiteasyfordeveloperstocontributecomponentstotheTLAecosystem,andtolocateandutilizecomponentsprovidedbyothercontributors.
Instantiatingasoftwaresystemrequiresresources,computingpower,diskspace,andnetworkconnectivity,allwhichmustbepurchased.TomakeitaneasydecisionfordecisionmakersandITpersonneltoendorsetheTLA,theTLAspecificationsmusthelpalleviatetheircostconcerns.OurSpring2017demonstrationatFortBraggutilizedAmazonWebServices(AWS)foracquiringourcomputingresources.Wesawfirst-handhowsuccessatenticingparticipantstocontributenewandunexpectedcomponents(inthiscasereal-timedashboards),couldleadtoanunexpectedspikeintheusageofAWSresources,whichhasadirectmonetaryimpact.Tofacilitatethebalancebetweencreativityandcost,theTLAspecificationmustencourageimplementationsthataregooddenizens,respectingthecostofcomputingandstorageresources.Thespecificationshouldmakeiteasyforcomponentstopublishtheircomputingresourceneedsandnegotiateforareasonableallowancewhentheyare"pluggedin."Additionally,theTLAspecificationmustallowtheimplementerthechoiceofplacingtheircomputationalresourcesinthecloud(e.g.,AWS)iftheydesire.However,thisshouldnotbemandated;theTLAspecificationshouldalsobecompatiblewithimplementationsthatutilizeamoretraditionalserver-basedinstallationwithinasingleorganization.Inaddition,theTLAspecificationmustenablecost-based,real-timethrottlingsothatITAdministratorscanthrottlecomponentsthatareusingmorethantheirfairshareofresourcestopreventcostoverruns.
BecausetheTLAisintendedtobeinstantiatedandleftrunningforlongperiodsoftime,potentiallymanyyears,performanceandmaintenanceneedsmustbeconsidered.ForasmoothlearnerexperiencetheTLAshouldalwaysappearresponsive.Tomakethiseasytoachieve,theTLAspecificationneedstoincludetheabilityforcomponentstopublishtheirqualityofservice(QoS)requirementsforinputthey
TLASpecificationRequirementsVersion:0.1.1 Page11of16
require,aswellastheQoSlevelstheycansupportfortheoutputtheyproduce.Byallowingcommunicationoftheseimportantaspectsinthespecification,itispossibletoassembleinstantiationsthatmeetbothreal-timeandnon-real-timeusecases.Similarly,componentscanfailovertime.Foracontinuous,smoothend-userexperience,theTLAspecificationmustenablefaulttolerance.Forexample,hotswappingmustbesupportedsothatafailedcomponentcanberecognizedandreplacedwithoutcompromisingsystemavailability.
AnotherramificationofTLAinstantiationsthathavelifespansnumberedinyearsisthatlargequantitiesofdatawillbeaccumulated.Thisisverydesirablefromaresearcherperspective.However,therearepracticalmatterstoconsiderregardingdatalifespan.Asdiscussedabove,throttlingmaybenecessarytopreventcostoverrunsinstoragebylimitingdataaccumulation.Additionally,theTLAspecificationneedstofacilitatedataremoval.Thismightbenecessarytolimithistoricaldatatoalife-spanthatisshorterthanthelifespanoftheTLAitself.Protectionofpersonalprivacyisanotherexampleofwhendatadeletionmaybeneeded(seePS4TLA:PrivacySupportfortheTotalLearningArchitectureSpecificationDocument,version0.1formoreinformationonprivacy).AnindividuallearnermaywishtoremovetheirprofileentirelyandleavetheTLAecosystemforreasonsofpersonalprivacy.Tosupportthis,theTLAspecificationwouldneedtoallowdeletionofdatathatisassociatedwiththatindividuallearner.
3.4 DATACENTRICThegoaloftheTLAistoimprovetheexperienceforthelearner,and,thus,improveoveralllearning.OneofthekeyoutcomesofYear1'sdesign-basedresearchwasrealizingtheextenttowhichthisboilsdowntofacilitatingdataexchanges.Ratherthankeepingdatarelevanttoindividuallearnerslockedawayinindividualsystemswhereitisonlyaccessiblefromwithinthesystemthatcreatedit,theTLAaimstofacilitatesharingdataacrosslearningsystems.Byopeningupcommunicationtoallowdatafromoneprovidertobeaccessedbyanotherprovider,amorecompletepictureoflearningemergesthatenablessmartcomponentstoproducebetterbehaviorbecausetheyhavemoreinformationaboutthelearnertoworkwith.Inthissection,wewillexploresomeofthedimensionsrequiredtoencouragedatasharing.
3.4.1 LotsofDataItbecameapparentoverthecourseofYear1'sdesign-basedresearchthattorealizeitsfullpotential,theTLAneedstoincreasetheamountofdatathatTLAparticipantscanshare.AtthebeginningofprototypinginYear1,weputforthaninitial,minimalsetofdatatosharewiththehypothesisthatitwassufficientfortheApril2017FortBraggprototype.Whatwelearnedoverthecourseofconstructingtheprototypewasthatmanymorekindsofdataweredesiredbyparticipantdevelopersthanweinitiallyanticipated.Andfurther,someofthedatawasnotjustdesired,itwasrequiredtomaketheprototypefunctional.SomeofthetypesofadditionaldataincludedmorexAPIverbs,morelearnerprofilefields,morelearnercontext,andmoredetaileddescriptionsofactivitiesandcompetencyalignments.Thetakeawaylessonwasthemoredatawecanshare,thebetter!
AsweexploredthedataexchangesnecessarytomaketheApril2017experimentalprototypefunctional,welearnedthatthedatasharedbytheTLAecosystemcoversmanyfunctionalareas.ItwasdifficultinthepreliminaryTLAspecifications(version0.1)toconveyhowandwhentouseeachtypeofshareddata,inpartbecauseitwasunclearhowtocategorizethedata.Thepreliminarycategorizationattemptwascomponentcentric,withsetsofdataorganizedbythecomponentsthatwouldstorethe
TLASpecificationRequirementsVersion:0.1.1 Page12of16
shareddata.Thisapproachprovedsub-optimalfortworeasons.First,therewerealreadytoomanydatamodels;itwasconfusingtotrytokeepthemallstraight.Second,acomponent-centriccategorizationdoesnotscalewellasusecasesareadded.Asdescribedinsection3.1.4,differentusecasesrequiredifferentsetsofinstantiatedcomponents.Therefore,categorizingthedatamodelsaroundause-casecentricaspectwillresultinanexplosionofdatamodelsasmoreusecasesareadded.AnalyzingthetypesofdatathatYear1haselicitedasvaluablehasrevealedthatasimplercategorizationispossible.AllTLAdatacanbedividedintothreecategorizationsthatareapplicableacrossusecases:
1. Learner: Datathatisaboutaspecific,individuallearner2. ScienceofLearning:Datadescribinghowhumanslearn3. Asset:Datadescribingthecharacteristicsofspecificsoftwarecomponents
Therefore,tobeeffective,theTLAspecificationsshouldhaveasmallnumberofdatamodelsthatareeasytounderstand,yetcapableofsharinglargequantitiesofdatasuchasthethreepresentedhere.Toscaleeffectively,theTLAspecificationsshoulduseanon-componentcentricdataorganizationscheme,sothatitisnotnecessarytoaddnewdatamodelseachtimeanewusecaseisidentified.
3.4.2 PlugandPlayAsanticipated,desireformanymoretypesofdatadescribingindividuallearnerswaselicitedthroughoutYear1,rangingfromdemographicsdatatocertificationstopreferences.Whileweanticipateddataaboutthelearnertobeaprimaryareaforexpansion,itwassurprisinghowmuchdataaboutsoftwareisneeded!Forexample,Recommendersoftwarecomponents(henceforthcalledRecommenders)needtoknowabouttheavailableActivitiesinordertomakegoodrecommendations.Itisnotenoughtoknowwhatalearnerhasexperiencedalreadyandwhattheyneedtoimprovein.Tomakeagoodrecommendation,theRecommenderalsoneedstobeabletoconsumedataaboutpotentialActivitiesthelearnercouldperform.Similarly,foraRecommendertopresentaremediationorchallengerecommendation,itmusthaveaccesstorelationshipsbetweencompetenciesandvariousscienceoflearningapproachestosequencingcontent.Assetandscienceoflearningmetadataiscriticalforenablingsmartsoftwarecomponentstoperformtheiralgorithmseffectively.Inadditiontoenablingsmartcomponents,theassetandscienceoflearningmetadataalsohelpsresearchersandcoursecurators.Withthisdataavailable,thesoftwarearchitecturecanallowmultipleunderlyinglearningtheoriestobe"pluggedin"sothatresearcherstestandrefinethem.Similarly,italsosupports"pluggingin"multipletheoriesforsequencingalearner'strajectorythroughavailablecontentsothatcoursecuratorscanchoosewhatbestsuitstheireducationalagendas.Therefore,theTLAspecificationneedstoprovidealocationforspecifyingmetadataaboutsoftwareandscienceoflearningmethodologiestoenable"pluggingin"softwarecomponentsthatprovidedifferingapproachestolearning.
3.4.3 ProducersandConsumersIndependentofthespecifictransportprotocolthatisemployed,ourrequirementsanalysishasrevealedthatallTLAdatasharingcanbedescribedusingaproducer/consumerdataflowmodel.Alltypesofdataeitherusedintheexperimentalprototype,orrequestedforfutureinclusion,havetwopropertiesincommon.First,eachdatumistheoutputofsomecomponent,ordataproducer,intheTLAecosystem.Second,eachdatumproducedcanbevaluableasinputtoanothercomponent,ordataconsumer.Thedataproducerdoesnotnecessarilyneedtoknowwhatthedataconsumerwilldowiththedata.However,forthedataconsumertomakeeffectiveuseoftheproduceddata,someamountofmetadatadescribinghowtoproperlyinterprettheproduceddataishelpful.Forexample,let'sconsideranActivity
TLASpecificationRequirementsVersion:0.1.1 Page13of16
thatproducesa"passed"xAPIstatement.Withoutknowingwhichcompetencythe"passed"statementisassociatedwith,itdifficultforanEvidenceMappertoknowhowtointerpretthemeaningof"passed"inrelationtocompetencies.ButiftheActivityalsoprovidesmetadataindicatingthecompetencythatthe"passed"statementisalignedwith,thenanEvidenceMappercanmakeaninferenceaboutthelearner'smasterylevelintheassociatedcompetency.WhiletheEvidenceMapperasthedataconsumerneedsthecompetencymapping,theActivity,ordataproducer,doesnotneedtoknowhowthedatatheypublishwillbeused.Thisishighlydesirable,asitfosterscreativity,encouragingnewdataconsumerstofindnewandnovelwaystousethelearningdatatobenefitthelearner.Tofacilitateaproducer/consumerdataflow,theTLAspecificationsshouldclearlydescribe:
• DataConcept:Whatdataisshared?• DataStructure:Howistheshareddatastructured?• ConceptualAPI:Whatcanbesaidaboutthedatabetweenproducersandconsumers?
ThetermConceptualAPIisusedtodistinguishthelogicaldataexchangethattakesplacefromtheover-the-wiretransportprotocolusedtosendthedata.TheconceptualAPIisconcernedonlywithwhatcanbesaid,nothowtosenditoverthewire.
3.4.4 ExtensibleBecausetheTLAisdesignedtoencouragenewusecases(asdiscussedinsection3.1),itisanticipatedthatthedatathatwillbeexchangedusingtheTLAwillgrowovertime.Howdoesthisimpactthespecification?ItmeanstheTLAspecificationsmustprovideamechanismforaddingnewdatarequiredbynewusecases,withoutrequiringaspecificationmodificationeachtimeanewpieceofdataisdesired.Additionally,wehaveestablishedthatTLAcommunicationbetweenproducersandconsumersiscenteredonexchangingdata.Sonaturally,ifthedataitselfisextensible,thenwhatcanbesaidaboutthedatamustalsobeextensible.Toaccomplishthis,theTLAspecificationshouldspecifydataexchangesinageneralwaythatwillbeflexibleenoughtofacilitateapplyingtheTLAtonewandunexpectedusecaseswithoutrequiringaspecificationmodificationeachtimeanewusecaseisintroduced.
APIspecifications,ingeneral,describedataexchangesandmanipulation.ExistingparadigmsforhowtostructureAPIspecificationsrangefromCRUD(create/read/update/delete)wheretheinterfaceisstrictlylimitedtosimpledatastorageoperationsthatdonotvarywiththedatatypeorusecase,toextensiveAPIsmadebyindividualvendorsthataretightlycoupledtobothusecasesanddatasuchastheWindowsAPI(https://msdn.microsoft.com/en-us/library/windows/desktop/ff818516(v=vs.85).aspx).BecausetheTLAmustaccommodatenewandunexpectedusecases,theTLAspecificationscannotincludeuse-case-specificassumptions.However,thedesign-basedresearchofYear1indicatesthatmorethansimpleCRUDoperationsaredesired.Forexample,let'sconsidertheuseofNuxeointhe2017FortBraggexperiment.Nuxeowasusedasastand-inActivityIndexbecauseitalreadyexistedandcouldberepurposedtoapproximateActivityIndexbehaviorwithoutalargecodinginvestment(valuableasNuxeoisnotintendedtobealong-termsolutionforanActivityIndex).OneofthekeydevelopercomplaintsaboutNuxeoinYear1wasitsclunkyqueryAPI.Thedatawasavailable,anditwaspossibletoretrieve,buttheAPIforretrievingitwassufficientlydifficulttousethatdevelopersavoidedusingit,insteadhand-synchronizingduplicateddatasetsinternally.Therefore,tobeeffective,theTLAspecificationsneedtoprovideadeveloper-friendlyAPIfordataconsumerstousetoacquireandfilterdataaproducercanprovide.Inparticular,somedataconsumerswillwantallthedatathataparticular
TLASpecificationRequirementsVersion:0.1.1 Page14of16
producercanprovide,whereasotherdataconsumerswillbemoreselectiveandwillwanttoexchangeonlysmallamountsofdata.
3.4.5 Real-timeDataProcessingandStoredDataHistoryAttheonsetofYear1,itwasnotclearhowmuchofthedataexchangedbetweenproducersandconsumersshouldbestored.Woulditbebettertoprocessthedataontheflyinthelivesystemandnotstoreitatall?The2017FortBraggexperimentwasinstrumentalinidentifyingananswer.Whilethetechnologydoesexisttoprocessbigdatainreal-time,thisdoesnotfullymeettheneedsofourpreliminaryusecase(meta-adaptation;seeTLAUseCases.docxformoreinformation).Duringtheexperiment,severaldatadisplaydashboardswerecreatedthatcoulddisplaylearnerprogressinformationindifferentwaysinreal-time.Thedashboardswerekeptupandrunningliveduringtheexperimentforlongperiodsoftime.Inadditiontotheanticipateduseofseeingalivesnapshotof"rightnow"inthesystem,timelinestylechartsthatthedashboardscreatedalsodisplayedavisualhistory.Thevisualhistoryaspectwasofjustasmuchinteresttoinstructors,learners,andstakeholdersasthemomenttomomentsnapshot.However,becausesomeofthedashboardprototypeswerenotstoringanyhistoryaftertheyexited,assoonasthedashboardwasclosed,thehistoricalviewwasgone.Similarly,ifthedashboardwasnotupandrunningwhenaninterestingeventoccurredinthesystem,thateventcouldnotbevisualizedafterwardifthedataabouttheeventwasnotstored.Thus,oneoutcomeoftheexperimentwasthatTLAspecificationmustsupportstoringdata,orhistory,tosupportofflinedataanalysisneeds.
Withinthesinglemeta-adaptationusecase,theexperimentalprototypealsodemonstratedthattheTLAspecificationneedstosupportreal-timedataexchangesaswellasnon-real-timedataexchange.Forexample,ifthelearnercompletesanActivity,thenthecompletioncouldtriggeraninferenceaboutthelearner'smasteryestimateinarelatedcompetency.Themasteryestimateupdatemight,inturn,causeaRecommendertosuggestthenextmostappropriateActivity.Ideally,thelearnershouldnothavetowaitanoticeableamountoftimeforthissequenceofeventstooccur.TheirperceptionshouldbethattheycompletedoneActivity,andimmediatelyhavearecommendationforthenext.Therefore,theTLAspecificationofwhatthedataisandhowdataproducersandconsumersexchangeitmustbecompatiblewithbothofflineandreal-timeprocessingneeds.
3.4.6 DistributedDataModelsWemustalsoconsiderwherethedatathatisstoredshouldreside.Wedescribedthreelogicaldatamodels(learner,scienceoflearning,asset)insection3.4.1.However,thelogicalcategorizationdoesnotnecessarilyneedtomapone-to-onewithabackingdatabase.Let'sconsidertwoelementsofthelearnerdatamodel:themasterestimatedescribingthelearner'sabilityinaspecificcompetency,andthelearninggoalsthatdescribewhichcompetenciesthelearnerneedstolearn.Logically,botharepartsofthelearnerdatamodel.However,physically,thesedifferentdataelementsmightresideindifferentcomponentsinstantiatedindifferentphysicallocations.TheTLAspecificationshouldsupportadistributeddatamodel,wherethedatacanresideindifferentphysicallocations,butbeaccessedbyalldataconsumersasthoughitwereasingle,globallyshareddatamodel.Thatis,whilethegoalsandmasteryestimatemightbeprovidedbydifferentcomponentsindifferentlocations,theybothcanbeconsideredassub-sectionsofthelogicallearnerdatamodel.ArrangingtheTLAspecificationinsuchawayallowsdifferentproviderstoaddnewaspectstothelearnerdatamodelwithoutrequiringthedatatobetransferredtoaphysicallycentralizeddatastoragelocation.Thisishighlydesirableasitwill
TLASpecificationRequirementsVersion:0.1.1 Page15of16
encouragebettercommunitycollaboration.Toachievethisgoal,theTLAspecificationneedstobeabletosupportdescriptivemetadatasothatdataproducerscanspecifywhatpartsofthelogicaldatamodelstheyproduce,anddecidewhetherornottheywillstorethedatatheyproduceforlaterretrieval.
3.4.7 PerformanceMindedScalabilitydemandsthatthespecificationapproachconsiderperformanceaswell.Wehaveconsidereddividingthelogicaldatamodelsphysicallytoallowcollaborationbetweenperformers.Giventhathistoricaldataisdesired,andwillgrowovertime,itislogicalthatsupportingscale-outNAStechniqueswillalsobenecessarytoprovidethestorageneededfortheexpandinghistoricalrecords.Theamountofdatastored,therateofdataaccumulation,andthedemandfordataretrievalfromdatastoresallimpacttheprocessingpowerandstoragespacerequiredtomeetthedemand.Asthedemandgoesup,loadbalancerswillneedtobeintroducedtoensureconsistentperformance.TheTLAspecificationofthedatashouldnotrestrictorprohibitscaleoutandloadbalancingtechniques,norshoulditmandatethem.Insteaditshouldseektobecompatiblewiththelateststateofthearttechniquesinloadbalancinganddatastoragetechniqueswithoutrequiringaspecificationmodificationtoachieveperformancegoals.
Inadditiontothestoragespaceandprocessingpowerrequiredtomanagethestoreddata,networkresourcesmustalsobeconsidered.Asanexample,somedataconsumersmightwishtoaskanLRSforallxAPIstatementsdescribingtheexperiencesofaspecificlearner,whereasothersmightwanttoknowonlythelastxAPIstatementaboutaspecificlearner.ImagineiftheonlywaytoretrieveinformationfromanLRSistoaskforallxAPIstatementsthatexist.Overtime,asthenumberofstoredxAPIstatementsgrows,thesizeoftheresponseto"givemeallxAPIstatements"growsaccordingly.Certainly,thedataconsumercouldretrievethemall,thenfilterthemastheylike.Butdoingthishasaperformanceimpact.Ifthedataconsumerisdoingthefiltering,thenallthedatatobefilteredmustfirstbetransmittedacrossthenetworkfromthedataproducertothedataconsumer.Amongotherperformanceconcerns,thisunnecessarilywastesnetworkbandwidth.Toalleviatethisproblem,theTLAspecificationshoulddescribewhatcanbesaidindataexchangesaboutthedatainawaythatispowerfulenoughtoenablefilteringtohappenatthedataproducer.Thiswillmakeitpossiblefordataconsumerstoaskmoreprecisequestionsaboutthedatasuchas"givemethelastxAPIstatementaboutaspecificlearner"sothattheyarerespectfulofsharednetworkresources.
Additionalperformanceaspectssuchasfaulttoleranceandthrottlinghavealreadybeencoveredinsection3.3.2.TheTLAspecificationofthedatamodelsandtheirAPIsmustbecompatiblewiththeperformancecharacteristicsthatsectiondescribes.
3.4.8 AutomationFriendlyDuringthe2017FortBraggexperiment,oneofthepainpointsencounteredwastheneedtomanuallyentermetadataabouteachActivity,whichincludedmanualentryofthecompetenciesassociatedwiththeactivity.Manuallyenteringthisinformationissubjecttohumanerror,andisverytimeconsuming.Whileitwastractabletoperforminthecontextofasmallexperiment,amanualsolutiondoesnotscaleverywell.Theobservationfromthedesign-basedresearchinYear1isthatifonlyamanualsolutionisavailable,itisverylikelythatthemetadataishighlylikelytobeincomplete.ActivitieswithinsufficientmetadatacannotbeusedeffectivelybyotherTLAparticipants,sincetheremaynotbeenoughmachine-readabledatadescribingthemtodeterminehowtoutilizethem.Toaddressthisdifficulty,theTLAspecificationsneedtoprovideprogrammaticmeansforaddingandmodifyingmetadatainawaythat
TLASpecificationRequirementsVersion:0.1.1 Page16of16
makesiteasytoautomatedataentry.Automationcanrangefromsimplyshiftingthedataentrytoadevelopertask(e.g.,allowingsmartcontenttoprogrammaticallyreporttheirowncharacteristics)allthewaytoplugginginsmartalgorithmstopopulatemetadata.TheTLAspecificationshouldbecompatiblewiththiswiderangeofautomationtechniques.
3.4.9 SecuritySimilartothediscussionoftransportsecurityinsection3.1.3,securityofthedataatrestmustalsobeconsidered.Thatis,storedhistory,asdiscussedinsection3.4.5,needstobestoredsafely.Aswithtransportsecurity,theat-restsecurityshouldbedecoupledfromthedescriptionofwhatthedataisintheTLAspecification.DoingsoallowsnewsecuritytechniquestobeincorporatedastheyemergewithoutrequiringachangetotheTLAspecification.