Composing, Reproducing, and Sharing...
Transcript of Composing, Reproducing, and Sharing...
![Page 1: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/1.jpg)
Composing,Reproducing,andSharingSimula5ons
BruceChilders(contact:[email protected];.edu)DebashisGanguly,WilliamC.GarrisonIII,DavidWilkinson,BruceR.Childers,AdamJ.Lee,andDanielMosse’Dept.ofComputerScience,UniversityofPi;sburgh
ModSim2016,Sea/le,Washington,August12,2016
![Page 2: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/2.jpg)
Star5nganewproject…
2
n You’vegotanewidea:PIMencryp?onenginetosupportaccesscontrols
l Accesscontrols,applica5onsusingaccesscontrolsl Memoryac5vityandarchitecturesimulator
n Star5ngpoint(TheIntrospec5veScien5st)l Hmmm.Iwonderwhatsimulatorsexist?l Oh,personYmadethismodel!CanIleverage&reuseitformywork?l Oops.Somethingismissinginpaper.Whatwerethosesimparams???l Yikes!Wait!Myresultsaredifferent???
Considerthis:Natureasked1,576scien?sts...over70%saidthey’dtriedandfailedtoreproduceanothergroup’sexperiments.
--Scien?ficAmerican,May28,2016
![Page 3: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/3.jpg)
Itreallyshouldn’tbethatdifficult...
3
n Modelingandsimula5onhasadvantagesl Frameworks(common?)forbuildingsimula5onsl Implementedmodelsassimula5oncomponentsl Parameters,inputs,testcases,scripts,outputs
n Wehavealltheingredients!!
PortunoDRAMsim2
![Page 4: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/4.jpg)
TheIngredients
4
n Portuno:Accesscontrolsimulator/analyzern SST:HostPortuno&composewithmemorysimulator
l Instan5ateasimulatorusedforexperimentsl Shareandcompose(reuse)simulaConmodelswithinSST
n OCCAM:Create,runandshareexperimentsl Defineworkflowsofdata,tools,resultsforexperimentsl Usesimulators/tools&experiments(w/resultsandvisualiza5ons)l Shareandcompose(reuse)toolsforexperimentworkflowsl RepeatabilityandmodificaConofexperiments
+ =SC16Tutorial:SolvingandSharingthePuzzle,Sunday,
Nov.13
![Page 5: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/5.jpg)
Portuno:AccessControl
5
GrantAccess
IssueKeys
Users
Admin Set of files
Encrypt
Encrypt
Encrypt
![Page 6: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/6.jpg)
Portuno:AccessControl
6
Re-issueKeys
RevokeUser2
Users
Admin Set of files
Re-encrypt
Re-encrypt
• Whatistheoverheadofre-encryp5on?• Whichaccesscontrolsystemisbestfit?
![Page 7: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/7.jpg)
Portuno:AccessControl
7
PortunoSystem1
System3
System2AccessControlSystems
Applica?onPayload
SystemConstraints
Expressiveness
CostMetrics
Suitabilityanalysisofaccesscontrolsystemsforgivenpayloadundersystemconstraints
• Monte-Carlosimulator• Inputscharacterizelikelihoodofindividualbehaviors• Individualinstancesindependentandparallelizable
![Page 8: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/8.jpg)
8
n Open,mul5scale,andscalableparallelexecu5onn Highlymodularframeworkthatisextensiblen APIstomonitor/measurerun-5mesta5s5csn Composeandintegratesimulators/models
èIdealframeworktobuildourco-designsimulatorl PortunoimplementedasaJavaframeworkl IntegratePortunointoSSTl Reuseexis5ngmemorysimulators(DRAMsim2,orourown,HMMsim)
Aparallel,discrete-eventsimulaConframeworkforscalabilityandflexibility
![Page 9: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/9.jpg)
Portunointegratedwith
9
SST::Link
SST::ComponentManagerEventHandler
SST::Component
Worker
EventHandler
SST::EventWorkPacket
JobScheduler
JobWorkers
SerializedParameters
Workloadtracesmappedto
implementa5on
MeasureCostAccrualbasedonCostFunc5onsunderconstraints
![Page 10: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/10.jpg)
ExperimentswithPortunoinSST
10
• Howtodefine,run&shareexperiments?• Howtomaintainprovenance?• Howtolimitburdenofpeerreviewandovercomelimita5onsofpageconstraints?
SSTPortuno
IntegratePortuno&SST
Portuno
![Page 11: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/11.jpg)
11
n Community-supportedexchangeforcura5onofsimula5on&experimentalresultsincomputerarchitectureandMORE!
Federated Group Collaboration • Collaborateonexperiments&tools• Dynamic,interac5veobjects• Leverageexperiments&tools
http://occam.cs.pitt.edu ACI-1535232 CNS-1305220 CCF-1148646
web portal
software services
Digital Resources
dispatch
retrieve
Repository
artifact
benchmark
tutorial
experiment
web portal
software services
Digital Resources
dispatch
retrieve
Repository
artifact
benchmark
tutorial
experiment
web portal
software services
Digital Resources
dispatch
retrieve
Repository
artifact
benchmark
tutorial
experiment
OCCAM
OCCAM OCCAM
Demand Resources
Digital Libraries
![Page 12: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/12.jpg)
Completeco-designenvironment
12
OCCAMSST
FinalIntegraBonSST
SSTPortuno IntegratePortuno
withSST
Portuno
OCCAM
SST
PortunoIntegrateSSTwithinOCCAM
① Create,runandanalyzeresults(throughsimplifiedGUI)② Collaborateandsharetoconductexperiments③ Reusetoolsandexperimentstoaccelerateresearch④ Repeatabilitywithprovenance⑤ EnablesnewpublicaBonmodelsthroughaccess(demo!)
![Page 13: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/13.jpg)
Whatdoesthisgiveus??
13
Composing, Reproducing, and Sharing Simulations
Area: ModSim Methods and Tools
Debashis Ganguly, William C. Garrison III, David Wilkinson,
Bruce R. Childers, Adam Lee, and Daniel Mosse
Department of Computer Science
University of Pittsburgh
Pittsburgh, Pennsylvania USA
Every year, research groups around the world contribute papersand artifacts to the computer science literature. In many areas, sim-ulation and modeling play key roles in bringing about these newcontributions. Simulation is used to test and validate new ideasprior to their implementation, and thus, the artifacts (software, datasets, benchmarks, etc.) used in simulation are fundamental to theempirical valuation of a research hypothesis.
Oftentimes, the primary focus of a paper is on the validation ofa central hypothesis, and the details surrounding the artifacts usedduring this process are sometimes scarce. Many researchers do notintend to build a foolproof software component to share with thecommunity. Artifacts may end up limited in scope or usability, andhidden assumptions may make the artifact difficult (if not impossi-ble) to reuse, extend, or compose. Many artifacts take a tremendousamount of effort to build and validate and, as such, may remain pri-vate to the research groups that invested in developing them in thefirst place. This limits their availability, increases the difficulty ofvalidating claims made in papers based on these artifacts, and limitsthe ability of others to build upon prior work.
Figure 1: State size vs. Number objects
Addressing this situation necessitates sharing and reproducibil-ity1. While this problem cuts across most CS disciplines, the mod-
case study is openly hosted in the OCCAM collaborative repository(http://occam.cs.pitt.edu) and integrated with Sandia’sStructural Simulation Toolkit (http://sst-simulator.org).
Our simulator, Portuno, conducts cost analyses to explore thesuitability of different access control approaches for a given appli-cation workload. Portuno has been used in an array of analyses, in-cluding evaluating group-centric approaches to information sharingand exploring the communication, computation, and administrativeoverheads associated with cryptographic enforcement of role-basedaccess controls (RBAC) on untrusted cloud platforms. Portuno usesprobabilistic actor-based models of user, administrator, and systembehaviors to generate application traces. These abstract traces arethen mapped into traces in concrete access control systems: thosethat are candidates for implementing the application. Costs are thenaggregated over these candidate system traces. Portuno supportsa wide range of design choices in its actor models, initial systemstates, and other parameters of an experiment. As such, openlysharing the choices that have been made and allowing other re-searchers to modify these choices can lead to a better understandingof the trade-offs among different access controls techniques.
To compose Portuno with other simulations, share the infrastruc-ture, and disseminate the experimental outcomes, it is integratedwith SST and incorporated in OCCAM. SST acts as the driver ofthe underlying access control models, which are implemented inJava. This is a novel use of SST as a backbone for probabilisticmodeling in an area other than computer systems simulation. Italso illustrates interoperability between SST and Java models.
The combination of OCCAM, SST, and Portuno leads to a seam-less environment that is more capable than the sum of its parts. Thisintegrated approach offers the capability to quickly define, run, vi-sualize, and share simulation artifacts and results over a huge de-sign space. It supports an end-to-end workflow for modeling andanalyzing access controls under a variety of scenarios, making iteasier to (a) use Portuno for access control analysis, (b) inspectand augment experiments done by others, and (c) modify Portunoin a contained environment. Simulation results are available fromOCCAM in dynamic, interactive analyses, representing the “pa-
OurModSim16PDFabstractGraphaddedwithlinktoexperiment
![Page 14: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/14.jpg)
Observa5onsfromcurrentwork
14
ObservaBonCryptographically-enforcedRBACpoliciesareprohibi5ve,eveninmoderatescale
How to decrease the overheads associated with enforcing cryptographic access controls in cloud environments?
OCCAM
SST
Portuno
![Page 15: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/15.jpg)
Workinprogress
15
n Pushre-encryp5ontothedata(don’tbringbulkdatatotheprocessortore-encrypt)
n Co-designsupportinHybridMemoryCubes(HMC)andtrustedexecu5onenvironmentslikeIntel’sSGX
Portuno GenerateTrace
HMC
Enc.Module
SST
VisualizeResults
Benefit of architectural enhancement to speedup the management of files on untrusted infrastructure.
① Composi5onwithcommonframework② Accelerateresearchbysharing③ Increaseimpactthroughaccess
![Page 16: Composing, Reproducing, and Sharing Simula5onshpc.pnl.gov/modsim/2016/Presentations/BChilders2.pdf · Composing, Reproducing, and Sharing Simulations Area: ModSim Methods and Tools](https://reader031.fdocuments.us/reader031/viewer/2022041012/5ebeccd64ed28e2d46440d01/html5/thumbnails/16.jpg)
16
Por5onsofthismaterialisbasedinpartuponworksupportedbytheNa5onalScienceFounda5onunderGrantNumbersCCF-142331,CNS-1012070,ACI-1535232,CNS-1305220andCCF-1148646.Anyopinions,findings,andconclusionsorrecommenda5onsexpressedinthismaterialarethoseoftheauthor(s)anddonotnecessarilyreflect
theviewsoftheNa5onalScienceFounda5on.
TheEndBruceChilders
UniversityofPi;[email protected];.edu
Severalpiecesofclipartinthispresenta?onarefromopenclipart.org