: Executable Extensions of the Bookshelf
description
Transcript of : Executable Extensions of the Bookshelf
![Page 1: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/1.jpg)
: Executable Extensions of the Bookshelf
Igor Markov
University of Michigan, EECS
DARPADARPA
![Page 2: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/2.jpg)
Outline
• A three-slide version of the talk– motivations + proposal + how it will help
• Basic use models– users and interfaces
– restrictions
• Existing VLSI CAD Bookshelf• Efforts related to our proposal• Details of the proposal• Sample flows, screenshots and conclusions
![Page 3: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/3.jpg)
Motivation
• Experiences from education– e.g., undergraduate courses
on algorithms and architecture– infrastructure for evaluation: auto-graders
• Infrastructure for collaborative research– can also benefit from automation– must support sharing, modularity and reuse – must scale, must be industry-compatible
• Modularity in implementation platforms
![Page 4: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/4.jpg)
Bookshelf.exe• Dynamic version of the [existing] Bookshelf
– Implementations+benchmarks = algo evaluations?
– Flow composition and high-level scripting
• Related efforts– SatEx, PUNCH, NEOS, OmniFlow
• Proposed solution: ``Bookshelf.exe’’ (bX)– Application Service Provider
– Interfaces
• Online reporting of simulation results
• Support for optimization-specific concepts
• Power versus ease-of-use and modularity
![Page 5: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/5.jpg)
Why We Need Bookshelf.exe?• Design flow prototyping
• SW maintenance: automation (cf. SourceForge)
• HW/SW incompatibilities, lack of CPU cycles– Standardization, consolidation and sharing
• Many published results cannot be reproduced– Simplified sharing and CAD IP re-use
• Lack of high-level, large-scale experiments– Web-based scripting, distributed execution– Open-source flows (with or w/o o.-s. components)
![Page 6: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/6.jpg)
![Page 7: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/7.jpg)
Basic Use Models
• Users (over the Web)– “anonymous” – registered (more features)
• User interfaces– HTML forms (including downloads, uploads,
high-level scripting and job control)– email notification– XML-RPC for contributed remote CPUs and data– NFS (where available) for file sharing
![Page 8: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/8.jpg)
Restrictions
• Uploaded files must match existing formats– Interface for adding new formats
• Execution in a “sandbox”– Cannot make network connections– Visible file-system restricted by the chroot() call– Configurable resource limits (memory, etc)
• Access permissions– Registered users can keep their results private– Benchmarks or solvers behind firewalls
![Page 9: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/9.jpg)
The VLSI CAD Bookshelf• Contents fairly popular: downloads & contributions
– benchmarks, implementations and comparisons– algorithm descriptions and analyses – evaluation methodologies (in English)
• New slots, benchmarks, codes added regularly• ICCAD, DAC, ISPD publications use the
bookshelf• IEEE Design and Test, May/June 2002• A static collection
– Manual addition of material– No automatic evaluation, reporting of results
![Page 10: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/10.jpg)
Other Efforts Related to “.exe”• SatEx http://www.lri.fr/~simon/satex/satex.php3
– Specialized in satisfiability problems• PUNCH http://punch.ece.purdue.edu
– Very broad selection of software (from StarOffice to Capo)– Local to Purdue
• NEOS http://www-neos.mcs.anl.gov/neos– Open-source, distributed architecture– Used primarily for linear and non-linear optimization
• OmniFlow: DAC 2001, Brglez and Lavana– http://www.cbl.ncsu.edu/OpenProjects/OmniFlow– Distributed Collaborative Design Framework for VLSI– GUI-based flow control, chaining of design tools
![Page 11: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/11.jpg)
SatEx• Continual evaluation and ranking of codes
– Results produced and posted automatically– Intuitive interface
• Popular– 93,707 hits March, 2000 – September 2001– 23 SAT provers, 32,610 runs – September 2001
• Limited scalability– One workstation (2yrs of CPU time)– Specialized to one application
![Page 12: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/12.jpg)
• Very general execution framework– From VLSI CAD to GUI-based office applications– Custom-designed file-system (Purdue hosts only)– 241,458 runs in 5 years (8,152 in VLSI CAD )– 20+ publications
• Only maintainer can add executables• No support for eval’n and chaining (flows)
– No stats for results of runs (cf. SatEx top 20)– No MIME-like data types– Difficult to use when multiple tools are involved
![Page 13: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/13.jpg)
• Open-source, distributed framework• Wide use, solid code base• Adding new implementations requires
maintainer intervention (< on PUNCH)– Each new code must come with a host– Distributed maintenance
• Loose data typing– No type system for data and implementations
• Compare to MIME
![Page 14: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/14.jpg)
NEOS: what can be improved
• Independent eval. and verification of results– e.g., PUNCH offers a WL-eval. from the bookshelf
• Real-time on-line reporting of results + stats
• High-level scripting and flow design– Scripts to control the execution and evaluation flows
• Pairing solvers with benchmarks– SatEx-like evaluation, but for multiple data types
![Page 15: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/15.jpg)
OmniFlow
• Context: collaborative VLSI Design– sharing computational resources, but not results
• Distributed over multiple hosts• Provides GUI-based flow control
– supports chaining of design tools– several hard-coded conditions for flow control– no support for execution conditional on results– no scripting language; limited by GUI
• Cannot dynamically add hosts
![Page 16: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/16.jpg)
Bookshelf.exe (1)
• Best of SatEx, PUNCH, NEOS and OmniFlow– Reporting style similar to SatEx (+ alternatives)– The versatility of PUNCH– Scalability and distributed nature of NEOS (or better)– Flow control as in OmniFlow or better
• New features– MIME-like data types and optimization-specific concepts – Automatic submission of binaries and source code– Chaining of implementations; scripting for flow control– Support for use models with proprietary data or code
![Page 17: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/17.jpg)
Bookshelf.exe (2)
• Scalability– Computation is distributed (unlike in SatEx)– Maintenance is automated (unlike in NEOS)
• Support for multiple use models– “adapts to users”– Multiple levels of expertise– Multiple levels of commitment – Sharing of public data– Hiding/protection of proprietary data– “Screen-saver” mode, cf. SETI@Home, Entropia, etc
![Page 18: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/18.jpg)
Sample Scenario (IWLS 2002 Focus Group 3)
• Question: is it possible to massage the logic of the netlist to improve routing congestion?
• Proposed research infrastructure: – IWLS benchmark API (Andreas Kuehlman)– Interface to Bookshelf formats– Layout generation (available in the Bookshelf)– Placement (several placers available in the B.)– Congestion maps (next slide)
![Page 19: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/19.jpg)
![Page 20: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/20.jpg)
Interface Issues• Transparent error diagnostics
– Greatly improve learning curve• Ownership, privacy, resource limits:
sample policy questions– Chaining jobs owned by different users– What jobs can be launched anonymously?
• Script composer versus programming– Flexibility versus learning curve– GUI implemented using HTML forms
(converts clicks and fill-in-the-blanks to scripts)
![Page 21: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/21.jpg)
Script Composer
• Built on top of a scripting language– Based on PERL (PERL functions available)
• Basic flows designed with HTML forms– Start with one ‘step’ and add more steps– API funcs: e.g.,“run ‘optimizer 2’ + store results”– Support for conditionals and iteration
• Scripts sent to back-end for execution as jobs
• Scripts can be saved, posted, reused
![Page 22: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/22.jpg)
Language-Level Support
• Type system for submissions, resultsand intermediate data– Algorithm implementations
• Deterministic and randomized optimizers, etc.
– Input data, results, status info (runtime, memory,…)– Common benchmarks
• Rules for matching submission types– E.g., match a placer with a LEF/DEF benchmark– Violations are reported to user as fatal errors
![Page 23: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/23.jpg)
Data Models
• Consistent data models needed for serious data flows and high-level experiments– e.g., integrated RTL-to-layout implementation
• Plan to use OpenAccess 2.0– Specs published in April 2002– Implementation and source expected next year
• Adjustments within bookshelf expectedin terms of open-source design flows– E.g., for industry SP&R integration
![Page 24: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/24.jpg)
bX: Structure
• bX front-end: mostly Web-based (+ email)• bX back-end
– Main server (job scheduling, reporting of results, etc)– Client software on computational hosts
• Network communications: XML RPC– RPC standard – XML data encoding– HTTP network transport – Compatible with C/C++, Perl, Python, etc.
![Page 25: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/25.jpg)
Implementation Status• So far main focus on the back-end• Back-end ver 0.1 functional on Linux
– BX state maintained in a database– Persistence, etc– Simple one-job demo (1 binary & 1 benchmark)
• Security features and basic policies– Sandbox execution and data type checks
• Front-end supports one-job demo• Next mile-stone: “10X10” demo (cf SAT 2002)
– Jobs automatically distributed and results posted
![Page 26: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/26.jpg)
![Page 27: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/27.jpg)
![Page 28: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/28.jpg)
![Page 29: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/29.jpg)
![Page 30: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/30.jpg)
![Page 31: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/31.jpg)
Conclusions• Bookshelf: popular, but can be improved• Bookshelf.exe: executable extensions• Goals
– Automate routine operations– Create open-source flows– Facilitate high-level, large-scale experimentation
• We plan to assimilate best features from related works + add new ones
• Started bottom-up implementation– Basic version of bX is working
![Page 32: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/32.jpg)
If we missed anything important, let us know
Thank you
![Page 33: : Executable Extensions of the Bookshelf](https://reader030.fdocuments.us/reader030/viewer/2022020801/5681481f550346895db54a0b/html5/thumbnails/33.jpg)