The Jakarta Tool Suite
description
Transcript of The Jakarta Tool Suite
![Page 1: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/1.jpg)
1
The Jakarta Tool Suite
Don BatoryDepartment of Computer Sciences
University of Texas at Austin
![Page 2: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/2.jpg)
2
Introduction Central problems in Software Engineering
stem from one fact:
Complaints about the lack of:– quality, performance, reliability,
maintainability, evolvability
software is written by hand
![Page 3: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/3.jpg)
3
Quality of Software Function of:
– basic design» “build it several times to get it right”» designs improved with experience
– programming staff
How can we do better?– to exploit previous experiences?– to improve quality of programmers?
![Page 4: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/4.jpg)
4
Future of Software Engineering
Automated production of rote software» major improvements in: performance, reliability,
quality…
Domain-specific component technologies» engineering approach that standardizes:
designs (programming problems) implementations (programming solutions)
![Page 5: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/5.jpg)
5
Generators Generators are tools that:
– transform component compositions into code– validate compositions automatically– automatic customizations and optimizations
Software quality improved– use good designs, expert programmers– expect good performance & reliability,
easier maintenance, extensibility
![Page 6: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/6.jpg)
6
What are Generators? Generators are compilers for:
– domain-specific languages – general-purpose programming languages with
domain-specific extensions
Problem:– generators are hard to build– programming infrastructure is lacking– 60% of effort focuses on infrastructure
![Page 7: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/7.jpg)
7
The Jakarta Tool Suite (JTS)– for building component technologies and
generators in Java» based on experiences in building multiple generators» compiler tools (POPART, Dialect, TXL), MS’s IP
– for language extensibility» creates preprocessors for domain-specific languages
(DSLs) or embedded DSLs
– for architectural optimizations needed for self-adaptive software
![Page 8: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/8.jpg)
8
Language Extensibility isn’t new...– LISP macros & CLOS meta-object protocols
– Language extensibility of JTS based on different packaging architectural extensibility
» same techniques used in avionics, network protocols, database systems were applied to programming languages
» unit of extensibility is a “layer” or “building block” of a family of programming languages
![Page 9: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/9.jpg)
9
This Presentation...– Jak Language
open, extensible version of Java its basic features
– Bali Generator of customized dialects of Jak (product-line of dialects of Java)
– Adaptive Software demo
– Status Report and Conclusions
![Page 10: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/10.jpg)
10
open, extensible superset of Javaexamine “base” extension
The Jak Language
![Page 11: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/11.jpg)
11
Support for Metaprogramming Extends Java with:
– AST constructors» represent and manipulate programs as data» adds LISP quote/unquote to Java
– AST traversals, manipulation» arbitrary program transformations
– Generation scoping» variant of hygienic macros
![Page 12: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/12.jpg)
12
AST Constructors Added constructors for code fragments of
different types (expression, statement, …)» similar to LISP quote
AST_Exp e = exp{ 7 + x*8 }exp;AST_Stm s = stm{ if (y>4) return r; }stm;
e.unparse( ); // outputs “7 + x*8”s.unparse( ); // outputs “if (y>4) return r;”
![Page 13: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/13.jpg)
13
Parameterization and Composition of Code Fragments
Explicit escapes» LISP unquote
AST_Exp e = exp{ 7 + x*8 }exp;
AST_Exp s = stm{ if (($exp(e))>4) return r; }stm;
s.unparse( ): // outputs:
“ if ((7+x*8)>4) return r; ”
![Page 14: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/14.jpg)
14
AST Constructors (Continued) Focusing on AST constructors for Java
(or extended Java)
Add AST constructors for other languages– CORBA IDL– Embedded SQL– (subsets of) C, C++– ...
![Page 15: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/15.jpg)
15
The Binding Problem
macro(x) { int temp = 2*x; … } Macro Definition
int temp = 5;macro(temp); Application Code
Bogus GeneratedCode
int temp = 5;{ int temp = 2*temp; … }
int temp = 5;{ int temp = 2*temp; … }
Hygienic, lexically scoped macros (HLSM)
![Page 16: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/16.jpg)
16
Generation Scoping Generalization of HLSM for generators
» Y. Smaragdakis added to Microsoft’s IP» validated with DiSTiL» convenient solution to “binding problem”
Different than HLSM» generators are not reflective, macro-based» GS allows you to maintain the name-space of the
generated program» generator language different than generated language
![Page 17: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/17.jpg)
17
AST Traversals Package/classes for depth-first searches,
manipulations» support arbitrary program transformations
AstCursor c = new AstCursor( );Ast_Node root = // root of AST to search
for (c.First(root); c.More( ); c.PlusPlus( ) ) if (c.node instanceof AstInterface) c.Delete( );
![Page 18: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/18.jpg)
18
How Jak Works
![Page 19: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/19.jpg)
19
SSTs versus ASTs Surface syntax tree (SST) is a syntactically
correct parse tree» tree constructors, modifiers guarantee syntactic
correctness» SSTs may not be semantically correct
Abstract syntax tree (AST) is a type-checked SST
» with annotations to symbol table» invoked by typecheck( ) method to root of SST
![Page 20: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/20.jpg)
20
Extensibility of SST/ASTs “Intentions”
– add AST nodes with domain-specific semantics
» P3 adds cursor, container types to Java
– at reduction time, intention nodes are replaced with their Java (or host language) implementations
» P3 replaces cursor, container AST nodes with their concrete Java implementations
![Page 21: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/21.jpg)
21
Jakarta Follows DSL Paradigm
domainspecificprogram
lexer and
parserAST
transformprogram
JavaprogramJak
Written in Jak
![Page 22: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/22.jpg)
22
More Detail: Processing in Phases
Parser creates SST of a domain-specific program
Jak type checks SSTto convert to AST
intentions
Jak replaces intentionswith SSTs
If necessary, SSTs can be type checked to form ASTs
![Page 23: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/23.jpg)
23
Big Picture JTS/Jak is technology marriage of:
– Java– metaprogramming concepts
» quote, unquote, generation scoping (HLSM)– intention-based programming and
transformation systems
Inherently “open” compiler
![Page 24: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/24.jpg)
24
How to... Produce lexers and parsers?
Produce transform program?
Where does GenVoca fit in?
answers in remainder of talk...
![Page 25: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/25.jpg)
25
Bali
GenVoca Generator of Precompilers of Java Dialects
![Page 26: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/26.jpg)
26
Family Languages Jak is a family of related languages
» versions with/without constructors » with/without P3 extensions» with/without CORBA IDL extensions…
Classic library scalability problem» n optional, largely orthogonal features» 2n different possible combinations of features» cannot build all 2n combinations by hand» can generate them
![Page 27: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/27.jpg)
27
Bali is a GenVoca Generator Bali assembles variants of Jak from
components Components encapsulate primitive
extensions » generation scoping» domain-specific generators like P3» CORBA IDL» …
Compositions of components specifies particular variant
![Page 28: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/28.jpg)
28
Two Aspects Tool for writing compilers for
Domain-Specific Languages (DSL)– looks similar to other DSL-compiler tools– specify syntax of DSL or language extension– annotated, extended BNF grammars
GenVoca generator– software component technology– architectural, extensibility aspects
![Page 29: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/29.jpg)
29
Domain-Specific Language Compiler Tool
![Page 30: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/30.jpg)
30
Bali Grammars Extended BNF grammar
» extended to handle repetitions» e.g., POPART
StatementList : ( Statement )+ ;
ArgumentList : Argument ( ‘,’ Argument )* ;
![Page 31: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/31.jpg)
31
Bali Grammars Annotated
» by class to instantiate when production is recognized» POPART, DIALECT, common OO design techniques
SelectionStmt : IF ‘(’ Expr ‘)’ Statement :: IfStm | SWITCH ‘(’ Expr ‘)’ Block :: SwitchStm ;
constructors: IfStm( token, token, ASTexp, token, ASTstm ) SwitchStm( token, token, ASTexp, token, ASTblk )
![Page 32: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/32.jpg)
32
Inheritance Hierarchies is deduced from DSL grammar
Rule1 : pattern1 :: C1 | Rule2 ;
C1
Rule1
Rule2
Rule2 : pattern2 :: C2 | pattern3 :: C3 ; C2 C3
![Page 33: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/33.jpg)
33
Bali Specifications Jak grammar
– 170 tokens– 400 productions– 800 lines
Bali grammar– 15 tokens– 20 productions– 73 lines
// bali spec
lexical patterns
%%
grammar rules
![Page 34: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/34.jpg)
34
What can be generated... Lexical analyzer
» Jlex (java version of lex)» Bernie Lofaso (Jakarta Team Member)
Parser» Java_Cup (java version of yacc)» Scott Hudson@Georgia Tech
Inheritance hierarchy and AST classes» constructors, unparsing, editing methods
Now movingto JavaCC
![Page 35: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/35.jpg)
35
What can’t be generated... Type checking, reduction, optimization
methods– AST node specific– hand code as subclasses to Bali-generated class
AstNode JTS kernel class
IfStm’ SwitchStm’ … Bali-generated classes
IfStm SwitchStm … Hand-coded subclasses
![Page 36: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/36.jpg)
36
Big Picture Traditional approach to building compilers
for domain-specific languages clean model of proven technology
Problem: don’t want to hand-code and hand-assemble each variant
too expensive
Solution: GenVoca component technology components encapsulate primitive extensions compose components to form variants of Jak
![Page 37: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/37.jpg)
37
GenVoca Generators
![Page 38: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/38.jpg)
38
GenVoca Model of parameterized programming
– addresses the architectural aspects of software– building blocks of domains are refinements of
fundamental domain abstractions– components are implementations of refinements – components composed by parameter instantiation
visualize component composition as stacking layers
![Page 39: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/39.jpg)
39
Quick Tutorial Components define algebraic operators Software system is type equation
System =System:
X
X
Y
Y [ ] Z Z[ ]
Question: what is relationship of components to classes?
![Page 40: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/40.jpg)
40
OO Realizations of Refinements A small-scale refinement adds new data
members, methods, and/or overrides existing methods to a class
class
subclass
subclassing relationship
![Page 41: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/41.jpg)
41
Large Scale Refinement Adds new data members and methods
simultaneously to several classes
class class class
subclass subclass subclass
![Page 42: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/42.jpg)
42
Relationship to GenVoca GenVoca components are large scale
– consistent refinements of multiple classes
application classes
![Page 43: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/43.jpg)
43
Relationship to JTSAstNode
With
Java
Kernel
Guard
Application Classes
![Page 44: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/44.jpg)
44
To Scale, Not to Scale...AstNode
J1 = Guard[ With[ Java[ Kernel ] ] ];
With
Java
Kernel
Guard
>500 classes
![Page 45: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/45.jpg)
45
Express Components as Mixin Layers
– Mixins are classes whose superclass is specified via a parameter
– Components are expressed as a mixin (whose superclass is the “lower layer”)» inner classes themselves are mixins
Component
x y z w
![Page 46: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/46.jpg)
46
Mixin-Layer Construction
Kernel
Java
With
Guard
Elegant model of component-based software construction
![Page 47: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/47.jpg)
47
Big Picture Bali is marriage of:
– DSL compiler technology – GenVoca generator technology
Bali synthesizes versions of Jak thru component composition– automatic synthesis of lexer, parser, transform
engine– can synthesize variants of other languages, too
![Page 48: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/48.jpg)
48
Current Status and Future Work
![Page 49: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/49.jpg)
49
SymTab
AST
Match
GScopeMetaprogrammingAST ConstructorsHygienic Macros
Compose
Macros
LayerDefComponent Definition
and Composition
P3Embedded DSLGenVoca Generator
for Containers
Current JTS Language
Java1.0
Java1.1Base Language1997
1998
Legend
![Page 50: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/50.jpg)
50
Current Status Jakarta now supports:
» AST constructors and escapes» generation scoping» parameterized inheritance (mixin layers)» P3 generator of container data structures
Add type checking to AST nodes» needed for semantics-based transformations
FSATS» Fire Support Automated Test System
![Page 51: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/51.jpg)
51
Self-Adaptive Software
– JTS represents applications as equations
– Users need help:» selecting components» knowing combinations that satisfy requirements
A
C
B
S =
S = A[ B[ C ] ];
express such knowledge as algebraic rewrite rules
![Page 52: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/52.jpg)
52
Design Wizard– Applies rewrite rules to optimize equations
– Space of all implementations = space of all type equations
– Rank “goodness” of a Teqn by generating cost functions that estimate performance
– Walk space (heuristically) to locate Teqn with cheapest cost
![Page 53: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/53.jpg)
53
Simple Application using P3 Spelling checker
» two containers, 1 for dictionary, another for document» initially are simple lists, optimally are hash or rbtrees
with hash comparison optimizations
Cont = list[transient] Code = rbtree[word, ... ]
![Page 54: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/54.jpg)
54
Self-Adaptivity Let application monitor
its workload
Design Wizard deduces better implementation
Jak regenerates application
Design Wizard
Application
regeneratedapplication
updatedrequirements,actualworkloads
Self-Adaptive Cycle
![Page 55: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/55.jpg)
55
Self-Adaptivity– Concept of rewriting type equations is domain-
independent
– Interesting problem when there are lots of components that implement the same functionality in different ways
this is where optimization problems arise
– Matter of analyzing additional domains and applying this technology
![Page 56: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/56.jpg)
56
Conclusions JTS integrates of different technologies:
» metaprogramming» domain-specific language compiler tools» architectural view: components/generators
Yields a powerful set of tools for creating » component technologies and their generators
Embody useful advance » generator infrastructure» automating software development
![Page 57: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/57.jpg)
57
The End
![Page 58: The Jakarta Tool Suite](https://reader036.fdocuments.us/reader036/viewer/2022081507/5681645e550346895dd631be/html5/thumbnails/58.jpg)
58
Timings How Jak works -14min Bali - 4min up to GenVoca - 9 min up to current status - 10 min current status - 11 min
Total: 48 min.