Code Generation

Course IN4308Master Computer Science

Delft University of Technology

Eelco Visserhttp://eelcovisser.org

Lecture 10May 4, 2010

Coming up

Lecture 8: Context-sensitive transformation

★ design 2

★ transformation with dynamic rewrite rules

Lecture 9: Static analysis & error checking

★ name resolution, reference resolution

★ type analysis

Lecture 10: Code generation

★ string templates, code generation by model transformation

★ concrete object syntax

Lecture 11: Code generation strategies

★ customization of generated code

Code Generation

Code Generation: From Model to Code

Model

★ structured representation (abstract syntax)

★ produced by parser

★ checked for consistency

★ transformed to core language

Code

★ program text

★ no structure

★ for consumption by interpreter or compiler

entity Blog { name :: String entries :: List<BlogEntry>}

CREATE TABLE IF NOT EXISTS ‘Blog‘ ( ‘id‘ int(11) NOT NULL auto_increment, ‘name‘ text, PRIMARY KEY (‘id‘)) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

CREATE TABLE IF NOT EXISTS ‘Blog_entries_BlogEntry‘ ( ‘parent‘ int(11) NOT NULL, ‘value‘ int(11) NOT NULL) ENGINE=MyISAM DEFAULT CHARSET=latin1 ;

Example: Data Model to Database Schema

Traversing Abstract Syntax Trees

Pattern matching

Field(name, SimpleType(x))

Accessor Functions

if(t instanceof Field) { name := t.name(); if(t.type() instanceof SimpleType) { ... }}

Print Statements

Printing Code

entity-to-sql = ?Entity(name, fields); <printstring> "DROP TABLE IF EXISTS ‘"; <printstring> name; <printstring> "‘;\n"; <printstring> "CREATE TABLE IF NOT EXISTS ‘"; <printstring> name; <printstring> "‘ ( \n"; <printstring> "‘id‘ int(11) NOT NULL auto_increment;\n"; <map(field-to-sql(|name); printstring> fields; <printstring> "PRIMARY KEY (‘id‘)\n"; <printstring> ") ENGINE=MyISAM DEFAULT CHARSET=latin1 "; <printstring> "AUTO_INCREMENT=1 ;\n\n"

Printing Code

Advantages

★ accessible (easy to implement in any language)

★ concrete syntax

★ good performance

Disadvantages

★ order of evaluation

★ code fragments disrupted to splice meta-data

★ no formatting

Composing Strings

String Composition

entity-to-sql : Entity(name, fields) -> <concat-strings> ["DROP TABLE IF EXISTS ‘",name,"‘;\n", "CREATE TABLE IF NOT EXISTS ‘",name,"‘ ( \n", "‘id‘ int(11) NOT NULL auto_increment, \n", <map(field-to-sql(|name));concat-strings> fields, "PRIMARY KEY (‘id‘)\n", ") ENGINE=MyISAM DEFAULT AUTO_INCREMENT=1 ;\n\n" ]

field-to-sql(|entity) : Field(name, SimpleType(type)) -> <concat-strings>["‘",name,"‘ ",<type-to-sqltype> type,",\n"]

String Composition

Advantages

★ concrete syntax

★ assembly of code fragments independent of order of evaluation

Disadvantages

★ disrupted code fragments

★ quotes

★ escapes

Template Engine

Template Engine

<<DEFINE entity_to_sql FOR entity>> DROP TABLE IF EXISTS ‘<<entity.name>>‘; CREATE TABLE IF NOT EXISTS ‘<<entity.name>>‘ ( ‘id‘ int(11) NOT NULL auto_increment, <<EXPAND type_to_sqltype FOREACH entity.fields>> PRIMARY KEY (‘id‘) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;<<ENDDEFINE>

DSL for model to text generation

Template Engine

Advantages

★ concrete syntax

★ less quoting

★ long code fragments

★ anti-quotation instead of string concatenation

★ no escapes of string meta characters

Disadvantages

★ no syntax check

★ result is string / printed

★ no structure

★ integrate separate language

String Interpolation

String Interpolation

entity-to-sql : Entity(name, fields) -> $[DROP TABLE IF EXISTS ‘[name]‘; CREATE TABLE IF NOT EXISTS ‘[name]‘ ( ‘id‘ int(11) NOT NULL auto_increment, [<map(field-to-sql(|name))> fields] PRIMARY KEY (‘id‘) ) ENGINE=MyISAM DEFAULT AUTO_INCREMENT=1; ]

field-to-sql(|entity) : Field(name, SimpleType(type)) -> $[‘[name]‘ [<type-to-sqltype> type]]

String Interpolation

Advantages

★ concrete syntax

★ assembly of code fragments independent of order of evaluation

★ long code fragments

★ anti-quotation instead of string concatenation

★ no escapes of string meta characters

★ formatting, relative indentation

Disadvantages

★ escaping of string interpolation meta characters

Composing ASTs

Code Generation by Model Transformation

Code represented as AST

★ (typechecked) structured representation

Text generation by pretty-printing

★ separate code formatting from code generation

Apply transformations after generation

★ aspect weaving

★ optimization

Composing Abstract Syntax Trees

grammar-to-signature : Grammar(ss, ps) -> Signature(ss, cds) where cds := <map(production-to-consdecl)> ps

production-to-consdecl : Prod(symbols, sym, annos) -> ConsDecl(x, types, sort) where x := <annos-to-constructor> annos ; types := <filter(symbol-to-type)> symbols ; sort := sym

annos-to-constructor : [Anno(Application("cons", [Constant(c)]))] -> constr where constr := <unescape; un-double-quote> c

Composing Abstract Syntax Trees

Advantages

★ syntax check through type system (if available)

★ automatic formatting

★ transformation after generation

Disadvantages

★ no concrete syntax

Deriving Rewrite Rules

How does this scale? private $name;function getName(){ return $this->name; }function setName($val){ $this->name = $val;}

generate-entity-field-code : Field(name, SimpleType(t)) -> [ InstanceVariable(Modifiers([Private()]),[Normal(Variable(Simple(name)))]) , FunctionDecl(get, [], [ Return(Some(ObjectAccess(Variable(Simple("this")), ObjectProperty(Simple(name)))))]) , FunctionDecl(set, [Param(Variable(Simple("val")))], [ Expr(Assign(ObjectAccess(Variable(Simple("this")), ObjectProperty(Simple(name))), Variable(Simple("val"))))]) ] with get := <camelcase>["get",name] with set := <camelcase>["set",name]

Concrete Object Syntax

Concrete Syntax Patterns

generate-entity-field-code : FieldDefinition(str_name, type) -> php-member* |[ private $str_name ; function str_get() { return $this->str_name; } function str_set($val) { $this->str_name = $val; } ]| where <is-primitive-type> type with str_get := <camelcase>["get",str_name] with str_set := <camelcase>["set",str_name]

Concrete Syntax Ingredients

Quotation of code fragment

★ |[ .... ]|

Disambiguation through tagging

★ php-member*|[ ... ]|

Meta-variables

★ $this->str_name = $val

Implementing Concrete Object Syntax

Extending meta-language

★ Stratego + PHP

Parsing meta-programs

Normalizing to core syntax

Extending the Meta-Language (1)

module Strategoexports context-free syntax Int -> Term {cons("Int")} String -> Term {cons("Str")} Var -> Term {cons("Var")} Id "(" {Term ","}* ")" -> Term {cons("Op")} Term "->" Term -> Rule {cons("RuleNoCond")} Term "->" Term "where" Strategy -> Rule {cons("Rule")}

Extending the Meta-Language (2)

module StrategoPHPimports PHPexports context-free syntax "php-member" "|[" ClassMember "]|" -> Term{cons("ToMetaExpr")} "php-member*""|[" ClassMember* "]|" -> Term{cons("ToMetaExpr")} variables "expr_"[A-Za-z0-9\_]+ -> Expr {prefer} "str_"[A-Za-z0-9\_]+ -> String {prefer}

Parsing Concrete Object Syntax

Field(name, SimpleType(t)) -> php-member|[ private $str_name; ]|

==> parse

Rule(Op("Field", [Var("name"), Op("SimpleType",[Var("t")])]), ToMetaExpr(InstanceVariable(Modifiers([Private()]), [Normal(Variable(Simple(meta-var("str_name"))))])))

==> meta-explode

Rule(Op("Field", [Var("name"), Op("SimpleType",[Var("t")])]), Op("InstanceVariable",[ Op("Modifiers", [Op("Cons",[Op("Private",[]),Op("Nil",[])])]), Op("Cons", [Op("Normal",[Op("Variable",[Op("Simple", [Var("str_name")])])]), Op("Nil",[])])]))

==> pretty-print

Field(name, SimpleType(t)) -> InstanceVariable(Modifiers([Private()]),[Normal(Variable(Simple(str_name)))])

Summary of Code Generation Techniques

Print statements

★ easy

String composition

★ out of order evaluation

Template engines

★ less quoting

String interpolation

★ template engine built-in

Abstract syntax

★ precise, transformation after generation

Concrete object syntax

★ precise

★ concrete

Schedule

Case 3

★ Deadline extension: May 9 23:59

Case 4

★ `take home exam’

★ Questions about lectures 11 to 14

★ Deadline: June 15

Design 2

★ Syntax for your DSL

Next

★ No lecture next week (May 11)

★ Lecture 11: code generation policies