1

Lecture 12

Implementing Small Languagesinternal vs. external DSLs, hybrid small DSLs

Ras Bodik Ali and Mangpo

Hack Your Language!CS164: Introduction to Programming Languages and Compilers, Spring 2013UC Berkeley

OverviewImplementation of DSLs

External vs. internal DSLsinternal == embedded in the host language

Deep embedding

Shallow embedding

Examples2

Styles of DSL implementationsExternal DSL

- own syntax (and parser)- dedicated interpreter

Internal DSL: deep embedding in the host language

- deep embedding: program exists as data (eg AST)

- the host language implements an interpreterInternal DSL: shallow embedding in the host language

- shallow embedding: DSL constructs are composed purely of host language constructs

- we are using the interpreter of the host language

note: instead of interpreter, we can use a compiler

3

External DSLs

4

Architecture of external-DSL implementationsExternal DSL is a standalone language

- just like “big” languages, Java, Python, C, …- domain programming abstractions with own

syntax

Similar interpreter/compiler architecture parser, optional analysis of AST, interpretation/compilation

If compiled, DSL need not be translated into code

- instead, it can be translated to a data structure which will be used by some other program P

- yes, P can be viewed as an interpreter of the DSL, and the data structure is an “AST”

5

ExamplesInterpreted external DSLs

External DSLs compiled to code

External DSLs compiled to a data structure

6

Internal DSLsshallow embedding

7

Embed your DSL into a host languageShallow embedding: the new language is a library written in the host language

sometimes called a “framework”

When DSL is implemented as a library, we often don’t think of it as a language

even though it defines own abstractions and operations

But the library implementation goes very far

8

Example 1: socketsSockets are library with own abstractions and rules

Socket f = new Socket(mode)f.connect(ipAddress)f.write(buffer)f.close()

Programming abstractions:socket, address, buffer

Usage rules:- do not write into a closed socket- do not close a socket twice

9

Usage rules (side note)How abstractions can be combined is part of the language.

With network sockets, these are enforced at runtime:

ex: sock.write() checks that socket has been open

This is analogous to dynamic (runtime) type checking

ex: in Python, 1 + “a” will fail at runtime, when types of 1 and “a” are checked in the + operation

10

Usage rules (side note)Could we enforce these socket rules at compile time?

This would be analogous to static type checking.

ex: in Java, int i; String s; i+s; fails at compile time even knowing what specific values i and s will take at runtime.

11

Usage rules (side note)

Could we enforce these socket rules at compile time?

Yes, use static types (as in say Java) and refine Socket into Socket, OpenSocket, ClosedSocket

How do we rewrite this code to use refined types?

Socket f = new Socket(mode)f.connect(ipAddress)f.write(buffer)f.close()

12

Example 2: regex matchingThe library defines:

i) functions that DSL programmers use to define a pattern

ex: pattern ("abc"|"de")."x" can be defined as follows:

def patt = seq(alt(prim("abc"),prim("de")),prim("x"))

ii) a function matches of a string against a pattern:

match(string, patt)13

Example 3:rfig: formatting DSL embedded into Ruby.

see slide 8 in http://cs164fa09.pbworks.com/f/01-rfig-tutorial.pdf

14…

The animation in rfig, a Ruby-based language

slide!('Overlays',

'Using overlays, we can place things on top of each other.', 'The pivot specifies the relative positions', 'that should be used to align the objects in the overlay.',

overlay('0 = 1', hedge.color(red).thickness(2)).pivot(0, 0),

staggeredOverlay(true, # True means that old objects disappear 'the elements', 'in this', 'overlay should be centered',

nil).pivot(0, 0),

cr, pause, # pivot(x, y): -1 = left, 0 = center, +1 = right

staggeredOverlay(true, 'whereas the ones', 'here', 'should be right justified',

nil).pivot(1, 0), nil) { |slide| slide.label('overlay').signature(8) }

15

DSL as a frameworkIt may be impossible to hide plumbing in a procedure

these are limits to procedural abstraction

We can use advanced language features, such as closures, coroutines

Framework, a library parameterized with client code• typically, you register a function with the library• library calls this client callback function at a

suitable point• ex: an action to perform when a user clicks on

DOM node

16

Example 4: jQueryBefore jQuery

var nodes = document.getElementsByTagName('a'); for (var i = 0; i < nodes.length; i++) { var a = nodes[i]; a.addEventListener('mouseover', function(event)

{ event.target.style.backgroundColor=‘orange'; }, false ); a.addEventListener('mouseout', function(event)

{ event.target.style.backgroundColor=‘white'; }, false ); }

jQuery abstracts iteration and events

jQuery('a').hover( function() { jQuery(this).css('background-color', 'orange'); }, function() { jQuery(this).css('background-color', 'white'); } );

17

Embedding DSL as a languageHard to say where a framework becomes a language

not too important to define the boundary precisely

Rules I propose: it’s a language if 1) its abstractions include compile- or run-time

checks --- prevents incorrect DSL programsex: write into a closed socket causes an error

2) we use syntax of host language to create (an illusion) of a dedicated syntax

ex: jQuery uses call chaining to pretend it modifes a single object: jQuery('a').hover( … ).css( …)

18

rake rake: an internal DSL, embedded in RubyAuthor: Jim Weirich

functionality similar to make– has nice extensions, and flexibility, since it's

embedded– ie can use any ruby commands

even the syntax is close (perhaps better):– embedded in Ruby, so all syntax is legal Ruby

http://martinfowler.com/articles/rake.html19

Example rake filetask :codeGen do # do the code generationend

task :compile => :codeGen do # do the compilationend

task :dataLoad => :codeGen do # load the test dataend

task :test => [:compile, :dataLoad] do # run the testsend 20

Ruby syntax rulesRuby procedure call

21

How is rake legal ruby?Deconstructing rake (teaches us a lot about

Ruby):

task :dataLoad => :codeGen do # load the test dataend

task :test => [:compile, :dataLoad] do # run the testsend

22

Two kinds of rake tasksFile task: dependences between files (as in

make)file 'build/dev/rake.html' => 'dev/rake.xml' do |t|

require 'paper' maker = PaperMaker.new t.prerequisites[0], t.name

maker.runend

23

Two kinds of tasksRake task: dependences between jobs

task :build_refact => [:clean] do target = SITE_DIR + 'refact/' mkdir_p target, QUIET require 'refactoringHome' OutputCapturer.new.run {run_refactoring}end

24

Rake can orthogonalize dependences and rules

task :second do #second's bodyend

task :first do #first's bodyend

task :second => :first

25

General rulesSort of like make's %.c : %.o

BLIKI = build('bliki/index.html')

FileList['bliki/*.xml'].each do |src|file BLIKI => src

end

file BLIKI do #code to build the blikiend

26

Internal DSLsdeep embedding

27

Deep embeddingRepresent the program as an AST

or with some other data structure

This AST can be interpreted or compiled

28

Parsing involved: DSL in a GP languageGP: general purpose language

29

Parsing involved: GP in a DSL languageGP: general purpose language

30

Reading Read the article about the rake DSL

31

Acknowledgements This lecture is based in part on

Martin Fowler, “Using the Rake Build Language”

Jeff Friedl, “Mastering Regular Expressions”

32