Domain Specific Domain Specific LanguagesLanguages

Lukas RenggliLukas Renggliwww.lukas-renggli.chwww.lukas-renggli.ch

ToDo- start with examples- less text

ToDo- start with examples- less text

1.DSL

– GPL vs. DSL

– Architecture

– Styles

– Embedding

2.Diesel

– Quasiquoting

– Diesel System

1. Language Aspects

Roadmap

‣ Martin Fowler, “Domain Specific Languages,” June 2008, Work In Progress. URL

‣ Diomidis Spinellis, “Notable Design Patterns for Domain Specific Languages,” Journal of Systems and Software, vol. 56, no. 1, February 2001, pp. 91—99. DOI URL

‣ Laurence Tratt, “Domain specific language implementation via compile-time meta-programming,” ACM TOPLAS, vol. 30, no. 6, 2008, pp. 1—40. DOI PDF

Literature

GPL GPL ←←→ DSL→ DSL

General Purpose Language

‣ Turing complete

‣ Well understood and widely used

‣ Applicable to a wide range of problems

GPL Pros

‣ Excellent support through IDEs

‣ Easy to find experienced developers

‣ Growth through libraries

GPL Cons

‣ Growth of language is (often) not possible

‣ Can be very verbose

‣ Lack of abstractions

Domain Specific Language

‣ Small language targeted at a particular problem domain

‣ Expressive in its own domain

‣ Often declarative

DSL Pros

‣ Tailored to a particular application domain [Mern05a]

‣ Expressive and easy to use [Huda98a]

‣ Little languages, little maintenance, increased productivity [Deur97a]

‣ Communication with domain experts [Fowl08X]

DSL Cons

‣ Extra investment

– Language engineering

– Learning a new language

‣ Weak IDE support

– Editor, Debugger, Refactoring

‣ Migration might be difficult

‣ Evolving into generality

ArchitectureArchitecture

computer: processor: cores 2 i386 disk: size 150 disk: size 75 speed 7200 sata

Processor p = new Processor(2, Processor.Type.i386);

Disk d1 = new Disk(150, Disk.UNKNOWN_SIZE, null);

Disk d2 = new Disk(75, 7200, Disk.Interface.SATA);

return new Computer(p, d1, d2);

Parse Generate

DSL Script Semantic ModelGenerated

Code

DSL Script

‣ A language to build, configure or do in your domain.

‣ Not necessary textual, could be graphical

‣ Might reuse syntax of host language

Semantic Model

‣ An in-memory representation of the subject the DSL describes.

‣ Sometimes this is the AST

‣ Separates parser and generation

Generated Code

‣ Executable representation of the DSL.

‣ Evaluation during parsing

‣ Interpretation of semantic model

StylesStyles

Styles

‣ External DSL

‣ Internal DSL

‣ Workbench

External

‣ DSLs that use a different syntax to the main language that uses them.

‣ Examples

– make, flex, yacc, bison

– XPath, SQL, regexp

– sed, awk

External Pros

‣ Language Designer

– Tools for compiler construction can be used

‣ Language User

– Simpler to use than a GPL

External Cons

‣ Language Designer

– Expensive to implement

‣ Language User

– Weak tool support

– Yet another language to learn

– Often targeted towards a particular GPL

– Often difficult to closely integrate with GPL

Internal

‣ DSLs that share the same syntax to the main language that uses them.

‣ Examples

– PetitParser (Smalltalk)

– rake, rspec (Ruby)

– jQuery (JavaScript)

– RPython (Python)

Internal

‣ A subset of the host language is used.

‣ Popular in Lisp, Scheme, Ruby, Smalltalk and JavaScript.

Internal Pros

‣ Language Designer

– No special tools required

– No new grammar required

‣ Language User

– Intermixable with GPL

– Tools continue to work

– No new language to learn

Internal Cons

‣ Limited expressivity

‣ Unnecessary syntactic noise

‣ Constrained by host language

Pattern ofInternal DSLs

Processor p = new Processor(2, Processor.Type.i386);Disk d1 = new Disk(150, Disk.UNKNOWN_SIZE, null);

Disk d2 = new Disk(75, 7200, Disk.Interface.SATA);

return new Computer(p, d1, d2);

Example

Function Sequence

computer(); processor(); cores(2); i386(); disk(); size(150); disk(); size(75); speed(7200); sata(); end();

Function Nesting

computer(

processor(

cores(2),

Processor.Type.i386),

disk(

size(150)),

disk(

size(75),

speed(7200),

Disk.Interface.SATA));

Function Chaining

computer() .processor() .cores(2) .i386() .end() .disk() .size(150) .end() .disk() .size(75) .speed(7200) .sata() .end() .end();

Closures

computer() do | c | c.processor() do | p | p.cores(2) p.i386() end c.disk().size(150) c.disk() do | d | d.size(75) d.speed(7200) d.sata() endendb.computer() do b.processor() do b.cores(2) b.i386() end b.disk().size(150) b.disk() do b.size(75) b.speed(7200) b.sata() endend

Literal Collections

[:computer,

[:processor,

[:cores, 2],

[:type, :i386]],

[:disk,

[:size, 150]],

[:disk,

[:size, 75],

[:speed, 7200],

[:interface, :sata]]]

Other Approaches

‣ Operator Overloading

– C++, C#, Smalltalk, Python, Ruby, ...

‣ Annotations

– Java, C#, Smalltalk, Python, ...

‣ Parse Tree Manipulation

– C#, Smalltalk

‣ Macros

– LISP, Scheme, Template Haskell

Language Workbenches

‣ IDEs designed for building DSLs.

‣ Common representation of host and domain specific languages.

‣ Examples

– JetBrains Meta Programming System (MPS)

– openArchitectureWare

Martin Fowler, Language Workbenches: The Killer-App for Domain Specific Languages?

EmbeddingEmbedding

Heterogenous

‣ At least one of the languages is largely, or completely ignorant of the existence of the other languages.

‣ Examples

– Preprocessors

– Stratego/XT, TXL

Homogeneous

‣ All languages are specifically designed to work with each other.

‣ Examples

– LISP/Scheme Macros, Template Haskell

– Nemerle, Metalua (Lua), xTc (C)

– Converge, Diesel

Heterogenous

Homogenous

Internal rake Diesel Pidgin

External makeConverge

Diesel Creole

QuasiquotingQuasiquoting

Compile timemeta

programming

DSL for AST construction

QuasiquoteMeta Level

SourcecodeBase Level

Quasiquote `` Unquote `,

Lisp Scheme

Template Haskell

Quasiquote

Smalltalk

Quote `

Quasiquote `` [| ... |] ``

Unquote , ${ ... } `,

Splice ,@ $< ... > `@

Quasiquote Example

raise: aNode to: anIntegeranInteger = 0

ifTrue: [ ^ ``1 ].anInteger = 1

ifTrue: [ ^ aNode ].^ ``(`,(self raise: aNode to: anInteger - 1) *

`,aNode)

power3: aNumber^ `@(self raise: ``aNumber to: 3)

Quasiquote Decompiled

raise: aNode to: anIntegeranInteger = 0

ifTrue: [ ^ RBLiteralNode value: 1 ].anInteger = 1

ifTrue: [ ^ aNode ].^ RBMessageNode

receiver: (self raise: aNode to: anInteger - 1)selector: #* arguments: (Array with: aNode)

power3: aNumber^ aNumber * aNumber * aNumber

DIESELDIESEL

Teaching IDEs new LanguagesTeaching IDEs new Languages

Floating Point Numbers

digit = "0" | "1" | ... | "9" ;number = [ "-" ] digit { digit }

[ "." digit { digit } ] ;

JParsec Parser

final Pattern digit = Patterns.range('0', '9');

final Pattern number = Patterns.seq(Patterns.isChar('-').optional(), digit.many(1), Patterns.seq(Patterns.isChar('.'), digit.many(1)).optional());

PetitParser

digit ^ $0 asParser / $1 asParser / ... / $9 asParser

number ^ $- asParser optional , self digit plus , ($. asParser , self digit plus) optional

Pidgin PetitParser

digit $0 / $1 / ... / $9

number $- optional , digit plus , ($. , digit plus) optional

Pidgin Transformations

DSLTreePattern newexpression: '`#literal'do: [ :ast | ``(`,(ast) asParser) ]

DSLTreePattern newexpression: '`variable'do: [ :ast | ``(self `,(ast name)) ]

DSLTreePattern newexpression: '`.statement'do: [ :ast | ``(^ `,(ast statement)) ]

Pidgin

In the domain of natural language, a pidgin is a grammatically simplified form of a language used for communication between people not sharing a common language.

Pidgin PetitParser

digit $0 / $1 / ... / $9

number $- optional , digit plus , ($. , digit plus) optional

Creole PetitParser

digit = "0" | "1" | ... | "9" ;number = [ "-" ] digit { digit }

[ "." digit { digit } ] ;

EBNF Parser

production = identifier "=" choice ";" ;choice = sequence { "/" sequence } ;sequence = element { element } ;element = option / repetition / literal / identifier ;option = "[" choice "]" ;repetition = "{" choice "}" ;literal = '"' string '"' / "'" string "'" ;

option = "[" choice "]" ;

super repetition ==> [ :ast | ``(`,(ast second) optional) ]

Creole

In the domain of natural language, a creole is a a mother tongue formed from the contract of two languages through an earlier pidgin stage.

Implementation

Tool Integration

Homogenous Embedding

Scoping of Languages

Demo

DIESEL Summary

‣ Language Workbench and Meta-Programming System

‣ Extension points in the compiler pipeline

‣ Tight integration with existing tools

DIESEL Pros

‣ Tools continue to work

‣ Homogenous Embedding

‣ Relatively simple implementation

‣ Incremental development of DSLs

DIESEL Cons

‣ Pidgin DSL is constrained by syntax

‣ Creole DSL requires you to write a parser and specify a transformation

‣ Language definitions are in “scripts”, there is no real model behind it

Language AspectsLanguage Aspects

Bending the Host LanguageBending the Host Language

Aspects

Aspect

Pointcut

Advice

Source

Language Aspects

Pointcut

Advice

AspectLanguage Source

CompilerHighlighterCompletion{

Start withhost language

Modular

Crosscutting

Integration

Demo

Bachelor and Master Projects

‣ Better editor integration

‣ Language Aspects for C#

‣ Transparent optimization of DSLs

‣ Refactoring and migration of DSLs

What you should know!

‣ What are the reasons for using DSLs?

‣ What are the reasons for not using DSLs?

‣ What is the difference between an API and an internal DSL?

‣ What is a language workbench?

‣ What is quasiquoting and why is it useful?

Can you answer these questions?

‣ How would you extend an internal DSL?

‣ How would you execute a graphical DSL?

‣ What problems do we face when modifying the language within an interactive environment?

‣ Why is it often difficult to achieve homogenous embedding?