DSL in Clojure

Misha Kozik @mishadoff

developers domainexperts

developer domainexpert

Badass Guys

developers domainexperts

developer domainexpert

Badass Guys

developers domainexperts

“DSL-powered”Badass Guys

not Turing Complete

SQLselect statement

DSL

SQLDSLquery plan

SQLalgorithms

DSL

SQLDSLnested loop join

#1#2#1

Outer Inner

#3

#1#2#1

#3#4

SQLDSLnested loop join

#1#2#1

Outer Inner

#3

#1#2#1

#3#4

SQLDSLhash join

#1#2#1

Outer Inner (hashed)

#3

#1 #2 #3

#2

#1 #4

cd632f

0980eb

48b5d4

SQLDSLhash join

#1#2#1

Outer Inner (hashed)

#3

#1 #2 #3

#2

#1 #4

cd632f

0980eb

48b5d4

SQLDSLmerge join

#106#123

Outer (sorted) Inner (sorted)

#133

#106#106#117#123

SQLDSLmerge join

#106#123

Outer (sorted) Inner (sorted)

#133

#106#106#117#123

SQLDSLit’s just a join

SQL-on-Everything

LINGUALCASCADING

HiccupDSL

HiccupDSL

HiccupDSL

HiccupDSL

LeinDSL

Binary Tree

Binary Tree

Internal DSL: TreeClojure way

headleft child right child

Internal DSL: TreeJava way

External DSL: Tree

wow, such declarative

Do not re

ad this c

ode!

Domain Expert way

Generative DSL: Macros

excerpt macro from core.async

InstaparseWhat if context-free grammars were as easy to use as regular expressions?

Context-Free Grammar

Context-Free Grammar

Context-Free Grammar

Context-Free Grammar

Context-Free Grammar

Balanced ParensExample

Balanced ParensExample

Clojure is a functional programming language better than java

English GrammarExample

English GrammarExample

Named Entity Recognition

Kiev is the capital of Ukraine

Entity:City Entity:Country

English GrammarExample

Sentiment Analysis

SQL-2003 GrammarExample

1300+ rules

http://savage.net.au/SQL/sql-2003-2.bnf

Integration Story

Grouping Metric

Grouping Metric

Grouping Metric

DSL to define formula

JVMWhy Clojure?

LispWhy Clojure?

Code is DataWhy Clojure?

ImmutableWhy Clojure?

REPLWhy Clojure?

PracticalWhy Clojure?

CommunityWhy Clojure?

Parser Development

insta/transformDevelopment

insta/transformDevelopment

Syntax ErrorException Handling

Tree TestsTesting

Expected Actual

Starting RuleTesting

Isolated Parsing

Performance TestsTesting

clojure.test does not support timeouts, write by yourself

Ambiguity TestsTesting

instaparse is able to handle ambiguity grammars though, still good to know where we behave poorly

Use Cases TestsTesting

Year over Year growth

InterfaceJava Integration

clojure-maven-pluginJava Integration

Demo Time

ANTLRAlternatives

http://stackoverflow.com/questions/1931307/antlr-is-there-a-simple-example

parboiledAlternatives

LessonsLearned

Lessons Learned #1

Too Much Rules100+ rules, possible to reduce to 50

Lessons Learned #2

Lein and Mavenare not friends

Lessons Learned #3

Code Coveragenot integrated

Lessons Learned #4

Versioningas a library

Lessons Learned #5

Evaluationis not a parser responsibility

Questions?