Programming Modeling
fromto
…and back again
Markus VoelterIndependent/itemis
[email protected]
„Houston,we have a
Problem!“
Programming Languages
C# C++ Java
Python
GroovyRubyErlang
CFortran
Programming Languagesare not
enough.expressive
C# C++ Java
Python
GroovyRubyErlang
CFortran
Programming Languagesare not
enough.high-level
C# C++ Java
Python
GroovyRubyErlang
CFortran
Programming Languagesare not
enough.abstract
C# C++ Java
Python
GroovyRubyErlang
CFortran
Programming Languagesare not
enough.domain-specific
C# C++ Java
Python
GroovyRubyErlang
CFortran
Programming Languagesare notenough.
C# C++ Java
Python
GroovyRubyErlang
CFortran
Programming LanguagesFormats
C# C++ Java
Python
GroovyRubyErlang
CFortranHTML
XMLCSSyaml
json
Programming LanguagesFormats
C# C++ Java
Python
GroovyRubyErlang
CFortranHTML
XMLCSSyaml
json JEEWPF
Rails
Django
JMS
lift
Frameworks
Programming LanguagesFormats
C# C++ Java
Python
GroovyRubyErlang
CFortranHTML
XMLCSSyaml
json JEEWPF
Rails
Django
JMS
lift
Frameworksare not enough.
not enough?for what?
not enough?for what?
not enough?for what?
Modeling… Higher Level… Domain Specific
Interpretation… Code Generation
Concepts & Notations
Modeling… Higher Level… Domain Specific
Interpretation… Code Generation
Concepts & Notations
Solves theProblem!
Modeling Tools Programming Tools
!=
Different Worlds
Modeling ToolModeling Tool
!=
Different Worlds
Different WorldsMix Models and Programs
Different WorldsAST Navigation & Query
Mix Models and Programs
Different WorldsAST Navigation & Query
Mix Models and Programs
Integration of 3GL code
Different WorldsAST Navigation & Query
Mix Models and Programs
Integration of 3GL codeCode Constraints
Domain SpecificNotations
andAbstractions
LibrariesFrameworks
(Fluent) APIs
Flexible! Limited!
GraphicalTextual
FormsTables
TextualTrees
Limited!Flexible!
CustomizeGenerator
orInterpreter
Limited!Flexible!
ReflectionMeta ProgramsOpen Compilers
Define customQueryorNavigate
Limited!Flexible!
AST APIsStatic Analysis
Transformor
Regex
CustomValidationorError Checks
Limited!Flexible!
IDE pluginsStatic AnalysisOpen Compilers
DifferentRepresentations
andProjections
Limited!Flexible!Text is TextCode FoldingTree ViewsVisualizations
Limited!Flexible?Python-to-C-likeInternal DSLs
Mixingand
ComposingLanguages
Embed-As-StringSpecific: LINQ
Mature!Brittle!
IDE Support
Scalable
Modeling Tools…!?
Usable
Mature!Brittle!
Testing
Debugging
?
Refactoring
Mature!Brittle!
Branching
VersioningDiff, Merge
for some tools…
Gets theGets some
some people doubt that…
Job Done!Jobs done.everybody agrees…
Whythe difference?
Modeling Programming
Modeling Programming
… (Mostly) Textual Notations
… Concrete Syntax Storage
… (Fancy) ASCII Editors
… Read-Only Visualizations
Modeling
… (Mostly) Graphical Notations
… Abstract Syntax Storage
… Projecting Editors
… Different editable views for model
Programming
… (Mostly) Textual Notations
… Concrete Syntax Storage
… (Fancy) ASCII Editors
… Read-Only Visualizations
Whythe difference?
… a Different Perspective
Programmingthe way we do
Modeling?
Modelingthe way we do
Programming?
Modeling ProgrammingProgramming Modeling
====
We don‘t want to
we want tomodel,
program!
We don‘t want to
we want tomodel,
program!… at different levels of abstaction… from different viewpoints… integrated!
We don‘t want to
we want tomodel,
program!… with different degrees of
… with suitable notations… with suitable expressiveness
domain-specificity
We don‘t want to
we want tomodel,
program!precise and tool processable
And always:
Where do we go from here?
A vision forprogramming
A vision forprogramming
EnablingTechnologies
A vision forprogramming
EnablingTechnologies
AvailableTooling
A vision forProgramming
Programming Languages
are not
enough.MODULAR
Programming Languages
are not
enough.COMPOSABLE
Programming Languages
are not
enough.CONFIGURABLE
Programming Languages
are not
enough.ADAPTABLE
Programming Language Syntax
is not
enough.FLEXIBLE
L
with many first class concepts!
Big Language?
L
Small Language?
and poweful conceptswith a few, orthogonal
my L
a b c
d e f
g h i
j k l
Modular Language
composable conceptswith many optional,
Like frameworks and libraries,
Modular Language
Like frameworks and libraries,
but with syntaxand IDE support
Modular Language
Growing A Language(Guy L Steele)
LanguageWorkbench
(Martin Fowler)
LanguageWorkbench
(Martin Fowler)
Freelydefine
integratethem
languages and
?
LanguageWorkbench
(Martin Fowler)
usepersistent
abstractrepresentation
LanguageWorkbench
(Martin Fowler)
language ::=schema
editorsgenerators
++
?
LanguageWorkbench
(Martin Fowler)
editingprojectional
LanguageWorkbench
(Martin Fowler)
persistincompletecontradictory
information
or
LanguageWorkbench
(Martin Fowler)
powerfulediting
testingrefactoring
debugging
language definitionIDE definition
implies
+groupware
LanguageWorkbench
(Martin Fowler)
support for„classical“
programming„classical“
modelingand+
Syntaxprimarilytextual
Syntaxprimarilytextual
with more symbolsthink: mathematics
Syntaxprimarilytextual
sometimesbox&line style
Syntaxprimarilytextual
sophisticatedvisualizations
Viewpointssuitableabstractions
notationsfor each
and
ViewpointsIntegrated
references
transitions
symbolic
seamlessand
via
ViewpointsBusiness
custompurpose-built
create/include
ViewpointsCustomNotations
businessexpert integration
real
Business
ViewpointsCustomNotations
businessexpert integration
real
Business
But that‘s another talk…
ViewpointsTechnical
predefinedlibrary
configure
Example Languages
matricesAdding
embeddedto C in an
environment.
(Seemingly)
Simple Example
Currently:
1 •Declare Data Structures in XML
2 •Generate Headers
3 •Implement manually in C
Currently:
Matrices
and generator
not supportedin XML format
Currently:
Tool team
… a lot of work
would have todo the extension
… busy… one central tool
Currently:
No real
… type checks
compiler supportin the resulting C code
… operator overloading… generics (matrix<int>)… matrix syntax?
Better Solution
types
matrixgeneric
vector and
Better Solution
matrixreal
literals vector
and
Better Solution
highlightssyntax
vectors for
and matrices
Better Solution
operator overloading
Better Solution
operator overloading
staticoptimization… symmetrical matrices… identity matrix… diagonal matrices
Better Solution
math notation
Better Solution
used only bythose who
separatea
module language
really need it
In addition: PLE Variability
variabilityannotating
expressionsto
arbitrary regions
In addition: PLE Variability
variabilityannotating
expressionsto
arbitrary regions
statically checked
In addition: PLE Variability
variabilityannotating
expressionsto
arbitrary regions
project variant
more later…(tool examples)
EnablingTechnologies
EnablingTechnologies
AdvancedParser
Generators
Modeling as Programmig
… (Mostly) Textual Notations
… Concrete Syntax Storage
… (Fancy) ASCII Editors
… Read-Only Visualizations
Custom Syntax
GraphicalTextualSymbolic++
IDESupport
TeamworkDebuggingCustom Editors
CompleteSymbolicIntegration
Goto DefFind RefsRefactoring
Xtext-like Toolsprovide editor support
TeamworkCustom Editors
Goto DefFind Refs
Refactoring
UnicodeLimited to
how to handlenon-character symbols
TextGraphics !=
two worlds…separate editors… per syntax/viewpoint… models can still be ref integrated
What if!combine languagessame seq<Char> as
different tokens?
Ambiguous!Not Parsable.
Bad Luck.
Traditionally:Language ::=
Token* | Rule*;
Traditionally:Language ::=
Token* | Rule*;
Token ::= Character*;
Traditionally:Language ::=
Token* | Rule*;
Token ::= Character*;
Rule ::= Token* | Rule;
Traditionally:Language ::=
Token* | Rule*;
1. Global Tokenization2. Grammar Recognition
2 Phases:
(aka Scanning)
(aka Parsing)
Modern:Language ::=
Rule*;Rule ::= Char* | Rule*;
Grammar Recognition1 Phase:
directly on char*
ScannerlessParsing
No Ambiguity
Always Parsable.when combining languages
(well, almost. There are some corner cases…)
EnablingTechnologies
ProjectionalEditing
Parser-basedtext… to tree… to text
Projectionaltree… to text-lookalike (editor)… to other trees … [*]… to text
Programming as Modeling
… (Mostly) Graphical Notations
… Abstract Syntax Storage
… Projecting Editors
… Different editable views for model
Programming as Modeling
… (Mostly) Graphical Any kind of Notations
… Abstract Syntax Storage
… Projecting Editors
… Different editable views for model
Language
There‘s no parsing.
Unique Language Element Identity.
Unlimted language composition.
Composition
Textual
Graphical
Semi-Graphical
treated the samecan be mixed
like ASCII }box & line
mathematical
FlexibleNotations
AutomaticIDE Extension
tool support is inherentfor languages build with
projectional tools
language definitionIDE definition
implies
MultipleNotations
… for the same concepts
e.g. in different contextsor for different tasks
PartialProjections
… different views… for different roles/people… only a particular variant
Storage!= Schema
… independent of language schema!variability annotations
… store arbitraty meta datachange logconflicting information
… „aspects“, overlay
ProgramsLive
think: spreadsheet
a change to one part of programcan lead to (dependent) changesin other parts
Tree Editing
… try to make it feel like text
… takes some getting used tobut: for more flexible notations a more general editing paradigm is needed
… is different from editing text
InfrastructureIntegration
… diff/merge must be in tool
… existing text tools don‘t work
… storage is not text
ProprietaryTools
… no interop
… no standards
AvailableTooling
AvailableTooling
EclipseXtext
http://eclipse.org/modeling
http://eclipse.org/xtext
Eclipse Xtext
Building Textual Editors
Eclipse Xtext
Building Textual Editors
Xtext: Specify Grammar
Xtext: Gen. Meta Model
Xtext: Constraints
Xtext: Generated Editor
Code Completion
Xtext: Generated Editor
Syntax ColoringCustom Keyword Coloring
Xtext: Generated Editor
Realtime ConstraintValidation
Xtext: Generated Editor
CustomizableOutlines
Xtext: Generated Editor
Code FoldingXtext: Generated Editor
Goto Definition Find ReferencesCross-File ReferencesModel as EMF
Xtext: Generated Editor
Xtext: Generated Editor
PlatformArchitecture
DSL
As you understandand develop
yourPlatform
Architecture…
Develop a language to express it!
Language resembles
architectural concepts…
Express the applications
with the language.
Architecture As LanguageAnother Story…
DEMO I
Building DSLs with Eclipse Xtext
AvailableTooling
AvailableTooling
Jetbrains‘
ProgrammingMeta
System
IntelliJ IDEAResharper
also do…
Q3 2009released in
1.1 RC1currently
Apache 2.0licensed under
Build new standalone DSLs
Build new standalone DSLsBuild DSLs that reuse parts
of other languages
Build new standalone DSLsBuild DSLs that reuse parts
of other languages
(MPS comes with BaseLanguage)extend base language
Java++
Build new standalone DSLsBuild DSLs that reuse parts
of other languages
(MPS comes with BaseLanguage)extend base languagebuild DSLs that reuse parts
of BaseLanguage
Java++
Java + Extension
Language Extension Example
Java
ReadWriteLock l = …l.readLock().lock();try { //code} finally {
l.readLock().unlock();
}
Java + Extension
Language Extension Example
Old
Java
ReadWriteLock l = …l.readLock().lock();try { //code} finally {
l.readLock().unlock();
}
Java + Extension
ReadWriteLock l = …lock (l) { //code}
Language Extension Example
Old New
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Structure Editor Typesystem Generator
Language Extension ExampleResult behaves like a native
base language construct
Language Extension ExampleResult behaves like a native
base language construct
Language Extension ExampleTranslated to regular Java code
based on the generator
package jaxdemo.sandbox.sandbox;
import java.util.concurrent.locks.Lock;
public class DemoClass {
private Lock lock;
public DemoClass() { try { this.getLock().lock(); SharedResouce.instance().doSomething(); } finally { this.getLock().unlock(); } }
private Lock getLock() { return this.lock; }
}
Example LanguagesUI Language
Example LanguagesHTML Templates
Example LanguagesPersistent Classes
DEMO I
Building DSLs with JetBrains MPS
AvailableTooling
AvailableTooling
Intentional SoftwareDomain
Workbench
Dec 15, 2008Version 1.0 released in
currently at 1.4
CommercialProduct
Eval availableupon request
Statemachine ExampleN
ative
Pro
jecti
on
Statemachine ExampleTa
bula
r Pro
jecti
on
Statemachine ExampleTe
xtua
l DSL
Pro
jecti
on
Statemachine ExampleJa
va P
roje
ction
Statemachine ExampleRu
by P
roje
ction
Statemachine ExampleTw
o pr
ojec
tions
sid
e/si
de
Statemachine ExampleTw
o pr
ojec
tions
sid
e/si
de
Pension Workbench Example
Text Editing Domain
Pension Workbench Example
Insurance Mathematics Domain
Pension Workbench Example
Pension Contract Rules Domain
Pension Workbench Example
All in one Document
Pension Workbench Example
Symbolically integrated
A Journey…
Are we there?
A Journey…
web email skype
xinglinkedin
[email protected]
http://www.xing.com/profile/Markus_Voelterhttp://www.linkedin.com/pub/0/377/a31
.coordinates