From Programming to Modeling And Back Again

Programming Modeling

fromto

…and back again

Markus VoelterIndependent/itemis

[email protected]

A Journey…

„Houston,we have a

Problem!“

Programming Languages

C# C++ Java

Python

GroovyRubyErlang

CFortran

Programming Languagesare not

enough.expressive

C# C++ Java

Python

GroovyRubyErlang

CFortran


enough.high-level

C# C++ Java

Python

GroovyRubyErlang

CFortran


enough.abstract

C# C++ Java

Python

GroovyRubyErlang

CFortran


enough.domain-specific

C# C++ Java

Python

GroovyRubyErlang

CFortran

Programming Languagesare notenough.

C# C++ Java

Python

GroovyRubyErlang

CFortran

Programming LanguagesFormats

C# C++ Java

Python

GroovyRubyErlang

CFortranHTML

XMLCSSyaml

json


C# C++ Java

Python

GroovyRubyErlang

CFortranHTML

XMLCSSyaml

json JEEWPF

Rails

Django

JMS

lift

Frameworks


C# C++ Java

Python

GroovyRubyErlang

CFortranHTML

XMLCSSyaml

json JEEWPF

Rails

Django

JMS

lift

Frameworksare not enough.

not enough?for what?

Modeling… Higher Level… Domain Specific

Interpretation… Code Generation

Concepts & Notations

Modeling… Higher Level… Domain Specific

Interpretation… Code Generation

Concepts & Notations

Solves theProblem!

Modeling Tools Programming Tools

!=

Different Worlds

Modeling ToolModeling Tool

!=

Different Worlds

Different WorldsMix Models and Programs

Different WorldsAST Navigation & Query

Mix Models and Programs



Integration of 3GL code



Integration of 3GL codeCode Constraints

pareCom

Domain SpecificNotations

andAbstractions

LibrariesFrameworks

(Fluent) APIs

Flexible! Limited!

GraphicalTextual

FormsTables

TextualTrees

Limited!Flexible!

CustomizeGenerator

orInterpreter

Limited!Flexible!

ReflectionMeta ProgramsOpen Compilers

Define customQueryorNavigate

Limited!Flexible!

AST APIsStatic Analysis

Transformor

Regex

CustomValidationorError Checks

Limited!Flexible!

IDE pluginsStatic AnalysisOpen Compilers

DifferentRepresentations

andProjections

Limited!Flexible!Text is TextCode FoldingTree ViewsVisualizations

Limited!Flexible?Python-to-C-likeInternal DSLs

Mixingand

ComposingLanguages

Embed-As-StringSpecific: LINQ

Mature!Brittle!

IDE Support

Scalable

Modeling Tools…!?

Usable

Mature!Brittle!

Testing

Debugging

?

Refactoring

Mature!Brittle!

Branching

VersioningDiff, Merge

for some tools…

Gets theGets some

some people doubt that…

Job Done!Jobs done.everybody agrees…

Whythe difference?

History?

Modeling Programming

Modeling Programming

… (Mostly) Textual Notations

… Concrete Syntax Storage

… (Fancy) ASCII Editors

… Read-Only Visualizations

Modeling

… (Mostly) Graphical Notations

… Abstract Syntax Storage

… Projecting Editors

… Different editable views for model

Programming





Whythe difference?

It is time for …

… a Different Perspective

Programmingthe way we do

Modeling?

Modelingthe way we do

Programming?

Modeling ProgrammingProgramming Modeling

====

We don‘t want to

we want tomodel,

program!

We don‘t want to

we want tomodel,

program!… at different levels of abstaction… from different viewpoints… integrated!

We don‘t want to

we want tomodel,

program!… with different degrees of

… with suitable notations… with suitable expressiveness

domain-specificity

We don‘t want to

we want tomodel,

program!precise and tool processable

And always:

Where do we go from here?

A vision forprogramming


EnablingTechnologies



AvailableTooling

A vision forProgramming


are not

enough.MODULAR


are not

enough.COMPOSABLE


are not

enough.CONFIGURABLE


are not

enough.ADAPTABLE

Programming Language Syntax

is not

enough.FLEXIBLE

L

with many first class concepts!

Big Language?

L

Small Language?

and poweful conceptswith a few, orthogonal

my L

a b c

d e f

g h i

j k l

Modular Language

composable conceptswith many optional,

Like frameworks and libraries,

Modular Language

Like frameworks and libraries,

but with syntaxand IDE support

Modular Language

Not a new idea…

Growing A Language(Guy L Steele)

LanguageWorkbench

(Martin Fowler)

LanguageWorkbench

(Martin Fowler)

Freelydefine

integratethem

languages and

?

LanguageWorkbench

(Martin Fowler)

usepersistent

abstractrepresentation

LanguageWorkbench

(Martin Fowler)

language ::=schema

editorsgenerators

++

?

LanguageWorkbench

(Martin Fowler)

editingprojectional

LanguageWorkbench

(Martin Fowler)

persistincompletecontradictory

information

or

LanguageWorkbench

(Martin Fowler)

powerfulediting

testingrefactoring

debugging

language definitionIDE definition

implies

+groupware

LanguageWorkbench

(Martin Fowler)

support for„classical“

programming„classical“

modelingand+

Syntaxprimarilytextual


with more symbolsthink: mathematics


sometimesbox&line style


sophisticatedvisualizations

Viewpoints

Viewpointssuitableabstractions

notationsfor each

and

ViewpointsIntegrated

references

transitions

symbolic

seamlessand

via

ViewpointsBusiness

custompurpose-built

create/include

ViewpointsCustomNotations

businessexpert integration

real

Business

ViewpointsCustomNotations

businessexpert integration

real

Business

But that‘s another talk…

ViewpointsTechnical

predefinedlibrary

configure

Example Languages

matricesAdding

embeddedto C in an

environment.

(Seemingly)

Simple Example

Currently:

1 •Declare Data Structures in XML

2 •Generate Headers

3 •Implement manually in C

Currently:

Matrices

and generator

not supportedin XML format

Currently:

Tool team

… a lot of work

would have todo the extension

… busy… one central tool

Currently:

No real

… type checks

compiler supportin the resulting C code

… operator overloading… generics (matrix<int>)… matrix syntax?

Better Solution

Better Solution

types

matrixgeneric

vector and

Better Solution

matrixreal

literals vector

and

Better Solution

highlightssyntax

vectors for

and matrices

Better Solution

operator overloading

Better Solution

operator overloading

staticoptimization… symmetrical matrices… identity matrix… diagonal matrices

Better Solution

math notation

Better Solution

used only bythose who

separatea

module language

really need it

In addition: PLE Variability

variabilityannotating

expressionsto

arbitrary regions



expressionsto

arbitrary regions

statically checked



expressionsto

arbitrary regions

project variant

more later…(tool examples)


AdvancedParser

Generators

Modeling as Programmig





Custom Syntax

GraphicalTextualSymbolic++

IDESupport

TeamworkDebuggingCustom Editors

CompleteSymbolicIntegration

Goto DefFind RefsRefactoring

Xtext-like Toolsprovide editor support

TeamworkCustom Editors

Goto DefFind Refs

Refactoring

UnicodeLimited to

how to handlenon-character symbols

TextGraphics !=

two worlds…separate editors… per syntax/viewpoint… models can still be ref integrated

What if!combine languagessame seq<Char> as

different tokens?

Ambiguous!Not Parsable.

Bad Luck.

Traditionally:Language ::=

Token* | Rule*;


Token* | Rule*;

Token ::= Character*;


Token* | Rule*;

Token ::= Character*;

Rule ::= Token* | Rule;


Token* | Rule*;

1. Global Tokenization2. Grammar Recognition

2 Phases:

(aka Scanning)

(aka Parsing)

Modern:Language ::=

Rule*;Rule ::= Char* | Rule*;

Grammar Recognition1 Phase:

directly on char*

ScannerlessParsing

No Ambiguity

Always Parsable.when combining languages

(well, almost. There are some corner cases…)


ProjectionalEditing

Parser-basedtext… to tree… to text

Projectionaltree… to text-lookalike (editor)… to other trees … [*]… to text

Programming as Modeling

… (Mostly) Graphical Notations




Programming as Modeling

… (Mostly) Graphical Any kind of Notations




Language

There‘s no parsing.

Unique Language Element Identity.

Unlimted language composition.

Composition

Textual

Graphical

Semi-Graphical

treated the samecan be mixed

like ASCII }box & line

mathematical

FlexibleNotations

AutomaticIDE Extension

tool support is inherentfor languages build with

projectional tools

language definitionIDE definition

implies

MultipleNotations

… for the same concepts

e.g. in different contextsor for different tasks

PartialProjections

… different views… for different roles/people… only a particular variant

Storage!= Schema

… independent of language schema!variability annotations

… store arbitraty meta datachange logconflicting information

… „aspects“, overlay

ProgramsLive

think: spreadsheet

a change to one part of programcan lead to (dependent) changesin other parts

Tree Editing

… try to make it feel like text

… takes some getting used tobut: for more flexible notations a more general editing paradigm is needed

… is different from editing text

InfrastructureIntegration

… diff/merge must be in tool

… existing text tools don‘t work

… storage is not text

ProprietaryTools

… no interop

… no standards

AvailableTooling

AvailableTooling

EclipseXtext

http://eclipse.org/modeling

http://eclipse.org/xtext

Eclipse Xtext

Building Textual Editors

Xtext: Specify Grammar

Xtext: Gen. Meta Model

Xtext: Constraints

Xtext: Generated Editor

Code Completion


Syntax ColoringCustom Keyword Coloring


Realtime ConstraintValidation


CustomizableOutlines


Code FoldingXtext: Generated Editor

Goto Definition Find ReferencesCross-File ReferencesModel as EMF


PlatformArchitecture

DSL

As you understandand develop

yourPlatform

Architecture…

Develop a language to express it!

Language resembles

architectural concepts…

Express the applications

with the language.

Architecture As LanguageAnother Story…

DEMO I

Building DSLs with Eclipse Xtext

AvailableTooling

AvailableTooling

Jetbrains‘

ProgrammingMeta

System

IntelliJ IDEAResharper

also do…

Q3 2009released in

1.1 RC1currently

Apache 2.0licensed under

Build new standalone DSLs

Build new standalone DSLsBuild DSLs that reuse parts

of other languages


of other languages

(MPS comes with BaseLanguage)extend base language

Java++


of other languages

(MPS comes with BaseLanguage)extend base languagebuild DSLs that reuse parts

of BaseLanguage

Java++

Java + Extension

Language Extension Example

Java

ReadWriteLock l = …l.readLock().lock();try { //code} finally {

l.readLock().unlock();

}

Java + Extension


Old

Java

ReadWriteLock l = …l.readLock().lock();try { //code} finally {

l.readLock().unlock();

}

Java + Extension

ReadWriteLock l = …lock (l) { //code}


Old New

Structure Editor Typesystem Generator

Language Extension ExampleResult behaves like a native

base language construct

Language Extension ExampleTranslated to regular Java code

based on the generator

package jaxdemo.sandbox.sandbox;

import java.util.concurrent.locks.Lock;

public class DemoClass {

private Lock lock;

public DemoClass() { try { this.getLock().lock(); SharedResouce.instance().doSomething(); } finally { this.getLock().unlock(); } }

private Lock getLock() { return this.lock; }

}

Example LanguagesUI Language

Example LanguagesHTML Templates

Example LanguagesPersistent Classes

DEMO I

Building DSLs with JetBrains MPS

AvailableTooling

AvailableTooling

Intentional SoftwareDomain

Workbench

Dec 15, 2008Version 1.0 released in

currently at 1.4

CommercialProduct

Eval availableupon request

Statemachine ExampleN

ative

Pro

jecti

on

Statemachine ExampleTa

bula

r Pro

jecti

on

Statemachine ExampleTe

xtua

l DSL

Pro

jecti

on

Statemachine ExampleJa

va P

roje

ction

Statemachine ExampleRu

by P

roje

ction

Statemachine ExampleTw

o pr

ojec

tions

sid

e/si

de

Pension Workbench Example

Text Editing Domain


Insurance Mathematics Domain


Pension Contract Rules Domain


All in one Document


Symbolically integrated

A Journey…

Are we there?

A Journey…

web email skype

xinglinkedin

[email protected]

http://www.xing.com/profile/Markus_Voelterhttp://www.linkedin.com/pub/0/377/a31

.coordinates

From Programming to Modeling And Back Again

Technology

Transcript of From Programming to Modeling And Back Again