Budapest University of Technology and EconomicsFault Tolerant Systems Research Group

IncQueryLabs Ltd. Itemis AG

XCORE MEETS INCQUERY:HOW THE NEW GENERATION

OF DSLS ARE MADE

Tamás Szabó (itemis AG), István Ráth (BUTE)With contributions from:

Ed MerksGábor Bergmann, Ábel Hegedüs, Ákos Horváth,

András Ökrös, Zoltán Ujhelyi, Benedek Izsó

Overview

Introduction Live graph search for EMF Query-based language engineering Conclusion

EMF-INCQUERY:LIVE GRAPH SEARCH FOR EMF

Queries in modeling tools Important!

oModel managemento Querieso Viewso Transformations

Incremental queries Key observation: model queries are expensive

o Models are large (1-5-10+ M elements)o Queries are complex (attribute value checks +

navigation/joins + iterations)o Industrial example: well-formedness validation of a 4M

element AUTOSAR model takes 30+ minutes• Despite being hand-coded in Java (no OCL, …) Usability issues Productivity issues

How to address this?o Lazy evaluationo Incrementality: calculate once, update afterwards

Comparison Normal queries

o Java, OCL, …

Live (incremental) queries

Query

Model

Evaluator

Results

Model change

Query

Model

Evaluator

Updated results

Model change

Result deltas

Efficient changepropagation

1. Always up-to-date resultswithout model traversal

2. Track changes of your modelin terms of queries

(Re-)traversal necessary ifthe model is changed

EMF-INCQUERY An EMF model query engine for live graph search

• Expressive query language based on graph patterns and Xbase

Language

• Incrementalevaluation based on the Rete algorithm

• Instantaneous response for complex queries over large models

Execution

• Incremental model processing (validation, synchronization, ...)

Tools

http://eclipse.org/incquery

Important use-case: Live abstractions

Complex model

abstract

Computed overlayaka. “View”

Id Label Prop0 Prop1

0 N1 a B

1 N2 c D

Items = SELECT …

Defined by a query

Important use-case: Live abstractions

Complex model

abstract

Computed overlayaka. “View”

Id Label Prop0 Prop1

0 N1 a B

1 N2 c D

2 N3 e F

Items = SELECT …

Defined by a query

Model Modification

Change notification

Query result update

UI update

Id Label Prop0 Prop1

0 N1 a B

1 N2 c D

INCQUERY Viewers

EMF ModelLive

Queries

2. ChangeNotifications

1. Model Modification

Live Queries

DerivedModel

UI

3. Continuous,efficientsynchronization

4. UI updates

Labeled, hierarchic property graph

On-the-fly abstractions over

the model

Visualize things that are not (directly) present in your model Provides an easy-to-use API for integration into your presentation layer

o Eclipse Data Bindingo Simple callbacks

DEMO Using pattern annotations for the specification of

on-thy-fly abstractions Using the INCQUERY Viewers Sandbox for

development and testing Visualizing large graphs with yFiles Using INCQUERY Viewers Extensions APIs in your

own apps

INCQUERY Viewers

VisualizationEMF instance model

Live abstractions: simple example Influence relationships in the Library

W1: Writer

B1: Book

: writers

W2: Writer

B2: Book

: writers: cites

Bx: Book

: citations : citations

: influencedBy

W1

W2

Transitive closure

Derived relationship

QUERY-BASED LANGUAGE ENGINEERING

Xcore Xcore

o Extended textual syntax for Ecore-based metamodeling, built on Xtext

o Fully fledged programming language

o To specify structure + behavior (operations, datatype conversion logic)

o http://wiki.eclipse.org/Xcore

Xbase• Xtext-based programming

language• Compiles to Java• Java type system• Supports advanced constructs

(closures, type inference, operator overloading, …)

• http://wiki.eclipse.org/Xbase

Derived features in Ecore Getters (and setters) implemented with Java Frequently used to transparently integrate

complex queries into the languageo Can also be specified by means other than Java

Nice propositiono Transparent: integrate queries into the model – no

utility classes neededo Extend the expressive power of metamodeling

Many language engineers are reluctant to use them – why?

The functionality challenge Typical modeling tool

architectureo Model: EMFo Controller: ~ EMF.Edito View: e.g. JFace Viewers

Change notifications are essential for this to work

EMF has nice notificationso Notifier - Adaptero … except for derived

features Model

Controller

View

Change notifications

The performance challenge What happens when a getter is called?

Dependencieso The value of “longName” depends on “title” and “writers”o The value of “shortName” depends on “longName” – explicit dependency

• … and thus on “title” and “writers” – implicit dependency

Performance challenge: getters may be called many timeso Make sure they execute quickly – but how?o Cache whenever you can – but how?

class Book {String title = "" // set a default valuerefers Writer[] writersderived String longName get {

return title + " (" + writers.size + ")"}derived String shortName get {

return longName.subString(0,5)}

}dependenciesTougher problem than you

might think:- DFs are used a lot in

complex DSLs- Query complexity can

explode easily

Solution: well-behaving derived features Key idea

o Changes in the value of “name” depend on changes in the values of “title” and “writers”

o Change notifications can be used to implement an efficient caching strategy

o As long as every feature is behaving well• i.e. sends notifications when necessary

But…o How do I discover implicit dependencies?o How do I get/send notifications for derived features?

Live graph search!

DEMO INCQUERY Java API Add-on APIs

o INCQUERY Validationo INCQUERY Viewers

Query integration options for INCQUERY

DEMO INCQUERY Java API Add-on APIs

o INCQUERY Validationo INCQUERY Viewers

Query-based derived featureso Automatically computes and maintains references

and attribute values o Transparent integration with Xcore and (pure) Xbase• Works with traditional genmodeled Ecore too

o Unique feature: well-behaving derived features with efficient and automated notification support

Query integration options for INCQUERY

Scalability? Yes!

• IncQuery can provide near zero response times for complex queries over models up to 14M elements

• Check http://incquery.net/publications/trainbenchmark for more details

Characteristic difference, note

the log scales

Eclipse OCL, SPARQL engines, Neo4j, …

CONCLUSION

What can I do with all this? – query-based live abstractions

Syntax Eclipse technology Pros

Trees, tables, Properties

(JFace viewers)EMF.Edit The real deal:

doesn’t hide abstract syntax

Diagrams GEF, GMF, Graphiti

Easy to read and writefor non-programmers

Textual DSLs Xtext Easy to read and writefor programmers

JFace, Zest, yFiles

Your tool!INCQUERY

ViewersMakes understanding and

working with complex modelsa lot easier

What can I do with all this? – query-based derived features

Boost the power and scalability of Ecore:live graph search tightly integrated into your DSLo Self validating models• 30+ min batch <1s on-the-fly

o Query-based traceability links / language integrationo INCQUERY Viewers “derived EObjects”o << your idea here >>

Get rid of lots of legacy stuffo Legacy model elementso Legacy helper/utility classeso Legacy UI boilerplate

Top 3 cool applications of INCQUERY

Well-formedness validation in a modeling tool for an automotive OEM

Development tools for live dependency analysis of a large software infrastructure model

Model-driven development toolchain for avionics systems,Based on a combination of Eclipse tech and MATLAB/Simulink

Three points to take home

1. Live abstractions are very powerful.2. Use well-behaving derived features to boost your

DSL’s expressive power.3. Xcore+INCQUERY is how the new generation of

DSLs are made

Final points Find all details, examples, demos at

o http://eclipse.org/incqueryo http://incquery.net

Check out the blog posto http:

//incquery.net/blog/2013/10/xcore-meets-incquery-how-new-generation-dsls-are-made-talk-eclipsecon-europe-2013

o Demo projectso Further reading (docs + the theory behind)

Release 0.8 (including all these new features) is scheduled shortly after the Xtext 2.5 release

Your contributions (feedback, forum posts, ideas, bugzillas, patches) are very welcome!