1 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Sustainable Service Design using Xtext

Michael Bischoff – arvato Systems GmbH

2 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Use case: Cross Channel Commerce

?

3 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

aroma® - the arvato Order Management System

4 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

aroma® - the arvato Order Management System

Databases: Postgres, Oracle, HANA, SQL Server

5 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

aroma® - the arvato Order Management System

Databases: Postgres, Oracle, HANA, SQL Server

Application Server: Wildfly, WebLogic

(CDI, JPA, JTA, JMS, JCA)

6 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

aroma® - the arvato Order Management System

Databases: Postgres, Oracle, HANA, SQL Server

Application framework: fortytwo

Application Server: Wildfly, WebLogic

(CDI, JPA, JTA, JMS, JCA)

7 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

aroma® - the arvato Order Management System

Databases: Postgres, Oracle, HANA, SQL Server

Application framework: fortytwo

Application Server: Wildfly, WebLogic

(CDI, JPA, JTA, JMS, JCA)

Product core: aroma®

8 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

aroma® - the arvato Order Management System

Databases: Postgres, Oracle, HANA, SQL Server

Application framework: fortytwo

Application Server: Wildfly, WebLogic

(CDI, JPA, JTA, JMS, JCA)

Product core: aroma®

Customizations (project specific)

9 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

What‘s my API?

10 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

What‘s my API?

KEY QUESTION:

WHAT‘S A VALID REQUEST?

11 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

I want a currency code!

It‘s a String! "Euro" ?

"€" ?

" EUR " ?

"EUR" !

Solution: Validate against Java or ICU „Currency“ class. This is an easy example, sometimes validation requires database lookups. What can we do in the outer API layers (before the request hits the datase...)? ISO 4217 (A3): A currency code is exactly 3 characters long, each of which is an upper case ASCII letter.

12 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

I want a currency code!

Java: not nullable string of length 3 (JSR 303 annotations)

XSD definition:

13 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

I want a currency code!

Java: not nullable string of length 3 (JSR 303 annotations)

XSD definition:

DIFFERENT DESCRIPTIONS PER API

(AND NOT REALLY CONCISE)

14 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL

Xtext based description language for Java data transfer objects Data type support:

all Java primitives all wrapped types objects, enums, immutable binary UUID, BigInteger, BigDecimal, some Joda time types (Java 8 date/time WIP) arrays, Lists, Sets, Maps (map keys limited to Integer, Long, String) enum sets (similar to C bitfields): great for the IoT!

Typedefs Type adapters (similar to XmlAdapter) for types which are not supported natively

15 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Example

16 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Example

Generate JAXB annotations for all classes in this package

17 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Example

Generate JAXB annotations for all classes in this package

Generate JSR303 annotations

18 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Example

Generate JAXB annotations for all classes in this package

Generate JSR303 annotations

enum type with possible single character serialized form („C“, „D“)

19 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Example

Generate JAXB annotations for all classes in this package

Generate JSR303 annotations

Specification of response object

enum type with possible single character serialized form („C“, „D“)

20 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Example

Generate JAXB annotations for all classes in this package

Generate JSR303 annotations

Specification of response object

Parser directives

enum type with possible single character serialized form („C“, „D“)

21 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Example

Generate JAXB annotations for all classes in this package

Generate JSR303 annotations

Specification of response object

Parser directives

enum type with possible single character serialized form („C“, „D“)

This is an XML root element

22 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Generated Output

POJOs (Java 6 compliant) Metadata (enhanced java.lang.Class) fields, getters, setters validation code code to freeze an instance (turn immutable) serialization / deserialization methods (reflection free)

invoke methods of MessageParser and MessageComposer different serializers / deserializers implement these interfaces

equals, hashCode defensive cloning methods

optional: XSDs

23 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Writing some nonstandard CSV

24 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Writing some nonstandard CSV

Create the configuration

25 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Writing some nonstandard CSV

Create the configuration

Create the serializer

26 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Writing some nonstandard CSV

Create the configuration

Create the serializer

Serialize and write the data

27 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Defining a type adapter

28 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Defining a type adapter

target Java type

29 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Defining a type adapter

Converter class specification target Java type

30 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Defining a type adapter

Converter class specification target Java type

desired external representation in serialized form

31 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Example: Defining a type adapter

Converter class specification target Java type

desired external representation in serialized form

32 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Android does not like XML a lot Generated JAXB annotations already cause issues

Delete with JARJAR or ProGuard? Compile run with suppressed JAXB annotations possible in Bonaparte

Bonaparte provides a resource friendly serialization format Generalization of CSV Immune to encoding conversions (ISO 8859 / UTF-8) or Win/UN*X end of line Upwards compatible changes: integral => fractional, enum token to string, added optional fields (also in superclasses) etc. Support for cyclic object nesting! Preserves identity of component objects

The Android Bridge

33 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Extension Mechanisms

34 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Extension Mechanisms

Directives for add on DSL – only stored in meta data

35 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Extension Mechanisms

Directives generating Java annotations – with or without parameter. Used by Xtend Active Annotations (DSL generates Java annotation for class or field)

Directives for add on DSL – only stored in meta data

36 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

The Bonaparte DSL – Extension Mechanisms

Directives generating Java annotations – with or without parameter. Used by Xtend Active Annotations (DSL generates Java annotation for class or field)

Directives for add on DSL – only stored in meta data

NO MONOLITHIC DSL!

- SEPARATION OF CONCERNS - PROJECT MODULE REQUIREMENTS

37 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Xtext DSLs + Free syntax definition + IDE integration + Solid basis for grammar and validations - Somewhat longer iteration cycle (build / install cycles) One DSL across multiple projects, keep stable

Xtend Active Annotations + Fast turnaround times + Ideal for straightforward code generation + Java / Xtend crossreferences in a single project (maven module) Active Annotations developed as part of the application Examples: Creation of data access objects (DAOs), mappers between DTO and generated JPA entity classes

Xtext and Xtend Active Annotations – A Dream Team!

38 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Principles of Sustainable Service Design according to Pleus, 2015: Technology-agnostic service definition Unified request/response Consequent contract first Technology bindings

Sustainable Service Design

39 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Use DSL to define logical data structure / syntactic validation While the DSL is closely Java related, code generation for other languages would be possible (a proprietary mapper to COBOL has been implemented for a subset of data types)

End points centrally implemented / created by Active Annotation all request objects are of type ServiceRequest, parameters inherited from RequestParameters all response objects inherit from ServiceResponse project specific additional fields via further inheritance

Implement specific formats / protocols via library xls(x), CSV, JSON, http parameters, Bonaparte formats, fixed field width formats, ... Integrate with vert.x 2 / vertx.3 event bus, Apache Camel, netty 4 In some cases, small enhancements of the DSL were required (Java Externalizable,

Hazelcast‘s Portable and Identified Data Serializable formats)

Sustainable Service Design as in Bonaparte / aroma

40 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Part II: Persisting your Data

41 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

DSL to generate JPA 2.x entity classes / DDL statements from DTOs DSL based on Bonaparte DSL Databases currently supported: Postgres 9.x ff, Oracle 11 ff, MS SQL Server 2012 ff, SAP HANA Included definition for tablespaces, synonyms, grants Surrogate keys as well as natural / composite primary keys Definition of unique and non unique indexes Support for @{One,Many}To{One,Many}, @Embeddable and @ElementCollection JPA 2.1 @AttributeConverters Serialized object storage (noSQL within SQL) User types for Joda time & serialized objects supported for Hibernate as well as EclipseLink List unrolling into enumerated fields Creation of multitenancy discriminators Timestamp / user ID tracking fields defined per category of entities Additional history tables (and corresponding triggers in Oracle PL/SQL) Mapping of enums to tokens

Addon DSL: JPA Persistence

42 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

JPA Persistence DSL Example

43 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

JPA Persistence DSL Example

Properties shared by multiple entities

44 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

JPA Persistence DSL Example

Inheritance type (joined or separate tables)

Properties shared by multiple entities

45 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

JPA Persistence DSL Example

Inheritance type (joined or separate tables)

Properties shared by multiple entities

multitenancy discriminator (homegrown, since not yet standardized in JPA 2.1)

46 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

JPA Persistence DSL Example

Inheritance type (joined or separate tables)

Properties shared by multiple entities

multitenancy discriminator (homegrown, since not yet standardized in JPA 2.1)

Shared columns (timestamp / user IDs) for auditing purposes

47 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

JPA Persistence DSL Example

Inheritance type (joined or separate tables)

Properties shared by multiple entities

multitenancy discriminator (homegrown, since not yet standardized in JPA 2.1)

Shared columns (timestamp / user IDs) for auditing purposes

JPA entity listener

48 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

JPA Persistence DSL Example

Inheritance type (joined or separate tables)

Properties shared by multiple entities

multitenancy discriminator (homegrown, since not yet standardized in JPA 2.1)

Shared columns (timestamp / user IDs) for auditing purposes

JPA entity listener

Generated Java class has more than 700 lines of code

49 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Extension concept Use Eclipse‘s extension points to add more plausi checks & code generators as indepent plugin

Be Polyglot! C#, C++ output

Message Version & Migration Support Support for incompatible message changes

(for example deletion of fields)

Swagger 2.0 Addon DSL IDE support for service description

Binaries on Maven Central

Future Plans

50 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Bonaparte DSLs and support Libraries https://github.com/jpaw (Apache 2 licensed)

aroma Order Management system http://aroma.arvato-systems.de/ (commercial)

Resources

51 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Questions

52 | commerce (NMA) | arvato Systems GmbH | May 19th, 2015

Contact

arvato Systems GmbH

Dr. Michael Bischoff

An der Autobahn 200

33333 Gütersloh

Germany

michael.bischoff@arvato.com

Phone: + 49 52 41/ 80-74413

Mobile: + 49 160 880 6 880

www.arvato-systems.com