Mastering Differentiated MDSD Requirements at Deutsche Boerse AG
-
Upload
heiko-behrens -
Category
Technology
-
view
1.076 -
download
1
description
Transcript of Mastering Differentiated MDSD Requirements at Deutsche Boerse AG
-
itemis AG 2009 All rights reserved
Mastering differentiated
MDSD requirements at
Deutsche Brse AG
Code Generation 2009
Heiko Behrens (itemis)
Karsten Thoms (itemis)
-
The project at Deutsche Brse
! Projects delivers a Global Trading System
! First (internal) user is ISE in NY
! Scalable, Distributed System
! Goes live in 2011
-
Structure of this talk
! 8 timesMotivation ! Applied Approach
! Inbetween: Demo of discussed ideas
! In the end: Some discoveries we made
http://www.bildtankstelle.de
-
Application ofCode Generation
in this project
-
Messaging Model
Structures
Messages Docu
HTML,
TextC++
Factories
Messaging Generator
Msg/Struct.
Adapters
Reference Data Model
Reference Data
Generator
Configuration Model
JPA Entities
Report
Java HTML
IML
Structures
Messages
Java
DAOs
Listener
Container Model
Containers
Filters
Boost
Operator
C++
XML
Model
Data
Access
Generator
XML
Transactional
Model
Queries
DAOs
C++
DB
Service
Generator
Hibernate
SQL
DDL Scripts
JAXB
XML
WSFacade
XSD
Val.List.
UnitTest
Validation
Listeners
WSFacade
DTOs
Persistence
Descriptor
IML
Datatypes
Model
UOW Exec.
Bean
Ref Data
Mngr Class
Global key
functionsDDL Scripts
SQL
UnitTest
Templates
DDL Scripts
(MySQL)
SQL
Excel Model
System Test
Generator
Python
Tests
Messages
Structures
Messages
Structures
C++ Python
Scripting Generator
Factories
Structures
Messages
.Net
.Net
Generator
OFi
Editor
Msg Client
Descriptor
-
Artifact sizes
Reference Impl.Sources (manual)Tests (manual)ModelTransformationGenerated
2.747 units
Actual numbers had to be replaced by
fictive units
-
Guess what!
Motivation:Code Generation
-
Why Code Generation?! Positive experience with former projects
! Prime example: mapping between messages and structures sending with a compact binary wire protocols
ComponentComponent
B1
B2
Structure B
S1
S2
SubStructure S
A2A3A1
Component
B1
B2
Structure B
S1
S2
SubStructure S
S2B2S1B1
-
Why openArchitectureWare?
Former times:
! Home-grown solution based on graphical modeling and proprietary tools
! Many different post-processors such as XSLT and Perl
Evaluation results of oAW for GTS
! Supports textual modeling
! Has mature tool support
! Works with different model types and output formats
-
UMLXpand
JavaC++Text
Different Input
XMIXML XMIXML
Same Transformation Language, Different Output ,
-
When in Rome do as the Romans do
Motivation: Address existing experience and perception
-
Modeling in UML
! Years of experience for modeling entities graphically (ER-Diagram)
! Established Tool (MagicDraw)
! Proven Technology (Profiles, Tagged Values)
! Great for overview perspectives
-
UML Reference Model
-
Textual Modeling
! Years of experience ;)
! Established tool support (grep, editor, mail, svn)
! Highly customizable ! DSL
! Inexpensive metamodel evolvement
! Great for detailed modeling
-
Textual Model
-
!""#$%&!$'!($)#*"
Motivation:Reduce Complexity of Modeling
-
Reducing Model Complexity
! Following Convention-over-configuration paradigm for modeling
! Only model necessary information
! Complete model according to conventions by M2M transformation
-
Convention over Configuration
can be derived
required model information
-
uml::Model modify (uml::Model this) : ... -> // Search for unnamed association ends and give them a default name this.eAllContents.typeSelect(Property) .select(x|x.association!=null).nameUnnamedEnds() -> ...
/** * This function sets a default name for an association's end property if the property * has no name and it is navigable. */ Void nameUnnamedEnds(Property this) : if !isNamed() && isNavigable() then ( ("setting default name for association end "+opposite.type.name+" -> "+type.name).info() -> setName(computeDefaultName()) ) -> this;
Convention: Navigable association ends withoutname get the name of the referred Entity. Use plural for to
many relationships.
-
Convention over Configuration
-
uml::Model modify (uml::Model this) : ... -> // Search for unnamed association ends and give them a default name this.eAllContents.typeSelect(Property) .select(x|x.association!=null).nameUnnamedEnds() -> ...
private create uml::Property createTechnicalId(RDMProfile::Entity entity): ("Creating technical id attribute for entity "+entity.name).debug() -> entity.ownedAttribute.addToFront(this) -> this.setName("id") -> this.setType(findDataType(Long) -> this.setLower(1) -> this.setUpper(1) -> this.setIsUnique(true) -> this.setVisibility(uml::VisibilityKind::^private) -> this.applyStereotype("RDMProfile::Key") -> this.setTaggedValue("RDMProfile::Key", "keyType", RDMProfile::KeyType::PK)-> this.applyStereotype("RDMProfile::Field") -> this.setTaggedValue("RDMProfile::Field", "uniqueGroup", "444")-> this;
Convention: All entities without attribute get a new property with name id of type Long, and Stereotype
is applied.
-
Combination of model types
Domain modelUML2platform independent
entity Market
{
relatedTo MarketAbstract with EAGER loading and cascade with PERSIST
relatedTo MarketGroup with LAZY loading and cascade with PERSIST
}
Configuration modeltextual DSLplatform dependent
-
entity Market
{
relatedTo MarketAbstract with EAGER loading and cascade with PERSIST
relatedTo MarketGroup with LAZY loading and cascade with PERSIST
}
public class Market implements RDSEntity, Serializable { ... @OneToMany(mappedBy = "market", cascade = CascadeType.PERSIST) @Basic(fetch = FetchType.EAGER) private List marketAbstracts = new ArrayList(); @ManyToMany(cascade = CascadeType.PERSIST, targetEntity = MarketGroup.class, mappedBy = "markets") @Basic(fetch = FetchType.LAZY) private List marketGroups = new ArrayList();
Combination of model types
-
Avoidable Accidents
Motivation:Detect errors early
(c) Ernest von Rosen, www.amgmedia.com
-
context R
DMProfile:
:EntityRel
ation
(memberEnd
.select(e|
e.upper ==
-1).size
== 2)
WARNI
NG "Many t
o many rel
ation " +
memberEnd.
type.name.
info() +
as owner o
f the rela
tion (owne
rOfAssocia
tion is no
t set)!" :
ownerO
fAssociati
on != null
;
context u
ml::Associ
ation if m
emberEnd.t
ype.select
(e|e.isAct
ive()).siz
e>1 && (me
mberEnd.se
lect(e|
e.upper ==
-1).size
== 2) && m
emberEnd.s
elect(e|e.
aggregatio
n.toString
()!="none"
WARNI
NG "Many t
o many rel
ation " +
memberEnd.
type.name.
info() + "
must be o
f type Ent
ityRelatio
n
and define
one of th
e entities
as owner
of the rel
ation (ste
reotype no
t applied)
!"
RDMPro
file::Enti
tyRelation
.isInstanc
e(this);
context P
roperty if
class.isA
ctive() &&
(associat
ion == nul
l)
ERROR
"Property
" + name
+ " in ent
ity "+clas
s.name+" r
eferences
an entity,
but is n
ot an
associatio
n!":
!RDMPr
ofile::Ent
ity.isInst
ance(type)
;
cont
ext Proper
ty if clas
s.isActive
() && asso
ciation ==
null && ^
default!=
WARNI
NG "Proper
ty "+loc()
+" has an
empty stri
ng (not nu
ll) as def
ault value
"
="";
cont
ext Proper
ty if clas
s.isActive
() && RDMP
rofile::En
tity.isIns
tance(clas
s) && asso
ciation!=
ERROR
"Class "+
type.name+
" must be
stereotype
d as Entit
y, since i
t is assoc
iated by E
ntity
"+class.na
me+" in it
s property
"+name :
RDMPro
file::Enti
ty.isInsta
ncef(type)
;
context P
roperty if
class.isA
ctive() &&
RDMProfil
e::Entity.
isInstance
(class) &&
associati
on==
ERROR
"Only Fun
ctional Da
tatypes or
Functiona
l Enumerat
ions are a
llowed as
property t
ype.
Violated f
or "+loc()
:
RDMPro
file::Func
tionalData
Type.isIns
tance(type
) ||
RDMProfile
::Function
alEnumerat
ion.isInst
ance(type)
;
Detecting errors early
! Check constraints implemented for
! Every assumption or modeling restriction
! Modeling error that lead to generator or artifact errors (e.g. Whitespace in names)
! Inter-model consistency checks
! Risk of modeling errors decreased
-
context uml::Association if memberEnd.type.select(e|e.isActive()).size>1 && (memberEnd.select(e|e.upper == -1).size == 2) && memberEnd.select(e|e.aggregation.toString()!="none").size == 2 WARNING "Many to many relation " + memberEnd.type.name.info() + " must be of type EntityRelation and define one of the entities" +
" as owner of the relation (stereotype not applied)!" : RDMProfile::EntityRelation.isInstance(this);
context Property if class.isActive() && (association == null) ERROR "Property " + name + " in entity "+class.name +
" references an entity, but is not an association!": !RDMProfile::Entity.isInstance(type);
context Property if class.isActive() && association == null && ^default!=null WARNING "Property "+loc() +" has an empty string (not null) as default value" : ^default.trim()!="";
Constraints with Check language
-
Integration
! Static Typing while developingchecks and transformations
! IDE integration for workflows invocation
! Continuous Generation (server-side)CMake / Ant / Maven
! Automated execution after SVN checkout
-
Demo! Graphical EMF model
! Enhanced with textual DSL (with Xtext)
! Validation
! Code Generation
! Changes textual DSL
-
Standing on the shoulders of giants
Motivation:Reusing generator aspects
-
Cartridge adaption! Using OS cartridges as
closed package
! Required model structure created through M2M transformation
! Required changes added non-invasive through Xpand/Xtend AOP support
! Redefined functionality concentrated in a small adapter cartridge project
Annotation
Template
EntityClass
Template
Mapping
Template
Attribute
Template
Class
Template
Operation
Template
Fornax Hibernate Cartridge
Fornax JavaBasic Cartridge
Aspect
Template
M2M
Transformation
Adapter
Workflow
Cartridge
Workflow
Cartridge
Workflow
Fornax Adapter Cartridge
DTO
Template
WebService
Template
UnitTest
Template
Project Cartridge
Cartridge
Workflow
-
uml::Model modify (uml::Model this) : this.applyProfile(getPersistenceProfile()) -> // Create real enumerations for FunctionalEnumerations. This must be done first since // we map datatypes after that this.eAllContents.typeSelect(RDMProfile::FunctionalEnumeration).adaptEnumeration() -> entities().adaptEntity() -> adaptEmbeddedKeys() -> this;
RDMProfile::Entity adaptEntity (RDMProfile::Entity entity) : debug("Adapting entity: "+ entity.name) -> entity.applyStereotype("Persistence::Entity") -> entity.setTaggedValue("Persistence::Entity", "tableName", entity.name.asTableName())-> entity.testAndAssignInheritanceStrategy()-> entity.attribute.select(a|a.association==null).adaptAttribute() -> entity.attribute.select(a|a.association!=null).adaptRelation() -> entity;
//In case the entity extends another class, assigne the persistence stereotype for //strategy on the uml::Generalization//TPC="TABLE_PER_CLASS" | J="JOINED" | ST="SINGLE_TABLE";Void testAndAssignInheritanceStrategy(RDMProfile::Entity this): if !general.isEmpty then assignInheritanceStrategy(); Void assignInheritanceStrategy(RDMProfile::Entity this): let gener = generalization.first(): gener.applyStereotype("Persistence::"+getInheritanceStrategyFromDardl().mapToStrategy()) -> if(getInheritanceStrategyFromDardl() == "SINGLE_TABLE") then ( gener.setTaggedValue("Persistence::TablePerClassHierarchy",
"discriminator_value", getDiscriminatorValue())-> gener.setTaggedValue("Persistence::TablePerClassHierarchy",
"discriminator_superClass_column", gener.general.name+"_type")-> gener.setTaggedValue("Persistence::TablePerClassHierarchy",
"discriminator_superClass_type", "INTEGER") ) ;
UML2 model modification with Xtend
-
around org::fornax::cartridges::uml2::javabasic::extensions::DataType::NormalizedDefaultValue(uml::Property property): internal_getNormalizedDefaultValue(property);
private String internal_getNormalizedDefaultValue (uml::Property property) : JAVA gts.ise.refdata.oaw.fornax.util.Extensions.getNormalizedDefaultValue(org.eclipse.uml2.uml.Property);
AROUND org::fornax::cartridges::uml2::javabasic::templates::Documentation::documentation FOR uml::Classifier targetDef.proceed() EXPAND JPAAnnotationsENDAROUND
DEFINE JPAAnnotations FOR Persistence::Entity @javax.persistence.Entity @javax.persistence.Table(name="asTableName()") EXPAND InheritanceAnnotations @javax.persistence.NamedQueries({ @javax.persistence.NamedQuery(name="findnames", query="SELECT e from name e") })
IF !isAbstract @javax.persistence.EntityListeners (getQualifiedPackageName("ListenerOnce").nameValidationListener.class) ENDIF EXPAND UniqueConstraintsENDDEFINE
AOP with templates and functions
-
Arange your ideas
Motivation:Managing Manual Code
-
Sucessfully applied MDSD best practices
! Separate generated from manual code
! Avoid check-in generated code
BaseClass
AbstractA AbstractB
ConcreteBSome
Manual
Code
ConcreteA
src-gen
always generated
src-once
generated once
src
manually written
BaseClass
ConcreteB
AbstractB
ConcreteCSome
Manual
Code
ConcreteA
AbstractD
ConcreteD
-
Adjusted Folder Layout
-
Design your tools
Motivation: Managing Generator
Complexity
-
Reference Implementation
! Manually implemented
! Far more than a prototype!
! Compilable, deployable, executable, testable
! Covers every architectural concept
! Code seperated in to-be-generated and manual code
-
Reference Model
! Contains every supported modeling concept
! Not necessary excerpt from the real domain, but helps for understanding
! Try to minimize the model size
-
Testing the reference
! Reference Implementation is tested intensively and automatic
! Unit tests and integration tests with high coverage
! Tests against generated artifacts
! Real application does not need to be tested in the same extend
-
generated from reference model
manually implemented
reference
-
EntitiesAttributesAssociationsModel metrics
-
Motivation: Optimizing Generator Speedand Readability
-
Refactorings / Profiling
! Reference Implementation/Unit-Tests ensure valid output
! Static Typing of Xpand supports development
-
HTML report
-
HTML report
-
standard format (gprof)
-
standard format (gprof)
-
standard format (gprof)
-
Most significant problems
! Repeated calls of functions
! Suboptimal algorithms/data structures
! Branches with best-case scenarios
! Profiling reduced generation timefrom 28 min downto 3 min
-
Discoveries
-
Value of Cartridges
! Project benefited from cartridges developed by OS community
! But: Largest part of the code generators are totally project dependent
-
Early adopters
! Using the bleeding edge of Eclipse Modeling was challenging
! But: Continuous improvement of used technologies thanks to direct feedback with OS community
! And: Significant performance and stability gains since early adoption
-
Learning curve
! Establishing MDSD and tools was hard for unexperienced developers at the beginning
! But: Coaching helped to fill the gap fast
! And: Now developers educate others to use MDSD tooling
-
Reference Implementation
! Reference Models and Reference Implementation are invaluable
! Reducing the complexity
! Enable refactoring
! But: Effort to create and maintain a real Reference Implementation quite high
-
Personal blogshttp://www.1160pm.net http://kthoms.wordpress.com http://www.itemis.com