THE NEW CMW-RDAMOTIVATION, DESIGN AND TECHNICAL CHOICES
BE/CO TC28TH JUNE 2012, GENEVA
A. Dworak
BE-CO-IN, CERN
2
Agenda
28th June 2012
Sum up of the previous report Middleware prototyping
Transport Serialization
Design concepts Keep proven solutions Modularity Data Model Transport Client Server
Work progress
Andrzej Dworak, The new CMW-RDA
Andrzej Dworak, The new CMW-RDA 3
Previous reports
28th June 2012
“Middleware Trends and Market Leaders 2011” 14/10/2011, ICALEPCS 2011 20/10/2011, BE-CO-IN TC
Andrzej Dworak, The new CMW-RDA 4
Previous reports
28th June 2012
“Middleware Trends and Market Leaders 2011” 14/10/2011, ICALEPCS 2011 20/10/2011, BE-CO-IN TC
Requirements gathering, user interviews, internal reviewsShortcomings of the system, missing functionality
Andrzej Dworak, The new CMW-RDA 5
Previous reports
28th June 2012
“Middleware Trends and Market Leaders 2011” 14/10/2011, ICALEPCS 2011 20/10/2011, BE-CO-IN TC
CORBA: C++ and Java implementations differ Shrinking community Lack of new releases and bug fixes Expensive on resources Complex error prone API Problems with asynchronous and
non-blocking communication No support for pub/sub Missing support for versioning or
flexible interfaces (hard to upgrade)
Current design: Monolithic (hard to maintain, extend
and test) Not scalable (poor parallelization) Not optimal design makes it
unnecessarily heavy and slow Lack of QoS, message priorities Missing server-side management of
client connections After all the years and fixes it is BIG
Ma
jor
pro
ble
ms
with
CO
RB
A
and
cu
rre
nt c
ode
ba
se u
sin
g it:
Fro
m u
sers
’ p
ersp
ectiv
e
Service instability Client may influence Server Fails when overloaded Does not scale Impossible to optimize the usage
Unnecessarily limited (e.g. connection #), slow and heavy.
Needs workarounds (e.g. Proxy) Unfriendly API Lack of full diagnostics
===========================================================
Andrzej Dworak, The new CMW-RDA 6
Previous reports
28th June 2012
“Middleware Trends and Market Leaders 2011” 14/10/2011, ICALEPCS 2011 20/10/2011, BE-CO-IN TC
Andrzej Dworak, The new CMW-RDA 7
Previous reports
28th June 2012
“Middleware Trends and Market Leaders 2011” 14/10/2011, ICALEPCS 2011 20/10/2011, BE-CO-IN TC
Andrzej Dworak, The new CMW-RDA 8
Previous reports
28th June 2012
“Middleware Trends and Market Leaders 2011” 14/10/2011, ICALEPCS 2011 20/10/2011, BE-CO-IN TC
IceYAMI4
Presented so far
Andrzej Dworak, The new CMW-RDA 9
Middleware prototyping
28th June 2012
More testsScalabilityCorrectnessError handlingExceptional situations
Prototyping of the system parts
IceYAMI4
USE
Andrzej Dworak, The new CMW-RDA 10
Serialization Prototyping
28th June 2012
Necessary to transport data between different architectures Currently self-describing serialization
Based on CORBA (on top of IDL) Stay with self-describing serialization, chose better library
Used so far in RDA No need to manage IDLs Easier to upgrade or change the protocol Need to encode type description
Slightly slower Slightly bigger output
Review of +10 serialization libraries Testing and prototyping with:
Google Protocol Buffers MessagePack Apache Avro Compared with CORBA
MessagePackUSE
Andrzej Dworak, The new CMW-RDA 11
More details at:
28th June 2012
MiddlewareGeneral review and tests
http://wikis/display/MW/CMW+ReviewPresentation of the review and test results
http://wikis/download/attachments/49218519/FRBHMULT05_talk.pdf?version=1&modificationDate=1328608532000
More details on the 1st step also in A. Dworak “Middleware trends and market leaders 2011”
More details on the 2nd step also in A. Dworak “The new CERN Controls Middleware”
SerializationPresentation with review details and test results
http://wikis/download/attachments/49218519/Self-describing+Serialization.pdf?version=1&modificationDate=1328608647000
Andrzej Dworak, The new CMW-RDA 12
Design concepts – proven concepts
28th June 2012
Keep the proven concepts: Device/property model Communication through get, set and subscribe/publish
calls Properties represented as dynamic Data objects Self-describing serialization Authorize and authenticate user access
Andrzej Dworak, The new CMW-RDA 13
Design concepts – modularity
28th June 2012
Less coupling between system parts Easier to test and maintain
dependency injectionmocking
Will allow to exchange parts of the system (e.g. transport = ZeroMQ or serialization)
Will help during the migration process (old RDA to new RDA)
Andrzej Dworak, The new CMW-RDA 14
Design concepts – transport
28th June 2012
Transport executed asynchronously and independently of the user code execution
Separate channels for different types of communicationAdministrativeGet and SetSubscriptions
Optimization Internal and through user API
Intelligent buffering on the transport level Immutable data allows better and safer parallelization 0-copy (when sending, for receiving not 0 but also optimized)
Extended diagnostics
Andrzej Dworak, The new CMW-RDA 15
Design concepts – data model
28th June 2012
Property as a set of named fields with dynamic types Current type set extended to support:
Fixed length integer types (C++ int64_t, int32_t, int16_t, int8_t)Multidimensional arraysNested Data objects
Management of data ownershipAllows 0-copy for arrays and nested data objects
Context class with direct support forFiltersRBAC tokens
Andrzej Dworak, The new CMW-RDA 16
Design concepts – client
28th June 2012
Can get, set and monitor values of properties Uses predefined policies which will handle communication events
and errors Connect/reconnect policy Subscribe policy
Underlying communication is always non-blocking and asynchronous
For convenience API allows to perform the calls as Asynchronous Synchronous Buffered asynchronous Buffered synchronous
RBAC role may guarantee a client a higher priority Client may be managed on-line (similarly to servers and cmw-
admin)
Andrzej Dworak, The new CMW-RDA 17
Design concepts – server
28th June 2012
Asynchronous execution of implemented get, set and (un)subscribe handlers
Non blocking data publishingMay set delivery policy (normal/background)Message priority and QoS
Extended management (also on-line via CMW Admin)Turn on/off get, set, publishing(Un)block client / low-priority clientsDisconnect client / low-priority clientsSend commands to clients
Ping/hello Disconnect request “Failover to” request
Top Related