1 CORE – COmmon Reference Environment How it works Jean-Pierre Kent 11 January 2012.
-
Upload
hector-bruce -
Category
Documents
-
view
216 -
download
2
Transcript of 1 CORE – COmmon Reference Environment How it works Jean-Pierre Kent 11 January 2012.
1
CORE –COmmon Reference
EnvironmentHow it works
Jean-Pierre Kent11 January 2012
Contents
• Introduction– Design ≠ implementation
• Overview– A model of the user’s experience
• Presentation of the information model
2
Design ≠ Implementation
• Goal of design:
– Deliver a concept apt to contribute
to the industrialisation of official
statistics (ref: HLG-BAS Vision)
• Result:
– A model that exceeds the capacity
of a 1-year project3
Design ≠ Implementation
• Goal of implementation
– Deliver Proof Of Concept with:
• Platform independence
• Model-driven
• Model-driven
• Result:
– Implementation of a subset of the
model 4
design environment
execution environment
What is Platform Independence?• Once implemented, a service can run on any
platform (e.g. .Net, Java, ...)
• A process engine running on a platform (e.g. Java) can control services running on another platform (e.g. .Net)
• A process can be distributed:
– Manage microdata at Statistics Netherlands
– Do aggregation at ISTAT
– Produce the SDMX output at Eurostat
– Under control of a process engine running at INSEE
5
Model-Driven: cutting
costs• Traditionally:
1. Designer makes models
2. Developer creates the system
3. System runs.
• In a model-driven environment:
1. Designer makes models
2. System runs
6
Model-Driven:
effectivity
7
Fle
xibi
lity
Standardisation
Taylor-made
Standard package, ERP, CRM, DMS...
Model-Driven
Spaghetti
Model-driven: benefits
• Cost reduction: less manual work
• Reliability: manual work is error prone
• Time to market: less manual work
• Standardisation: system enforces standards
• Flexibility: incremental development, agile maintenance
• Reliability: build processes from well-designed and well-tested services
• Strict separation of design and execution
• Focus on process quality as a source of product quality
• … and some more
Questions (Part 1)
?9
Overview
• What users see and use
– This is not the information model
– Nor the technical model
– But a model of the user’s
experience
10
11
Data Set+PreparesProcess Administrator
CORE Run Time
Process Administrator
+Runs Process
Process Step Data Set
Data Set DescriptionOrdered list of column names and types
12
Data Set Kind
CORE Design time
+Designs
Process Designer
Process
Process Step
CORE Service
Service Designer
NSI Community
List of required column kinds
Column kind: generic definition of a column, consisting of a data type and a statistically defined role ( e.g. dimension, measure, classification level, calibration weight)
This is where GSIM
comes in
Ordered list of column names and types
Data Set Description +Designs
13
CORE: the whole
picture
Ordered list of column names and types
Data Set Description +Designs
List of required column kinds
Process Administrator
Process+Runs
Data SetProcess Step+Prepares
Process Designer
+Designs
NSI Community
Data Set Kind
CORE Service
Service Designer
Column kind: generic definition of a column, consisting of a data type and a statistically defined role ( e.g. dimension, measure, classification level, calibration weight)
How do services
interact?• You can use different tools for
different services
– e.g. SPSS, SAS, R ...
• Different tools expect different
data formats
• Conversions are inevitable14
Conversions are
expensive!• Between 2 formats
– A B: 2 conversions
• Between 3 formats
– A B: 6 conversions
C
• Between N formats: N2-N15
CORE reduces N2-N to
2*N• Standard CORE data format
• Conversion to and from CORE format
Tool XInput (X)Output (X)
Model (CORE)
Convertor
Input (CORE) Output (CORE)
Model (X)
Convertor
Convertor is tool-specific – not service-specific
Model (X)
Model (CORE)
Convertors are format-specific, not service-specific
Questions (Part 2)
?17
18
CORE: the whole
picture
Ordered list of column names and types
Data Set Description +Designs
List of required column kinds
Process Administrator
Process+Runs
Data SetProcess Step+Prepares
Process Designer
+Designs
NSI Community
Data Set Kind
CORE Service
Service Designer
Column kind: generic definition of a column, consisting of a data type and a statistically defined role ( e.g. dimension, measure, classification level, calibration weight)
The information model
• What the designer sees
– Data set description and
data set kind
– Service and data set kind
Data Set Description
Data Set Kind
CORE Service
Data set definition
package
e.g. microdataset, aggregated data set, classification structure
e.g. variable, measure, dimension, level, weight
The type of a column definition should be one of the allowed types of its column kind
Cell
value
Data set kind
namemin number of rowsmax number of rows
Column kind
namemax number of occurrencesmin number of occurrences
1..n1..n
column kinds
<<set>>
Column definition
namecolumn id
column kindcolumn definition
Row
row id
1..n1..ncols
<<ordered>>
Data set definition
namelink to documentationlocation
data set kind
1..n1..n
column definitions
<<ordered>>
Value type
name1..n1..n allowed types
<<set>>
type
Data set
0..n0..nrows
data set definition
Descriptor
name
0..n0..n
descriptors
descriptor type
Actual descriptor
value
descriptors
descriptor
Data Set Description
Data Set Kind
Communication
channels• Manage communication between a
service and the execution environment
• Give support to Plug-and-Play coupling
• Implement messaging
Data Set Kind
CORE Service
5 types of channels
Channel
Data set kind(f rom Data set def initions)
Rule kind(f rom Rules)
GSIM object description(f rom Messages)
Parameter(f rom Parameters)
Expected free-style argument(f rom Messages)
4 types of messages
• Service signature message
• Service configuration message
• Service execution message
• Service output message
Service signature
message• Service communicates to its
environment the channels
that it supports.
• Channels constrain the kinds
of information
– expected during execution or
during configuration
• e.g. data set kind
“microdata”
– to produce during execution.
• e.g. data set kind
“aggregate”
Service AEnvironment
I expect a fixed number of data sets of the following kind: […] I expect an unbounded number of rules of the following kind: […] I
expect values for the following other parameters: […]
Service signature message
What is your signature?
channels
Service configuration
message
• Environment communicates
details about data sets that
will be offered to the service
during execution.
– e.g. number of columns of a
data set
– value types for each of these
columns
• This information must fit in
the channels specified in the
service signature message
– e.g. a data set description
must match the expected data
set kind
Service AEnvironment
The data sets you’ll receive during execution are of the following form: […] Please receive the following rules and other parameters: […]
Service configuration message
channels
Service execution
message• Service is requested to
execute itself
• Service is offered a number of data sets and business objects as input.
• This information must comply with the service's signature message
• It must be consistent with the data set and business object details the service is configured with through the service configuration message
– e.g. a data set must match the expected data set description
Service AEnvironment
Please start execution and receive the following data sets as arguments: […]
Service execution message
channels
Service output
message • A service ends its
execution by sending a
service output message.
• The result of the service
execution is documented
by data sets which
match the service's
configuration message.
– e.g. the output data sets
are consistent with their
data set descriptions.
Service AEnvironment
Please receive the following data sets as my output: […]
Service output message
What is your output?
channels
Questions (Part 3)
?28