1 Quick Guide to CAM Blueprints Using blueprints to develop XML exchange templates and schema (CAM...
-
Upload
brenden-hallmark -
Category
Documents
-
view
223 -
download
0
Transcript of 1 Quick Guide to CAM Blueprints Using blueprints to develop XML exchange templates and schema (CAM...
1
Quick Guide to CAM Blueprints
Using blueprints to develop XML exchange templates
and schema
(CAM – Content Assembly Mechanism Specification)
Author:
David RR Webber
Chair OASIS CAM TC
January, 2010
http://www.oasis-open.org/committees/cam
2
Introduction
• Configuring formal information exchanges using XML transactions today requires a set of exchange schema that describe those interactions
• An exchange schema provides the structural layout for the XML transaction along with rules detailing the content expected
• Exchange schema development requires knowledge of XML best practices, industry specifications, data modelling techniques and guidelines for exchanging XML transactions
• How does the CAM blueprint approach enable a shift away from a purely manual process today to one that can be highly automated, consistent and repeatable?
3
Solution Approach• Top Down development
• Allows developers to quickly outline the information exchange• Provides XML dictionaries of canonical reusable components• CAM tool then expands the details and fill-in the blanks• Review in visual editor to confirm or extend results• Iterative approach until complete
• Automated exchange package generation• Schemas, XML, documentation, mapping crosswalk• Test generation of example XML with rules validation
• Principles and Rules evaluator• Provides best practices for interoperability • Checks for specific schema techniques and details
• Dictionary driven reuse• Provides industry components / Enterprise Data Model• Ensures consistency of definition and use• Roll your own dictionaries
4 January, 2010 – CAM Draft Specification Development Related Materials
Agenda• Blueprints and Dictionaries
• Exchange Assembly Process Steps Overview
• Leveraging re-use – canonical XML dictionaries
• Introducing Exchange Assembly Blueprints
• Modelling information structures in XML
• Using namespaces and prefixes
• Constructing your Blueprint templates
• Running the Expander
• Finishing your Exchange Template
• Generating Exchange Artifacts• NDR evaluation, Exchange schema, mapping crosswalk, XML
instances, realistic data use, business rules documentation
• Summary
5 January, 2010 – CAM Draft Specification Development Related Materials
Web based
Desktop
Excel
Search tools
Exchange Assembly Process Overview
Project InceptionProject
Inception
dictionariesdictionaries
Exchange Content
Modelling
Exchange Content
Modelling
Blueprints Dictionary
Dictionary
Expand and Evaluate Results
Expand and Evaluate Results
Review / Tailor Exchange Structures
Review / Tailor Exchange Structures
Generate Package
Artifacts and
Test them
Generate Package
Artifacts and
Test them
Exchange Package
Exchange Package
Completed
Exchange Structures
NDR Evalu
- ation
Dictionary Generation /Selection
Completed Exchange artifacts
Test Cases
Validation Engine
XMLexamples
XSD schemas
Subset SchemaCrosswalk
MappingSpreadsheet
RulesReport
RenderingStylesheet
Import XSD
Naming / NDR /
spell check Generat
e Dictionar
y
Import EDM
Test Examples and Rules
Model Exchange
Expand, Review, Refine
CAM Enabled
11
22
33
44
55
66
BusinessModelling
Requirements
Charter
Work Group
Cross Referenc
e Mapping
XSD Schema Writer
Schema Generation
6 January, 2010 – CAM Draft Specification Development Related Materials
Leveraging Re-use :Canonical XML Dictionaries
• What is a canonical dictionary
• Styles of dictionary
• How do I obtain a dictionary?
• How do I use it with CAM?
7 January, 2010 – CAM Draft Specification Development Related Materials
Canonical XML dictionary
• A collection of distinct components that represent discreet business information for an application domain
• Includes singleton components and combinations of related components together as sub-assemblies
• Information is represented in a simple neutral conceptual data format that captures the critical concepts about the data e.g. name, description, content type, contextual usage pattern, hierarchy
• Wikipedia definition:http://en.wikipedia.org/wiki/Canonical#Computer_science
8 January, 2010 – CAM Draft Specification Development Related Materials
XSD Schema v CAM Dictionary• There are generally two “styles” of XSD schema
• Distinct information exchange schema structures• Collections of schema model components, possibly abstract,
and possibly recursive
• CAM dictionaries built from distinct exchange structures generally mimic those components directly
• Parent / Child sets are visible in dictionary as ABIE/BBIE pairings
• “children” column in dictionary contains direct descendents• “context” column reflects component relationships
• CAM dictionaries built from models with abstract and recursive parts may have fragments of components only
• Assembly of exchange structures using references to piece parts, rather than complete structure assemblies
• Reuse is then around these extension part collections• Context may need to be overridden to meet desired use pattern
9 January, 2010 – CAM Draft Specification Development Related Materials
Available XML Dictionaries• LEXS 3.1.4 dictionary• OASIS EDXL dictionary• OASIS EML dictionary
• NIEM 2.1 dictionaries• CBRN dictionary• Emergency dictionary• Family dictionary• Immigration dictionary• Infrastructure dictionary• Intelligence dictionary• Justice dictionary• Maritime dictionary• Screening dictionary• Trade dictionary• NIEM core dictionary• Immigration blueprint
Available from download site
direct link:
http://sourceforge.net/projects/camprocessor/files
XMLXMLXMLXML XMLXML XMLXML XMLXML XMLXML
+ includes spreadsheets and sample blueprint
Packaged with CAM editor
see dictionary folder of install
+ spreadsheet
+ blueprint samples
XMLXMLXMLXML
XMLXML
Note: Those marked in bold are model style dictionaries with recursive components.
10 January, 2010 – CAM Draft Specification Development Related Materials
Drag and Drop dictionary into Spreadsheet
Drag and Drop XML dictionary
format into Microsoft Excel
Drag and Drop XML dictionary
format into Microsoft Excel
Item namesItem names
Child item indicatorChild item indicator
List of Child itemsList of Child items
Parent item indicatorParent item indicator
Unique IDUnique ID
Item PropertiesItem Properties
11 January, 2010 – CAM Draft Specification Development Related Materials
Creating your own Dictionaries
• The CAM toolkit includes dictionary generation tools• can harvest components from existing exchange schema
• will import model components from ERwin modelling tool
• can ingest and use subset schema or component schema
• A separate companion Quick Guide describes the dictionary creation processes and tools
• Look for existing dictionary packs from Sourceforge files location of CAM downloadhttp://www.sourceforge.net/projects/camprocessor/files
12 January, 2010 – CAM Draft Specification Development Related Materials
Using Dictionaries
• Determine your business information exchange components at conceptual level
• Search and locate candidate components from appropriate domain dictionary collections
• Catalogue the parts to be used• Dictionary components can be referenced individually or
as collections by an assembly blueprint that puts them all together to create a complete information exchange
• Components can be selected from multiple dictionaries
• Note any new extension pieces as needed• A control file configures which physical dictionary files to
reference and associates one or more particular namespace prefixes with each dictionary
13 January, 2010 – CAM Draft Specification Development Related Materials
Dictionary Control File
Control files should be manually edited in a XML editor tool to make configuration changes as needed (e.g. arrowed text areas).
Associates physical dictionary files to namespace prefix details
14 January, 2010 – CAM Draft Specification Development Related Materials
Introducing Exchange Assembly Blueprints
• What is a blueprint template• Example blueprints• Blueprint tools overview• How do I create a blueprint?• How do I use it with CAM?
15 January, 2010 – CAM Draft Specification Development Related Materials
Introducing Blueprints
• A blueprint is the outline of your information exchange and the top level sets of components needed
• Allows rapid iterative exchange structure design• Start with a basic idea of what is needed, extend and refine as you clarify
groups of components and their relationships
• Can import components from one or more domain dictionary collections
• Allows you to focus on the conceptual and hierarchical information model components and their purpose• The low level specific information details will be filled in for you by the
expander tool from the matching dictionary definitions
• Review results and add local extensions / exclusions as needed
• Blueprints themselves also have high re-use value• Can be sub-assemblies and patterns not just exchange models
16 January, 2010 – CAM Draft Specification Development Related Materials
Example BlueprintsLEXS messaging blueprint
Reusable messaging envelope constructs
OASIS EDXL HAVE message
Business functional components
Message handling,
delivery and controlPayload goes here
Top level sets of
business
information
components
Individual
component
these examples available from CAM editor install package~ CAMeditor\eclipse\workspace\CAMEditor\dictionary\blueprints\
17 January, 2010 – CAM Draft Specification Development Related Materials
Blueprint Tools Overview
Expander Tool
Expander Tool
Industry dictionaryIndustry
dictionary
Domain dictionaryDomain dictionary
Component Definitions
44
Component Definitions
Web toolWeb tool
ExcelExcel
Search
Tools
Search
Tools
22
BlueprintDesignerBlueprintDesigner
11
InsertDictionary
ParentComponents
InsertDictionary
ParentComponents
33
Completed ExchangeTemplate
Completed ExchangeTemplate
55
18 January, 2010 – CAM Draft Specification Development Related Materials
How do I create a blueprint?
• Use CAM Eclipse visual editor• create basic blueprint from scratch or from XML instance
• Clone and edit an existing blueprint• use a XML editor to manipulate raw XML of blueprint (e.g.
search and replace namespace details)
• modify blueprint using CAM Eclipse visual editor
• Beware of namespaces!• You need to pre-declare namespaces in the blueprint –
that may appear later in the finished expanded exchange
• Use CAM Eclipse visual editor root node menu to add namespace definitions
• Tip: use your XML editor to cut and paste namespaces into raw blueprint XML file directly
19 January, 2010 – CAM Draft Specification Development Related Materials
Blueprint Structure Overview
Namespace
declarations
Short Name for Exchange
Root node name and optional namespace
XML container elements to facilitate use patterns
Dictionary component references
Dictionary Type annotation reference
(note red paper clip icon)
20 January, 2010 – CAM Draft Specification Development Related Materials
Adding Components from a Dictionary
• Understanding the 4 basic types of component referencing available
• Direct component (BBIE)
• Direct collection (ABIE)
• Indirect type (extension)
• Indirect sub-component (contextual re-use)
• Control of the XML structure hierarchy• Grouping collections of components
• Extending an existing component
• Building new extension from existing components
• Namespaces – when to use them
21 January, 2010 – CAM Draft Specification Development Related Materials
Example XML Techniques
Direct component referencing (BBIE)
Direct collections referencing (ABIE)
Indirect type extension via annotation
Type
(red paper clip icon denotes annotation)
Building new component from existing ones
Grouping collection of components
Namespaces – root namespace optional (aaa)
Dictionary component namespaces are required
(Immigration Alien Alert example blueprint)
22 January, 2010 – CAM Draft Specification Development Related Materials
Making New Blueprint from scratch
Open new template
11
Assign root element22
33
Complete structure by adding elements and dictionary components
23 January, 2010 – CAM Draft Specification Development Related Materials
Running Blueprint Expander
• Blueprint check list • Using Expander tool dialogue• Reviewing results• Hints and Tips
24 January, 2010 – CAM Draft Specification Development Related Materials
Using Your Blueprints
• What you need -• CAM Eclipse visual editor• Blueprint template• Dictionary or dictionaries of components• Dictionary control file
• Contains linkage of namespace prefix to each dictionary• Contains physical filename path of each dictionary
• Getting Started• Install CAM editor tool from Sourceforge.net• Open example blueprint from EDXL folder
~ CAMeditor\eclipse\workspace\CAMEditor\dictionary\blueprints\EDXL
• Try blueprint expander with control file EDXL-dictionary-control-file.xml
25 January, 2010 – CAM Draft Specification Development Related Materials
Running Blueprint Expander
Select Dictionary Control File
Pick output result
filename (default is
-expanded.cam)
Determine rule generation
selections from dictionary
definitions
Note: entries are “sticky” so
are remembered for next time
This example is available in the NIEM dictionaries download pack from Sourceforge
26 January, 2010 – CAM Draft Specification Development Related Materials
Blueprint followsmodel Console Log detail
Blueprint Expander Example
Exchangecomponents outline
EXCHANGE BLUEPRINTEXCHANGE BLUEPRINT
COMPLETED EXCHANGE TEMPLATECOMPLETED EXCHANGE TEMPLATE
11
33
StructureDetails Expanded
Expander Tool
Expander Tool
22
Dictionary Lookups
Dictionary Lookups
27 January, 2010 – CAM Draft Specification Development Related Materials
Reviewing Results
Exchange Structure items
Rules for each item
DomainComponents Namespaces
1122
33
28 January, 2010 – CAM Draft Specification Development Related Materials
Hints and TipsBlueprint Remedy
Node not expanded, or removed! Override default dictionary structure context definition using an annotation Type reference to the node instead – see Alien alert example blueprint – aaa:OfficerDetails
Added parent and child combination. All children added instead of just one.
Expander inserts all children of parent. Either delete unwanted children from expanded result (see next section), or create custom parent node, and pick children components directly – see Alien alert blueprint example.
Node not expanded Make sure namespace prefix and spelling exactly matches dictionary and dictionary control file prefixes.
Undefined namespace prefix error when opening expanded template in CAM editor
Expander added dictionary components from another namespace. Declare that namespace prefix back in blueprint and re-run expander again
Expanded result missing all desired sub-components
Create your own blueprint extension parent and combine parts to create desired result – see Alien alert example – aaa:FurtherDetails
Expanded result has too many sub-components
In expanded result mark unneeded parts with “excludeTree” rules, then run File / Export / Compress to strip out; or use extension technique above to pick only few child elements instead of whole parent
29
Completing the Exchange Template
• Adding nodes• Deleting nodes• Editing Rules• Adding annotations
30 January, 2010 – CAM Draft Specification Development Related Materials
Exchange Template Editor
Rules for each item
Can add new custom domain Items
1122
33
44May need to add a namespace for new items
31 January, 2010 – CAM Draft Specification Development Related Materials
Deleting Exchange Components
Right click for popup menu22
From Rules exclude item or tree
Select item to be removed11
33
44 After marking all items to be excluded, save template, then run Files / Export / Compress template to physically remove them.
32 January, 2010 – CAM Draft Specification Development Related Materials
Editing Rules
Right click for popup menu
Select item to add new rule11
22
33Use rule wizard to complete
33 January, 2010 – CAM Draft Specification Development Related Materials
Adding annotations
Right click for popup menu
Select item to add new annotation11
22
33Use annotation wizard to complete
34
Generating Exchange Artifacts
• Dictionary crosswalk mapping • Exchange Schema • XML instances • Business rules documentation
35 January, 2010 – CAM Draft Specification Development Related Materials
Exchange Generation Steps
Toolkit for exchange artifacts
generation
Toolkit for exchange artifacts
generation
Suite of menu options and tools in desktop CAM toolkit editor
(each run custom xslt scripts on the exchange CAM template xml)
CAM Template
1. Run template Evaluation Report
2. Compare to industry dictionary
- create mapping spreadsheet
- create crosswalk xml
Generate business rules report
3. Generate exchange XSD schema
4. Generate XML test instance(s)
22
11
33
44
55
EXCHANGE TEMPLATEEXCHANGE TEMPLATE
Structure
Rules
Documentation
36 January, 2010 – CAM Draft Specification Development Related Materials
Using NDR Evaluation Report
• Provides scoring, alerts, warnings and potential
issues including:
• Naming and content model conventions
• Naming and restriction consistency checks
• Interoperability enablers/inhibitors checks
• Rules integrity and duplicates
• Statistics on exchange size
• Spell checking on component names
Access from Tools menu, Evaluate Template option
37 January, 2010 – CAM Draft Specification Development Related Materials
Evaluation Report (NDR) example
Part of an example report for LEXS getDataItemRequest template displayed in toolkit HTML viewer
38 January, 2010 – CAM Draft Specification Development Related Materials
Compare to industry dictionary
• References industry dictionary of names and properties• Matches on physical names• Reports mapping details• Compatible with Microsoft Excel• Report can be used to do spell checking• Generates crosswalk xml file• When NIEM-dictionary selected:
• generates NIEM SSGT wantlist.xml file
• provides % re-use statistics for NIEM components( note NIEM-dictionary.xml is located in install directory
~ CAMeditor\eclipse\workspace\CAMEditor\dictionary folder )
39 January, 2010 – CAM Draft Specification Development Related Materials
Example cross-reference spreadsheet
Formatted view in Microsoft Excel of import of cross-reference report details (from generated XML file)
40 January, 2010 – CAM Draft Specification Development Related Materials
Generate Documentation and Schema
• Documentation:• Create HTML report of exchange schema details and
associated content and business rules
• Report layout and content designed to be reviewed by business analysts
• Schema:• Generate XSD schema for exchange
• Customizable exchange folder layout management by namespace for extension, subset and exchange schema components
• Writes XSD schema in syntax that is clear, simple and compatible with deployment tooling environments
41 January, 2010 – CAM Draft Specification Development Related Materials
Business Rules Documentation
Part of the example rules for LEXS getDataItemRequest template displayed in toolkit HTML viewer
42 January, 2010 – CAM Draft Specification Development Related Materials
Export Exchange to XSD Schema
Completed Exchange Structure
Set Exchange Output Option
Complete set of exchange schemas generated
Export Template to Exchange XSD:
43 January, 2010 – CAM Draft Specification Development Related Materials
Exchange Schema Generated
Each namespace file is import for those specific type definitions
Reviewing XSD results in a schema editor tool
Set of XSD files with filename and namespace suffix
44 January, 2010 – CAM Draft Specification Development Related Materials
XML Testing Examples Generation
• Support for software development testing process• Designed to allow creation of concrete realistic
examples not just random value based• Hinting system allows insertion of actual test system
values into XML examples• Can create both valid and invalid examples to support
unit testing of application software• Exclude capability allows generator to create examples
that contain only a portion of the entire exchange• Control over random seed value used allows re-
generation of identical test cases
45 January, 2010 – CAM Draft Specification Development Related Materials
XML example generation wizard
View of CAM toolkit with LEXS getDataItemRequest and dialogue for XML test example generator tool
46 January, 2010 – CAM Draft Specification Development Related Materials
Running validation rules tests
• Built-in CAM validation engine allows testing of XML instances against actual exchange rules (CAMV).
• Critical to ensure that the exchange validates actual live production example scenarios correctly
• Allows deployed solution to match exchange schema details
• Errors can be reviewed interactively in exchange visual interface
• Post-processing of validation results allows unit regression tests to be created with reporting of errors, warnings and information level notes
47 January, 2010 – CAM Draft Specification Development Related Materials
Run Exchange Template
Pick XML test case to validate Run validationReview results in visual editor
48 January, 2010 – CAM Draft Specification Development Related Materials
Example Exchange Packaging Details
Package Artefact Description
CAM Generates Files
NDR Evaluation Report Evaluation of exchange template compared to set of Naming and Design Rules (NDR) and interoperability factors / principles
Crosswalk XML Itemized list of each dictionary component element and attribute included in the exchange.
Exchange Schema Base document schema that defines the full XML structure for the exchange and is generally named after the exchange itself.
Extension Schema Specification for extended components—separate local name-spaces of components not contained in dictionary
Sample XML Instance Example instance(s)
Documentation
Master DocumentationThe Master Document is the main document for which all of the context and details around the exchange are explained. This
document includes, the overview, as well as details surrounding the exchange, business drivers and requirements
Business rules Business rules in plain or structured English, generated by CAM development tool.
Mapping to Dictionary Mapping of domain components, tagged with constraints (i.e., cardinality, etc.) to dictionary components as a spreadsheet.
Extended componentsComponents created because they were not in dictionary—may be part of mapping spreadsheet and include structure and definitions
of new components.
49 January, 2010 – CAM Draft Specification Development Related Materials
Summary
Dictionary driven exchangesBlueprint enabled reuseAutomated exchange package generationAlignment to NDR Principles and RulesTesting and validation support
50 January, 2010 – CAM Draft Specification Development Related Materials
Review• Top Down development
• Reference dictionary components
• Create exchange blueprint
• Run Expander tool
• Refine desired structure in visual editor
• NDR Principles and Rules• Best practices for interoperability and schema techniques
• Dictionary driven reuse• Select Enterprise Data Model and industry components
• Ensures consistency of definition and use
• Automated exchange package generation• Schemas, XML, documentation, mapping crosswalk
• Test generated example XML with rules validation
51 January, 2010 – CAM Draft Specification Development Related Materials
Links and Resources
• CAM Toolkit download• http://sourceforge.net/projects/camprocessor
• NIEM dictionaries and sample blueprint• http://sourceforge.net/projects/camprocessor/files
• NIEM tools and IEPD tools tips and hints• http://wiki.oasis-open.org/cam/NIEM_Tools_and_CAM