3 the system architecture
Transcript of 3 the system architecture
BEXIS Tech Talk Series
#3: The System Architecture
Javad ChamanaraMarch 2016
BEXIS Tech Talk #3: The System Architecture 2
Recall from the first talk
• Requirements– Data Lifecycle Management– Generic– Extensible– Portable– Scalable
BEXIS Tech Talk #3: The System Architecture 3
Requirements -> DLM
• Flexible Data Structures• Data Submission• Validation• Preserving• Metadata Management• Versioning
BEXIS Tech Talk #3: The System Architecture 4
Recall from the second talk
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
«use»
«use»
«use» «use»
«use»
BEXIS Tech Talk #3: The System Architecture 5
Core Conceptual Model
SearchPublishing
CMLand Use
ReservationData Submission
BEXIS Tech Talk #3: The System Architecture 6
DLM APIs
Core Functions
Data Structure Mgmt.Data Mgmt.
Metadata Mgmt.Sem. Data Mgmt.
BEXIS Tech Talk #3: The System Architecture 7
Multi DBMS data access
DB2 PgS
Data Access
Core Functions
...
RelationalXMLGIS
Multi DBMS supportVendor unlocking
ORM supportCustomization support
Data Structure Mgmt.Data Mgmt.
Metadata Mgmt.Sem. Data Mgmt.
BEXIS Tech Talk #3: The System Architecture 8
Access to the Core Functions
DB2 PgS
Data Access
Core Functions
UI Framework
...
Modularity
Integration
BEXIS Tech Talk #3: The System Architecture 9
Access to Core Functions
DB2 PgS
Data Access
Core Functions
UI
UI Framework
...
Modularity
IntegrationSynthesis Work
Semantic Search
Analytics
User Defined
Spatial Querying
External Tools
Web Services
Archiving
Import/ Export
Publishing
BEXIS Tech Talk #3: The System Architecture 10
Security
DB2 PgS
Data Access
Security
Core Functions
UI
UI Framework
...M
odularity
IntegrationSynthesis Work
Semantic Search
Analytics
User Defined
Spatial Querying
External Tools
Web Services
Archiving
Import/ Export
Publishing
BEXIS Tech Talk #3: The System Architecture 12
Architectural Patterns
• Generic User Interaction• MVC• APIs, Web APIs• Data Access• Services
BEXIS Tech Talk #3: The System Architecture 13
Generic User Interaction
Presentation
User
Services Core Services Security Data Access
BEXIS Tech Talk #3: The System Architecture 14
Presentation: MVC
class mvc
Controller
ModelView
«observe»
«observe»
BEXIS Tech Talk #3: The System Architecture 15
Simplified MVC
class mvc
View ViewModel
Controller
«observe»
Dependency
BEXIS Tech Talk #3: The System Architecture 16
Classes Engaged in an Interaction
Controller
View
Service Framework API
Data
GIS Data
ViewModel
«Observe»
«use»
«use»
BEXIS Tech Talk #3: The System Architecture 17
BExIS MVC Sequence Diagram
User
Controller Model ServiceViewRouting Engine
Request()
action()
GetData()
Populate()
Bind()
Render()
BEXIS Tech Talk #3: The System Architecture 18
MVC Constraints
• Passive Models• Request more data• Manipulate entities
• Models are POCOs• Models have no access to services• Models may have builder/ transformers• Views can interact with controllers’ actions only
– Interchanging data– Calling controllers’ actions either sync or async
• No lazy loading in the Views (populated models+ajax)
BEXIS Tech Talk #3: The System Architecture 19
Controller
View
Service Framework API
Data
GIS Data
ViewModel
«Observe»
«use»
«use»
Presentation
Data
Business
BEXIS Tech Talk #3: The System Architecture 20
Integration
• Service Based– Modules provide a set of services
• HTTP/S + Auth– Internal authentication– Remote LDAP/ Active Directory integration
• Authorization– Restricts access to controllers’ actions– Public actions– Public data
BEXIS Tech Talk #3: The System Architecture 21
Public APIs
• Web API– API namespace
• Module/Controller/Action• Verb• Data
• www.example.com/dcm/datasets/1• www.example.com/dcm/datasets/1/versions• www.example.com/dcm/versions/10• Querystrings are allowed• No versioning on APIs! Always the latest!!
BEXIS Tech Talk #3: The System Architecture 22
Modularity
• Modules– A set of related user facing functions– To extend the application’s functionality
• Layering– Presentation
• MVC– Service
• Access to Core services, DLM, external services, etc.– Data Access
• Own Database/data• Direct access to the DB/data (not recommended)
BEXIS Tech Talk #3: The System Architecture 23
Modularity
• Integration– Shell integration: menus, themes, scripts, etc.– Authorization
• Introducing actions, to be secured, to the system• Automatic security trimming/ action authorization
• Notes– Deployment– Hot plug-abilities– Upgrading/versioning
BEXIS Tech Talk #3: The System Architecture 24
Data
• Relational data– Schema evolution– Schema generation– Data versioning– Vendor/tool independence
• XML data– File based XML– RDBMS based XML
• GIS data– External systems– Integrated querying– Security
BEXIS Tech Talk #3: The System Architecture 25
Data Access
• ORM– NHibernate– Vaiona
BEXIS Tech Talk #3: The System Architecture 26
Data Access Patterns
• Repository pattern– CRUD on entities– Querying using
• LINQ to objects• HQL• Native SQL (queries, views, stored-procedures)• Vendor specific customization
– Relationship traversal• Laziness/Eagerness• object graph traversal
– Automatic entity state/context management– Based on: http://martinfowler.com/eaaCatalog/repository.html
BEXIS Tech Talk #3: The System Architecture 27
Data Access Patterns
• Unit of Work pattern– Business Transaction Management– Concurrency Control– Operating on multiple repositories– Transactions
• Distributed Transaction Coordination• Transparent transaction management• Ambient transaction detection/reuse
– Web request/session/user aware transaction mgmt.– Automatic commit on request/session termination– Based on: http://martinfowler.com/eaaCatalog/unitOfWork.html
BEXIS Tech Talk #3: The System Architecture 28
Data Access Patterns
• Aggregates– Cluster of entities and their dependents
• Datasets and their Versions• Units of measurement and their conversion formula
– Simplify the service layer– Packaged as units of work– Better business level operation/transaction mgmt.– Look at: http://
martinfowler.com/bliki/DDD_Aggregate.html
BEXIS Tech Talk #3: The System Architecture 29
Data Access Patterns
• Materialization– Complex entity serialized as XML– Stored in field of a table row– Materialize on read
• Reads the XML and loads the entity– Dematerialize on write
• Serializes the object as an XML document– Used in Data Tuples of dataset versions
BEXIS Tech Talk #3: The System Architecture 30
ORM example
• Dataset as an Aggregate area• DataTuple• DataCell• Versions• Associations
BEXIS Tech Talk #3: The System Architecture 31
Dataset aggregate: conceptual diagram
Dataset
Dataset VersionDataset Stage
Tuple
DataValueData Cell
Data Structure
1
+Current Stage
1
10..1
1
1+Variable Values
{Delta Association,Only Structured Data}
BEXIS Tech Talk #3: The System Architecture 32
Dataset aggregate: class diagram
BEXIS Tech Talk #3: The System Architecture 33
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
BEXIS Tech Talk #3: The System Architecture 34
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
Identification: it holds information about primary keys, key generation, etc.
BEXIS Tech Talk #3: The System Architecture 35
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
Payload: is the business data that the table holds
BEXIS Tech Talk #3: The System Architecture 36
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
Associations: relationships between this table and others (also itself). Connected and not connected foreign keys fall into this category, too.
BEXIS Tech Talk #3: The System Architecture 37
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
Versioning: information about version number and timestamp. This is the technical data level versioning to be used in concurrency control. It is not the business level data versioning.
BEXIS Tech Talk #3: The System Architecture 38
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
Audit: the information about the person or the system who has changed the data in addition to the time, action, and other information
BEXIS Tech Talk #3: The System Architecture 39
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
State: shows the current status of the data record. It can be set by business workflows or quality control, e.g., “Approved”, “Rejected”, “Published”, “Locked”, etc.
BEXIS Tech Talk #3: The System Architecture 40
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
Technical: this part is used for development purposes and is not intended to be used by data owners or data consumers
BEXIS Tech Talk #3: The System Architecture 41
General Record Structure
Identification
Payload Associations
Versioning Audit State Technical Extra
Extra: It is an XML field to provide a place for minor extensions and reduce the amount and frequency of changes in the database schema (and sometimes data). Also, it can be considered as an exertion point for customization in different scenarios.
BEXIS Tech Talk #3: The System Architecture 42
ORM mapping: The entity
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="BExIS.Dlm.Entities" namespace="BExIS.Dlm.Entities.Data">
<class xmlns="urn:nhibernate-mapping-2.2" name="Dataset" table="Datasets">
</class>
</hibernate-mapping>
BEXIS Tech Talk #3: The System Architecture 43
ORM mapping: Non payload attributes
<id name="Id" type="Int64"> <column name="Id" /> <generator class="native" /></id><version name="VersionNo" type="Int32"> <column name="VersionNo" /></version> <property name="Extra" type="System.Xml.XmlDocument, System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"> <column name="Extra" not-null="false" sql-type="xml" /></property>
BEXIS Tech Talk #3: The System Architecture 44
ORM mapping: State attributes
<component name="StateInfo"> <property name="State" type="string"> <column name="State" /> </property> <property name="Timestamp" type="DateTime"> <column name="StateTimestamp" not-null="false"/> </property> <property name="Comment" type="string"> <column name="StateComment" />
</property></component>
Same for Creation Info and Modification Info
BEXIS Tech Talk #3: The System Architecture 45
ORM mapping: Payload attributes
<property name="LastCheckIOTimestamp" type="DateTime"> <column name="LastCheckIOTimestamp" /></property> <property name="CheckOutUser" type="string"> <column name="CheckOutUser" /></property>
BEXIS Tech Talk #3: The System Architecture 46
ORM mapping: Associations
<set name="Versions" inverse="true" lazy="true" order-by="Timestamp desc" cascade="all-delete-orphan"> <key> <column name="DatasetRef" /> </key> <one-to-many class="DatasetVersion" /></set>
BEXIS Tech Talk #3: The System Architecture 47
Dataset aggregate: Database Schema
BEXIS Tech Talk #3: The System Architecture 48
How to work with the model
DatasetManagerService dms = new DatasetManagerService();
Dataset ds = dms.CreateEmptyDataset(sds, rp, ms);
DatasetVersion workingCopy = dms.GetDatasetWorkingCopy(ds);
// edit the version
dms.CheckInDataset(ds, “new version is created.", user);
BEXIS Tech Talk #3: The System Architecture 49
Inside the “Check In” service method
public void CheckInDataset (Int64 datasetId, string comment, string userName) { using(IUnitOfWork uow = this.GetUnitOfWork()) { IRepository<Dataset> repo = uow.GetRepository<Dataset>(); Dataset ds = repo.Get(…).FirstOrDefault();
// Check-in logic here… repo.Put(ds); uow.Commit(); } } }
BEXIS Tech Talk #3: The System Architecture 51
Outlook
Whats next in the talk series?• General purpose components• Third party components• Cross-cut functionalities
BEXIS Tech Talk #3: The System Architecture 5252
Thanks!Questions?
Contact:[email protected]://fusion.cs.uni-jena.de/bexis
Acknowledgment