THE IMPORTANCE OF DATA MODELS IN ENTERPRISE …ASG Enterprise Data Intelligence Core Capabilities...
Transcript of THE IMPORTANCE OF DATA MODELS IN ENTERPRISE …ASG Enterprise Data Intelligence Core Capabilities...
© 2016 ASG Technologies Group, Inc.All rights reserved
THE IMPORTANCE OF DATA MODELS
IN ENTERPRISE METADATA
MANAGEMENT
October 19, 2017
© 2016 ASG Technologies Group, Inc.All rights reserved
HAPPINESS IS…SOURCE: 10/18/2017 TODAY SHOW – “THE BLUE ZONE OF HAPPINESS” – DAN BUETTNER
• 3 Close Friends
• Get a Dog
• Good Light
• Get Religion
• Get Married…. Stay Married
• Volunteer
• “Money will buy you Happiness… well, it’s more about Financial Security”
• “Our Data Models are now incorporated within our corporate metadata repository”
© 2016 ASG Technologies Group, Inc.All rights reserved
3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES
1. Happiness is individually sought and achievable
© 2016 ASG Technologies Group, Inc.All rights reserved
3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES
1. Happiness is individually sought and achievable
2. Data Models can in be incorporated into your corporate metadata repository
© 2016 ASG Technologies Group, Inc.All rights reserved
3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES
1. Happiness is individually sought and achievable
2. Data Models can in be incorporated into your corporate metadata repository
3. ASG Technologies can provide an overall solution with services to accomplish #2 above and more for your company.
© 2016 ASG Technologies Group, Inc.All rights reserved
Data model imports to the metadata collection
View and search capabilities
Traceability of physical and logical models
End-to-End Data Lineage, impact analysis
Metadata workflow and lifecycle management
AGENDA
© 2016 ASG Technologies Group, Inc.All rights reserved
• ERwin
• IDERA Er-Studio
• IBM InfoSphere Data Architect
• Oracle SQL Developer DM
• SAP Power Designer
• Sparx Enterprise Architect
• Others …
• Forward / Reverse Engineering
• Mapping– Conceptual
– Logical
– Physical
• Data Dictionary Standards
• More …
• Metadata Repository– Data Model
– Business Glossary
– Reference Data
• End-to-End Data Lineage
DATA MODELING LANDSCAPE
DATA MODEL TOOLS DATA MODEL CAPABILITIES EMM
DATA MODELING TO ENTERPRISE METADATA MANAGEMENT – (EMM)
© 2016 ASG Technologies Group, Inc.All rights reserved
• ERwin
• IDERA Er-Studio
• IBM InfoSphere Data Architect
• Oracle SQL Developer DM
• SAP Power Designer
• Sparx Enterprise Architect
• Others …
• Forward / Reverse Engineering
• Mapping– Conceptual
– Logical
– Physical
• Data Dictionary Standards
• More …
• Metadata Repository– Data Model
– Business Glossary
– Reference Data
• End-to-End Data Lineage
DATA MODELING LANDSCAPE
DATA MODEL TOOLS DATA MODEL CAPABILITIES EMM
BASF – “WE DON’T MAKE THE WATER, WE MAKE IT CLEANER” ASG – “WE DON’T MAKE THE DATA MODEL, WE INCORPORATE IT WITHIN YOUR METADATA REPOSITORY”
© 2016 ASG Technologies Group, Inc.All rights reserved
• Business Glossary– Glossary, Contexts, Terms
• Business Packages– Business Rules, Processes, Critical Data
Elements, Policies, Standards
• Business Applications– Application
• Reference Data– Contexts, Concepts, Values, Meanings,
Permissible Values, Hierarchies
• Data Models
– Entities, attributes, relationships, …
• Relational Databases– Tables, Columns, indexes, ...
• Transformations– Task, Maps, Transformations, …
• Business Intelligence– Reports, Cubes, Fields, Maps, …
• Data Structures– Files, fields, …
• Big Data– Tables, Files, Streams, …
Populating the Metadata RepositoryCould be manually input for the content on the left ….
© 2016 ASG Technologies Group, Inc.All rights reserved
• Business Glossary– Glossary, Contexts, Terms
• Business Packages– Business Rules, Processes, Critical Data
Elements, Policies, Standards
• Business Applications– Application
• Reference Data– Contexts, Concepts, Values, Meanings,
Permissible Values, Hierarchies
• Data Models
– Entities, attributes, relationships, …
• Relational Databases– Tables, Columns, indexes, ...
• Transformations– Task, Maps, Transformations, …
• Business Intelligence– Reports, Cubes, Fields, Maps, …
• Data Structures– Files, fields, …
• Big Data– Tables, Files, Streams, …
Populating the Metadata RepositoryCould be manually input for the content on the left …. But not for the content on the right !!
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Collection / Import Metadata Collection Architecture – Modelling Bus – (Bi Directional)
Compare
Convert to Standard Format
Tool File
RochadeModelling
Application
Determine Item Names
Names File
Export Existing Items
Rochade File
Import File
Import
• SAG Predict
• ARIS Toolset
• Borland
• CaliberRM
• Together Control/Center
• Casewise Corporate Modeler
• COOL:Biz ; COOL:gen
• PowerDesigner
• ER/Studio
• ERwin
• IBM/Rational
• Data Architect
• Software Modeler (RSM)
• Software Architect (RSA)
• System Architect
• Systems Developer
• Rose
• Rational Rose XDE
• mictroTool case/4/0
Enterprise Architecture
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Collection / Import Metadata Collection Architecture – Application Model – Physical and Logical Layer - Process
• Scan in the models
• Define rules and process• Conceptual and Logical entity names should match.
• There has to be a link between physical table with the logical entity in the application model
(as shown in figure)
• Custom work to provide audit reports, dashboards etc. For example:• Find database tables which are based on physical model.
• Find physical model entities which are not implemented in the database.
• Find missing links/mismatches the enterprise model and application models
• Standard model linker will link model to physical data.
• Custom linker is required to create the link between conceptual and logical/physical models.
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence Browser – View
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence Browser – View
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence Browser – Search
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence Browser – Search - Account
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence MyInfoAssist – Search – Account – Logical and Physical views
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence Traceability – Logical and Physical views
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence Traceability – Logical Model Reporting
© 2016 ASG Technologies Group, Inc.All rights reserved
IMPLEMENTING DATA LINEAGE
We build out a reverse tracing methodology and base line for comprehensive and accurate end to end data lineage.
Change DetectionBusiness TraceabilityBusiness Glossary
ID Critical Data Scan Lineage
Lineage
Automate
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence End to End Zero Gap Data Lineage – Account – Business application view
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence End to End Zero Gap Data Lineage – Account – High Level view with Impact Analysis
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence End to End Zero Gap Data Lineage – Account – Detailed Level view with ETL and Impact Analysis
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence End to End Data Lineage – Account – Detailed Level view – Impact Analysis and Visualization
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence End to End Data Lineage – SnapShot – Lineage Comparison – Impact Analysis and Visualization
1) Current WIP Lineage
2) Click to Compare WIP changes with Prod (high level) within the Rochade Browser
3) WIP/Production compare at a column level
Legend:Grey=same in bothRed=DeletedGreen=Added
© 2016 ASG Technologies Group, Inc.All rights reserved
Metadata Repository Data Intelligence Metadata Workflow and lifecycle management
Governance Board
Business &Data Stewards
© 2016 ASG Technologies Group, Inc.All rights reserved
ASG Enterprise Data Intelligence Core Capabilities New words here….
27
01
02
03
05
04
DATA GOVERNANCEEnterprise Metadata, Business
modeling, CDE subscription, Workflow,
Issue management
DATA LINEAGELineage Life Cycle, Snapshot,
Stitching, Impact Analysis, Data
Lineage Appliance
AUTO DATA COLLECTIONApplication discovery &
understanding via 230+ Scanners
for structured & unstructured data
DATA DISCOVERYGlobal search, drill-down
REGS & COMPLIANCE
SUPPORT
Attestation documents and Reports
© 2016 ASG Technologies Group, Inc.All rights reserved
SUMMARY DATA MODELING TO ENTERPRISE METADATA MANAGEMENT
• Automated scanners or buses reduce complexity and increase accuracy
• Look for one solution that can provide full EMM
• Implementation Services will be key
Industry Leading Metadata Best Practices
Services Packages
SaaS Hosting & Maintenance
Metadata Practice
Experts
Training & Consulting
Scanner Development
Custom Development
On-premise & SaaS Implementation Services
Serv
ices
Re
sou
rces
© 2016 ASG Technologies Group, Inc.All rights reserved
3 POINTS TO REMEMBER – QUIZ TIME SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES
1. Happiness is individually sought and achievable
© 2016 ASG Technologies Group, Inc.All rights reserved
3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES
1. Happiness is individually sought and achievable
2. Data Models can in be incorporated into your corporate metadata repository
© 2016 ASG Technologies Group, Inc.All rights reserved
3 POINTS TO REMEMBER SOURCE: 10/19/2017 DAMA NYC – NOONTIME SPEAKER SLOT – MIKE WANYO – ASG TECHNOLOGIES
1. Happiness is individually sought and achievable
2. Data Models can in be incorporated into your corporate metadata repository
3. ASG Technologies can provide an overall solution with services to accomplish #2 above and more for your company.
QUESTIONS?
© 2016 ASG Technologies Group, Inc.All rights reserved
Mike Wanyo
Strategic Sales Specialist
914.921.7386