AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With...
Transcript of AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With...
![Page 1: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/1.jpg)
`
AI-Powered Data Cataloging Virtual Summit
Navigating Data Lineage
With Rabobank and Informatica
![Page 2: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/2.jpg)
Requirements:
Data Quality
Data Governance
Data Analytics
Data Privacy and Security
Regulatory Compliance
Data Lineage traces data from source to destination, covering the entire lifecycle of data. It includes
information about changes to data during its journey.
Data Lineage: A Business Imperative
![Page 3: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/3.jpg)
3 © Informatica. Proprietary and Confidential.
Data Lineage: The Foundational Use Case
• Dev Operations: Change Management & Impact Analysis - what-if analyses for changes
• Operational Efficiency: Eliminate proliferation, duplication, data silos, reduce costs
• DW/Apps Modernization: Complete understanding of the data landscape to enable app modernization & cloud migration
…and AI use cases
• Explainable AI & Data Science Governance: Track and assess data used to train models, govern AI projects. Support Explainable AI. Ensure training data variety.
Increasingly “IT” use cases are coming to the forefront…
![Page 4: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/4.jpg)
4 © Informatica. Proprietary and Confidential.
Data Lineage: Key Informatica CapabilitiesUser experience is key to understanding lineage, custom views at all levels and detail for every use case
• Business and logical views
• Dataset level lineage
• Detailed field level lineage
• Drill down details on data transformation logic
• Summarized, level-wise views
![Page 5: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/5.jpg)
5 © Informatica. Proprietary and Confidential.
Data Lineage: Key Informatica CapabilitiesComprehensive lineage to support all use cases requires wide and deep metadata connectivity as well as active metadata-driven intelligence
• Automatic data lineage stitching
• Discovered and curated
• Impact analysis with shareable exports
• Automatic lineage derivation from code: SQL Scripts, stored procedures, BI reports, ETL jobs & mappings
• Change notifications
• Data Similarity discovery
• Other relationships (joins, usage)
![Page 6: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/6.jpg)
Gaurav Pathak Anil Bandarupali
Speakers
Senior Director,Product Management
Senior Software Engineer, Data Management
![Page 7: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/7.jpg)
Rabobank experience with EDC
Anil Bandarupalli
Enterprise Data Catalog
![Page 8: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/8.jpg)
• Introduction to Rabobank
• Rabobank vision & strategy
• Data Management Office
• Data Governance team
• Scope of Data Lineage
• Data Lineage use case#1
• Data Lineage use case#2
• Lessons learned
• EDC next steps for Rabobank
Reading Guide
8
![Page 9: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/9.jpg)
2nd largerst bank in Netherlands101 local Rabobanks409 offices1.9 million members6.5 million private customers0.8 million commercial customers
Rabobank in the Netherlands
Mission Growing a better world together
Domestic Retail Banking
Almost 8.3 millioncustomers
7.3 million Dutch customers 1.0 million international customers
41 countriesInternational
Market Leader in Financing, Food and Agriculture
Private Sector Lending to Trade, Industry And Services
Rabobank at a GlanceSituation on December 31, 2018
9
![Page 10: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/10.jpg)
Growing a Better World Together
10 StrategicTop Priorities
• 100% Digitalconvenience in everything
• Top customeradvice nearby
• Growth with innovation
• Top performance• Optimal balance sheet• Exceptionally good
execution
• Focus on social responsibility and sustainable contribution
• Involved members andcommunities
• Inspired employees• One-Rabobank culture
HOW?Through our values and behaviours
I dare to make a difference for the world
I make you betterI am doing the right thing
exceptionally wellI go the extra mile
for my clients
Banking for the Netherlands Banking for Food
Excellent Customer Focus
Rock-solidBank
EmpoweredEmployees
Meaningful Cooperative
We are client-driven and action-oriented
We are purposeful and courageous
We are professional and considerate
We bring out the best in each other and keep learning
10
![Page 11: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/11.jpg)
Data Management Office
11
Data Management Office
Dat
a Li
neag
e
Dat
a A
rchi
tect
ure
Dat
a Q
ualit
y
Dat
a D
efin
itio
ns
Ente
rpri
se D
ata
Rec
ord
Keep
ing
Ref
eren
ce D
ata
Man
agem
ent
Dat
a Ex
chan
ge
Proc
ess
& C
ontr
ols
The Data Management Office supports the data governance structure within Rabobank.
The Rabobank Managing Board has mandated the Data Governance Board to increase the management of data with the following goals:• Define, approve and communicate data
strategies, data policies, data standards, data architecture, procedures and metrics
• Track and enforce compliancy of data policies, standards and architecture
• Initiate, track and oversee the delivery of data management projects and services
• Manage the resolution of data related issues
• Increase understanding and promote the value of data assets
![Page 12: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/12.jpg)
Data Management & Data Logistics Department
12
Data Management
& Data Logistics
Data Quality
![Page 13: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/13.jpg)
Why Data Lineage So Important?
13
Having Data Lineage for Enterprise will create business value in four areas:
Regulatory Requirements
Use end-to-end lineage BCBS 239 and other regulatory mandates
Data Quality Management
Data lineage improves the overall data quality by reducing the duration of root cause analyses process of Data Quality Issues
Change management
Provides the possibility to view the impact of proposed changes to the data
integration environment
Data Integration
Creates a better understanding what the data means, where it came from, where it is used
and how it has been transformed
Business Value of Data Lineage
![Page 14: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/14.jpg)
Group Data layer
Group Reporting and Calculation
** ** ******
**** **** **
** **
GROUP
** ** **
****
**
**
**
**
Group Risk and Finance sources
**
****
****
**
****
** **
****
******
** **
**
Group
** **
LBB Risk LBB Finance
**
**
**
**
**
**
**
**
****
**
**
**
** ** **
**
**
**
**
BU Reporting and Calculation
BU Data Layer
BU Risk or finance sourcesWRR
** Data Exchange function
**
Data Warehouse level
Back / Mid office level
Front office level
FinancierenSparen Beatalingsverkeer Klant
** **
**
**
**
**
**
** **
**
**
** **
**
**
**
**
** **
**
**
**
**
****
* *
**
**
********
Dat
a Fl
ow
** System names are masked for security reason
PowerCenter
Scope of Systems f0r Data Lineage
14
![Page 15: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/15.jpg)
EDC — GUI interface
15
![Page 16: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/16.jpg)
Using EDC for ComplianceUse case for BCBS#239 compliance
*BCBS#239 is Regulatory compliance on Data Governance
![Page 17: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/17.jpg)
Manual vs Automated Lineage
17
Existing situation manual lineage in Excel
Automated lineage in EDC
![Page 18: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/18.jpg)
Understanding Lineage Can Be Too Complex
18
![Page 19: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/19.jpg)
Deriving Business Lineage Out of EDC
19
![Page 20: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/20.jpg)
EDC for Data GovernanceUse Case, Integrating EDC into Data Governance Center
![Page 21: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/21.jpg)
Integrated View in Data Governance Center
21
Data Lineage starts in EDC
Reporting Repository
Business Data Element
Logical Data Attribute
Physical Attribute
FIND
YOU
R DATA
![Page 22: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/22.jpg)
Horizontal Lineage in EDC— Technical Lineage
22
FOLLOW YOUR DATA
![Page 23: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/23.jpg)
• EDC catalog search is very powerful like Google for enterprise catalog
• EDC has a big list of connectors
• EDC has open API, makes it easy to integrate with other tools
• Technical data lineage can be too much information for business users
• When you are looking at a big lineage diagrams, it is not possible to restrict lineage for selected tables
Lessons Learned
23
![Page 24: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/24.jpg)
• Continue expanding data lineage for enterprise
• Adding business terms to data attributes
• Using data domain discovery for privacy regulations
• Deriving business lineage out of EDC
EDC Roadmap for Rabobank
24
![Page 25: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/25.jpg)
2525
Bedankt voor uw aandacht!
![Page 26: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/26.jpg)
Demo
![Page 27: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/27.jpg)
27 © Informatica. Proprietary and Confidential.27 © Informatica. Proprietary and Confidential.
Learn More
1. Don’t miss Keynotes and Deep-Dives at the AI-Powered Data Cataloging Virtual Summit:• Market and Analyst Perspectives featuring New York Life, Tableau, and Amalgam Insights
• Data Cataloging Solution Theaters featuring Maersk, Nissan, Rabobank and Biogen
2. Stop by an Informatica World Tour near you:• Chicago Sept-11 | Washington, DC Oct-15
• Frankfurt Oct-8 | London Oct-9 | Paris Oct-10
3. Watch a Product Webinar:• Advancing Analytics Maturity with an Intelligent Data Catalog: with Mattel and Aberdeen
• Meet the Expert PM Webinar: EDC 10.2.2 Release Deep-Dive & Demo
![Page 28: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data](https://reader034.fdocuments.us/reader034/viewer/2022042112/5ec5e541dd25fa06af0cb7cc/html5/thumbnails/28.jpg)
`
Thank You