The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps...
Transcript of The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps...
![Page 1: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/1.jpg)
1 | Proprietary & Confidential
Machine Learning Data Catalog:
The Foundation for Enterprise Data Literacy
Vince [email protected]
![Page 2: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/2.jpg)
2 | Proprietary & Confidential
Enterprises with the most advanced analytics capabilities outperformed competitors by wide margins, with leaders…
The innovators have already reinvented themselves
5xas likely to make
market decisions faster than peers
2xas likely to use data
frequently when making decisions
2xas likely to be in the top
quartile of financial performance
![Page 3: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/3.jpg)
3 | Proprietary & Confidential
![Page 4: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/4.jpg)
Surveys Describe the Problem to the Majority
Alarming results from the latest New Vantage Partners Survey:
• 72% have yet to forge a data culture• 69% have not yet created a data-driven org• 53% are not treating data as a business asset• 52% are not competing on data & analytics• The % of firms identifying themselves as
data-driven has declined for 3 years
![Page 5: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/5.jpg)
5 | Proprietary & Confidential
We Are Here
![Page 6: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/6.jpg)
6 | Proprietary & Confidential
![Page 7: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/7.jpg)
The ProblemFinding and using the right data
requires a lot of knowledge
![Page 8: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/8.jpg)
8 | Proprietary & Confidential
Time spent navigating systems and documentation
Emails, Chat
Business Glossary
Sharepoint / Wikis
Source Code
BI Tools
CSV Files
Databases
Hadoop
![Page 9: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/9.jpg)
9 | Proprietary & Confidential
And leads to falling backon tribal knowledge
![Page 10: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/10.jpg)
10 | Proprietary & Confidential
Analysis takes weeks to months rather than seconds
• Check the schema• Sample the data• Find primary & foreign keys
• Find business definitions• Write metrics• Validate accuracy
of the numbers
• Ask around• Send emails• Maybe find an outdated wiki
Find the right data(3-6 Weeks)
Understand the data(1-2 Days)
Trust the data(1-2 Days)
Write the query(1-10 Hours)
• Determine joins• Filter the data• Write your first query
Total time spent:
Up to 2 months
![Page 11: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/11.jpg)
11 | Proprietary & Confidential
Enhance the Analytical Productivity of each analyst and business user
by up to 50%
ProduceAccurate Documentation
by up to 40% faster.
Build business value by speeding analysis and achieving Reduced
Time-to-Insight
Data in hours that’s trusted and impactful
Standard Timeline 1-2 months
Total time spent: Up to 2 months
Timeline with a data catalog
Reclaimed time for deeper insights
1-2 days
![Page 12: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/12.jpg)
The Solution
![Page 13: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/13.jpg)
13 | Proprietary & Confidential
Catalogs enable you to quickly self serve
You used to goto a store
You used to have a rolodex and a network
You used to get help from a librarian
is a trusted catalogthat helps you
quickly surfthe web
is a trusted catalogthat helps youquickly find aprofessional
is a trusted catalogthat helps you learn
quickly about product sold on the web.
![Page 14: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/14.jpg)
What is a Data Catalog?
![Page 15: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/15.jpg)
15 | Proprietary & Confidential
An enterprise data catalog helps business and technical users quickly
find, understand and trust data.
![Page 17: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/17.jpg)
17 | Proprietary & Confidential
What Problems Can a Data Catalog Solve?
Find
Understand
Trust
Use
Re-Use
“Data is scattered across multiple sources and silos”
“It takes too long to find data associated with business problems”
“Every source has a different set of tools and interfaces”
![Page 18: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/18.jpg)
18 | Proprietary & Confidential
What Problems Can a Data Catalog Solve?
Find
Understand
Trust
Use
Re-Use
“KPI’s and glossary terms are disconnected from the data”
“We don’t know who is the SME to learn more about the data”
“We have to talk to many different people to understand the business
context”
![Page 19: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/19.jpg)
19 | Proprietary & Confidential
What Problems Can a Data Catalog Solve?
Find
Understand
Trust
Use
Re-Use
“We don’t have an effective way to communicate trusted or deprecated
data”
“Documentation is out of date and not trusted”
“We spend the first 15 minutes of each meeting validating definitions”
![Page 20: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/20.jpg)
20 | Proprietary & Confidential
What Problems Can a Data Catalog Solve?
Find
Understand
Trust
Use
Re-Use
“Each project starts from scratch”
“90% of the reports already exist somewhere but we can’t find them”
“It takes a long time for a new analyst / data scientist to become self
sufficient”
![Page 21: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/21.jpg)
21 | Proprietary & Confidential
What Problems Can a Data Catalog Solve?
Find
Understand
Trust
Use
Re-Use
“Inefficient queries are taxing our resources”
“We don’t have an effective way to share queries”
“We’d like to get more people to write queries like Ruth”
![Page 22: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/22.jpg)
22 | Proprietary & Confidential
By 2020, organizations that offer users access to a curated catalog of internal and external data will realize 2x the business value from analytics investments than those that do not.
Rita Sallam, Gartner VPMagic Quadrant for Business Intelligenceand Analytics Platforms
![Page 23: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/23.jpg)
Top 10 Recommendations for Selecting a Data Catalog
![Page 24: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/24.jpg)
24 | Proprietary & Confidential
10. Automatic Catalog Page Creation (multiple data sources; cloud & on prem)
9. Automatic Metadata Collection
8. Automatic Lineage Creation
7. Simple Communication of Trusted Data & Changes
6. Robust Business Glossary
5. Machine Learning (Auto-Titling of Data Objects in Natural Language; Search “Page Rank”)
4. Semantic Search
3. Simplicity - Clean UX, easy to navigate for technical and non-technical users
2. Collaboration / Crowdsourcing of Metadata
1. Automatic Identification of Behavioral Patterns (Top Users, Most Popular Data Objects)
Top Ten
![Page 25: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/25.jpg)
25 | Proprietary & Confidential
“The best advice I have for senior leaders trying to develop and implement a data culture is to stay very true to the business problem: What is it and how can you solve it?”
Keep Your Eyes on the Prize
Rob Casper, Chief Data Officer, JPMorgan ChaseMcKinsey Quarterly 2018
![Page 26: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/26.jpg)
26 | Proprietary & Confidential
“We are moving from a culture of reporting to a culture of analysis. We have to get this right. Everyone is a data analyst”
Nishant UpadhyayVP of Information & Data Management
American Family Insurance
![Page 27: The Foundation for Enterprise Data Literacy Data Catalog ... · An enterprise data catalog helps business and technical users quickly find, understand and trust data. ... “Each](https://reader034.fdocuments.us/reader034/viewer/2022042620/5f39ddf96a9d0223c347e74a/html5/thumbnails/27.jpg)
27 | Proprietary & Confidential
contains
contains
dupl
icate
of
populates
has visited
has queried
contains
has visited
publ
ished
populates
The best catalogs surface relationships between people and things