Unleashing the value of metadata with Talend

©2016 Talend

Unleashing the value of Metadata with Talend

The Mandate for Metadata Management

2 Auditability

Accessibility

Agility IT needs to move faster and innovate more than ever

More regulations and need for accountability need a map to track and trace the data journey

Strong governance as a prerequisite to engage a wider audience of business users with self service

e.g. in Banking: Risk and Finance data aggregation and reporting (Basel III BCBS 239)

The need for governance and data lineage

Impact of Risk Data & Technology Programs

21Bof annualbenefits

Typical budget For Platforms

125MInvestment

>2%reduction

Reduced capitaland operations costs

Source: Mc Kinsey & Company: Capturing value of BCBS 239 and beyond

Why Now ?

“The metadata management tool market will double in size in 2017, driven by the

information of everything, proliferation of data from the Internet of Things (IoT), big data

and data lake forces, and requirements for organizations to know what data they have, as

information is recognized as an asset with an attached financial value.”

The five pillars for managing Metadata with Talend

Talend

Studio

Talend

Metadata

Bridge

Talend Big Data

With Cloudera

Navigator

Talend Data Preparation

Talend Metadata Manager

Metadata-driven, with zero coding

Single point of control for Metadata

Dependency checks and propagation

Design once, propagate across any tools

Apply change and enforce standards

Accelerate migrations

Data Lineage with depth on Hadoop

Track, classify, and locate data & data flows

Data Governance on Hadoop

Data inventory for self-service access

Auto-discovery with smart guidance

Facet based search

Risk and compliance with data transparency

Increase agility across the data landscape

Turn data into a business language

(*) Apache Atlas integration planned in Talend 6.3

Metadata-Driven Studio

Transformation path from source to target

Transformation lineage from target back to source

Not just files & tables

- Fined-grained metadata to the attribute- Automatically propagate changes

Export & share

https://help.talend.com/display/TalendDataFabricStudioUserGuide61EN/V.+Metadata+Management

https://help.talend.com/display/TalendDataFabricStudioUserGuide61EN/V.+Metadata+Management

Talend Metadata Bridge

• Available from Talend 5.6.2

• Over 100 connectors to import/export metadata from virtually any data-driven tools and platforms

• Capabilities• Imports/Exports from design and

modeling environments• Integrates with Databases, Big Data and

development platforms• Reduces update & conversion costs

and accelerates ETL offloading projects• Facilitates IT and Lines of Business

collaboration for data flows specification

Hadoop needs Metadata Management

Lots of dataLots of Data Types

Lots of LanguagesLots of Frameworks

One data,many replicas, many models

+

Hadoop meets Data Governance with Cloudera and Talend

The first and only Cloudera Navigator integration that includes Spark Data Lineage

A visual approach for data flow design, fully integrated with Cloudera Navigator

Depth, breadth and reach for Hadoop Data Governance

End-to-end data traceability, including upstream before the data reaches Hadoop

Business users empowerment with metadata through business glossaries and data catalogs

Deep control over your Hadoop flows

Extend beyond Hadoop

Reach new audience with Big Data

Reclaim scattered files from users’ hard drives

Single point of inventory and access for self service data

Trace source, ownership, actions

Improve accessibility, Auto-discover, recommend

Data Inventory (from Talend Data Preparation 1.2)

Govern

Collaborate

Control

Guide

Talend Metadata Manager

• Available from Talend 6.1 (Dec, 2015)

• Allows to harvest, govern, and share metadata across data platforms (BI, Big Data, modelling tools, ETL, etc.)

• Provides : • Business Glossary

• Shared definition and collaboration around data definition and policies

• Enterprise Data Governance & Auditing• Data Flow Lineage and impact analysis, across

systems and tools

• Ensures Data Compliance• through model versioning, comparisons and

reporting

Talend Metadata Manager – Integrated Solutions

IT and corporate servicesLines of Business

Metadata Management

Solutions

Data Governance

Solutions

Any Enterprise Architecturefrom the Data Lake

to the Data Warehouse

Solutions to both business and technical users

With any vendors/tools

Talend Metadata Manager – The Big Picture

For today’s evolving

Enterprise Architectures

integrating

the Data Warehouse

Sem

an

tic L

ineag

e

Data FlowLineage Impact Analysis

Data

Warehousing

Data

Integration

Business

Intelligence

Data Governance

Data Standardization

Enterprise Architecture

Business Glossary

Semantic Mapping

Conceptual & Logical Models

Design Patterns & Reusable Components

Data Stores

& MDM

Bu

sin

ess A

pp

lica

tions

Bu

sin

ess R

ep

ort

s

and

the Data Lake

both

Talend Metadata Manager – Metadata Harvestinghttp://www.metaintegration.com/Products/MIMB/SupportedTools.html

Talend Metadata Manager – Enterprise Architecture

Talend Metadata Manager – Metadata Browsing

Talend Metadata Manager – Lineage & Impact Analysis

Talend Metadata Manager – Data Model Diagrams

Talend Metadata Manager – Data Governance

ISO 11179 standard based Business Glossary

Custom Attributes

Glossary Bootstrap from the existing enterprise data model in ERwin, and more

Bulk Change and Edit allowing for quick tabular editing and bulk changes to search results

Semantic Mappings

Governance Workflow provides a very flexible (customizable) workflow

Talend Metadata Manager –

Business Intelligence Tools

BI Report Documenter

Managed Business Intelligence: Live integration with BI Applications

Talend MM application as a powerful multi-vendor BI Web Portal

BI Report Documenter allowing users to

get live definitions from any objects on reports,

add them in place if missing,

trace data lineage from any object in reports,

find other related reports (even in other BI tools),

produce live report specific glossaries (e.g. Tableau worksheet)

Report Documenter

Business Glossary

Reuse

Terms

Create

Terms


Data Modeling Tools

Business Intelligence Tools

Metadata Authoring Apps:Data Modeler / Documenter

Data Modeler

Business Glossary

Data Stores

Active DataGovernance

DataStandardization

Automatic Live Updates of

any data store schema change

Reuse

Terms

Create

Terms

Forward Engineering to:

- to self service BI like Tableau

- or traditional BI Design Tools

SAP BusinessObjects UNV/IDT

IBM Cognos FM

MetadataHarvesting


Talend DI StudioTalend MM Data Mapper

Data MappingSpecifications

Metadata Authoring Apps:Data Mapper

Data Mapping Requirements Data Flow Implementation

Active DataGovernance

Data Flow Lineage& Impact Analysis

MetadataComparision

Requirement

Compliance

DI/ETL/ELT Tool

Generation

Data MappingDesign

Unleashing the value of metadata with Talend

Software

Transcript of Unleashing the value of metadata with Talend