The First Step in EIM: Big Data & Big Data Governance · 2013. 11. 6. · • Data driven mind-set...
Transcript of The First Step in EIM: Big Data & Big Data Governance · 2013. 11. 6. · • Data driven mind-set...
Proprietary & Confidential
The First Step in EIM
Big Data &
Big Data Governance
pg 2 Proprietary and Confidential
Table of Contents
• Big Data Overview
• Enterprise Information Management
• Big Data Management • Big Data Governance
• Ensuring Success
pg 3 Proprietary and Confidential
[ BIG DATA OVERVIEW ]
pg 4 Proprietary and Confidential
Big Data Definition
• Extremely large data sets that can’t be dealt with using traditional technologies
• Can be structured, non-structured or multi-structured
• Key characteristics:
Volume
Velocity
• Types of Big Data
Web and Social Media
Machine Generated
Biometrics
Variety Value
Unstructured Content Transactional
pg 5 Proprietary and Confidential
Business Drivers for Big Data
pg 6 Proprietary and Confidential
Big Data Industry Landscape
Industry Analysis
Computer and Electronic Products
and Information Sectors
• Computer and electronic products and information sectors have already been experiencing strong productivity growth and are poised to gain significantly from the use of big data
Finance, Insurance, and Government
• Finance, insurance and government are positioned to benefit as well, as long as barriers can be overcome e.g. overall ease to capture data, talent, IT infrastructure, low IT investment, data driven mind-set, data availability, etc.
Health Care • Health Care has shown early success in the use of big data
Retail and Consumer Products
• Big data offers significant new opportunities to create value (higher margins and productivity) in the retail industry
Manufacturing • Manufacturing has historically been a productivity leader, and big data can help
extend gains. Manufactures can use big data across the value chain
Public Sector
• The public sector faces a significant performance challenge due to : • Lack of talent • Data driven mind-set • Low IT investment
• The public sector leaders need to address these issues to use big data effectively
Researches state that, while the use of big data will matter across industry sectors, some sectors are set for greater gains. Opportunities and challenges vary from sector to sector:
Source Big Data: The Next Frontier, McKinsey Global Institute
pg 7 Proprietary and Confidential
Big Data Importance
• When integrated with other enterprise data, organizations can develop more insightful understanding of their business which can lead to:
A stronger competitive edge Improve business processes Greater product innovation and improvements Increase in growth and revenue Increased employee productivity through streamlined business processes
Source Big Data: The Next Frontier, McKinsey Global Institute
pg 8 Proprietary and Confidential
Implications of Big Data
• How will organizations have to be designed, organized, and managed?
• What existing business models are likely to be disrupted?
• How will organizations’ legacy business models and technology compete?
• How will business processes change?
• How will marketing functions and activities have to evolve?
• How will organizations leverage and value their data assets?
• How will executives help their organizations take advantage of the change that is under way?
• Where do they start and how? Current technologies and data management structures in organizations no
longer work in this new era of big data
pg 9 Proprietary and Confidential
[ ENTERPRISE INFORMATION
MANAGEMENT ]
pg 10 Proprietary and Confidential
A Comprehensive Framework
Provides a holistic view of data in order to manage data as a corporate asset
Enterprise Information Management
Information Strategy
Architecture and Technology Enablement
Content Delivery
Business Intelligence and Performance Management
GOVERNANCE
ORGANIZATIONAL ALIGNMENT
Content Management
Data Management Information Asset Management
pg 11 Proprietary and Confidential
Develop and execute architectures, policies and procedures to manage the full data lifecycle
How Big Data Fits
Enterprise Data Management Ensure data is available, accurate, complete and secure
Data Quality Management Data Architecture Data
Retention/Archiving
Master Data Management
Big Data Management
Metadata Management
Reference Data Management
Privacy/Security
DATA GOVERNANCE
pg 12 Proprietary and Confidential
[ BIG DATA MANAGEMENT ]
pg 13 Proprietary and Confidential
Foundation to Harness Internal and External Data
BDM provides foundational capabilities to integrate and analyze data from non-traditional data sources in order to find insights in new types of data
Process Automation
Architectural Improvements
Flexible Data Architecture
IT Transformation
and Adaptability
PAST PRESENT FUTURE
Transaction Management
Data Warehousing
Master Data Management
Integrated Information Management and Delivery
Process automation and management of transactions with application specific data within isolated business applications including ERP, CRM, SCM, eCommerce and other systems over the past decade
Data extraction and normalization for operational as well as management reporting and functional analytics. Data integrity and lack of standards have constrained the maturity of analytics in the past
MDM is management of foundational data domains that support core business processes, information and insight creation. It provides for flexibility data integration, directly supporting enterprise information architecture vision
EIM and adaptive architecture to deliver business capabilities and flexibility to future changes
Big Data Management
BDM is integrating and managing big data and its relationship across the enterprise through people, processes and technology. It provides opportunity to find insights in new types of data and content, to make organizations more agile, and to answer questions that were previously considered beyond reach
pg 14 Proprietary and Confidential
Big Data Lifecycle Process
Listen Capture Process
Integrate Analyze Consume
Measure Retain Destroy
pg 15 Proprietary and Confidential
Big Data Infrastructure & Architecture
As with any IT platform or a data warehouse, an infrastructure for big data has unique requirements. The end goal is to easily integrate big data with enterprise
data to allow complex and deep analytics
Distributed File Systems
Key Value Store
MapReduce Solutions Analysis & Reporting
DBMS OLTP
ETL Data Warehouse
No SQL Flexible
Specialized Developer-
centric
SQL Trusted Secure
Administered
Listen & Capture
Process & Integrate
Analyze, & Consume
Data Mining Dashboards
Measure & Report
pg 16 Proprietary and Confidential
Types of Data
Data Disciplines are expanding. Most types of data are not completely independent. Big Data often has a relationship to other data types. Management of these data sets addresses: • Data Quality
• Enrichment /Enhancement
• Relevance
• Privacy and Security
• Governance
Small Data
Big Data
Reference Data Master Data
Metadata
pg 17 Proprietary and Confidential
Data Types Work Together
Master Data
Enhanced/Enriched
Master Data (360 degree View)
Examples: • Social Media Influence • Social Media Account IDs • Demographic Information • Relationships • Email IDs • Validated Master Data
Examples include: • Address validation
through location broadcasts and geo-location data
Big Data (Interaction Data)
Big or Small Data
(Transactional Data)
Reference Data
(Statistic non-volatile data)
pg 18 Proprietary and Confidential
[ BIG DATA GOVERNANCE ]
pg 19 Proprietary and Confidential
Data Governance Definition
Data Governance is the organizing framework for establishing strategy, objectives and policy for effectively managing corporate data.
It consists of the processes, policies, organization and technologies required to manage and ensure the availability, usability, integrity, consistency, audit ability and security of your data.
Communication and Metrics
Data Strategy
Data Policies and Processes
Data Standards
and Modeling
A Data Governance Program consists of the inter-workings of strategy, standards, policies and communication.
pg 20 Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model • Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change Management
Data Governance Components
pg 21 Proprietary and Confidential
Competing Priorities
Business Insight
Security & Control
pg 22 Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model • Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change Management
Strategy
licocntta
etatata
Mass Communication • Individual Updates
Training• Stakeholder Management & Communication
i
• Extension of overall Data Governance Strategy and Scope
• Business purpose and value unique of Big Data
• Understanding of impacted business processes and key requirements
• Incorporation of new risks
pg 23 Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model • Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change Management
Organization
Ruless
dardsTaxon
g andg, and
• Training Strategy
•• IT• Tr• Stak• Defining
• New Stakeholders
• Extended participation at all levels to include Privacy, new Lines of Business
• Extended RACI to cover new data types
• Redefine role and scope of Data Steward; identify new stewards
• New roles (i.e. Data Scientists)
• New Regions
pg 24 Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model • Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change Management
Policies, Processes & Standards
s
onon
act && Rearene
Managership
CChhManaaaa
• Extension of Security, Privacy Policies
• Policies around data masking in testing and/or production, and “unmasking”
• Understanding of Intellectual Property considerations and Appropriate Use
• Extension of Data Retention Policy Archiving Storage Disposition
• Policy Enforcement
• Metadata, Classification
• New Definitions and Terms
pg 25 Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model • Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change Management
Measurement & Monitoring
P i d C fid i l
• S• T• M• C• S
& Readinesseadiness ess gement & Cp & Accoun
hhhhhhange aaaaaagement
• Re-evaluate Data Quality Standards, Thresholds and Metrics
• Data Availability requirements and monitoring
• Data Profiling rules & processes
• Monitoring of data movement and usage
• Track security, privacy
• Web metrics
pg 26 Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model • Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change Management
Technology
•••
•
•
d Analysis rogress f issues mprovemeg
• Integrating existing and Big Data Technologies, i.e. Master Data Management
• Big Data Lifecycle Management Data Compression & Archiving Requirements Regulatory Retention Requirements for Big Data Business Retention Requirements Data Volumes & Cost
• Metadata requirements
• New Sources
pg 27 Proprietary and Confidential
• Vision & Mission • Objectives & Goals • Alignment with Corporate
Objectives • Alignment with Business
Strategy • Guiding Principles
• Statistics and Analysis • Tracking of progress • Monitoring of issues • Continuous Improvement • Score-carding
• Policies & Rules • Processes • Controls • Data Standards & Definitions • Metadata, Taxonomy,
Cataloging, and Classification • Operating Model • Arbiters & Escalation points • Data Governance
Organization Members • Roles and Responsibilities • Data Ownership &
Accountability
• Collaboration & Information Life Cycle Tools
• Data Mastering & Sharing • Data Architecture & Security • Data Quality & Stewardship
Workflow • Metadata Repository
• Communication Plan • Mass Communication • Individual Updates • Mechanisms • Training Strategy
• Business Impact & Readiness • IT Operations & Readiness • Training & Awareness • Stakeholder Management & Communication • Defining Ownership & Accountability
Change Management
Communication
&veenveeny
g P
ta Owncou
• Extended Communication Plan, Awareness & Education
• New Stakeholders • Enhanced Goals, Priorities,
Concerns and Objectives
pg 28 Proprietary and Confidential
Big Data Governance & Technology Work Together
Big Data Strategy
Standardized Methods and Data Definitions (Metadata)
Roles and Responsibilities
Decision Rights, Arbiters and Escalation, Ownership & Accountability
Big Data Policies (Security, Privacy, Access, Retention)
Statistics / Analysis / Monitoring, reporting, Consumption
Filtering & Cleansing
Enrichment
Translation/Transforming
Run Algorithms
Data Processing Manipulation & Sorting
Data Analysis
Aggregate Results
Retain
Provide Guidance
Track Progress
Big Data Governance Big Data Technology
Create & Enforce Policies
Provide Feedback
pg 29 Proprietary and Confidential
Ensuring Success
• Organizational leaders must start identifying and assessing opportunities
• Leaders must understand the value in big data as well as how to unlock this value
• Leverage existing governance capabilities
• Determine both process and technical integration requirements
• Take a phased and iterative approach
“Perspectives on “data” as a single, amorphous resource will have to give way to more granular optimization effort that recognizes the complex variations of information that,
collectively, paint a picture of a given customer audience.” Winterberry Group Whitepaper – Marketing Data Governance - July 2013
pg 30 Proprietary and Confidential
[ SECTION TITLE ]
Proprietary & Confidential
[ QUESTIONS? ]
Proprietary & Confidential
The First Step in EIM
Contact Info
Kelle O’Neal [email protected]
415-425-9661 www.firstsanfranciscopartners.com
@1stsanfrancisco