BI Innovation Summit_Stetzler_2014_11_12

© 2013 YP Intellectual Property LLC. All rights reserved. YP, the YP logo and all other YP marks contained herein are trademarks of YP Intellectual Property LLC and/or YP affiliated companies. All other marks contained herein are the property of their respective owners.

EDW – A Dying Breed? In an Age of Big Data what is the role of traditional BI?Business Intelligence Innovation Summit, ChicagoEric Stetzler, YP

November 12, 2014


2

Overview

• This is a story of transformation.

• Transformation of YP as a company.

• Transformation of our data systems.

• Evolution of BI as we transition to a “big data” environment.

• Lessons learned and insights from our journey.


COMPANY TRANSFORMATION

3


4

YP started over 100 years ago as a publisher of print yellow pages.

• Mission: Help local businesses and communities grow

• Revenue $2 Billion +

• 5,500 + employees

• 2,000 + local sales reps

• 50 state footprint

• Spun out of AT&T several years ago

We are in the middle of a profound transformation …. In 2015

YP’s digital revenue will surpass print for the first time.


5

The Biggest Internet Company Nobody’s Ever Heard Of.

• Digital ad revenue $1 Billion +

• 80M monthly visitors on web and mobile

• YP.com is a top 30 web site in US

• 3rd largest in mobile ad revenue

• One of the largest Google Ad Words resellers

Our business generates a high volume of data, which required a transformation in data technologies.

Online Advertising

Mobile Advertising


DATA SYSTEMS TRANSFORMATION

6


7

Org Perspective – Where I fit In

Data Services

Technology

BI Product Management

Business Analytics

Finance

Business Analytics & BI • Lead a BI Prod. Mgmt. team

• Only team that uses the BI name

• Focus on requirements

• Not a technologist

• Led EDW team for several years

• Customer perspective

• Not a presentation about technologies


8

Data technology transformation parallels corporate evolution in three phases.

Reporting centric EDW centric Data Hub centric

Business Objects

Rep

ortin

gG

ridED

W

Ingenio AT&T Interactive YP

Distributed Data Processing (Clickstream data)

Data Hub(All data)

1 2 3

Domo IBICubesSQL Marts

SQLServer Vertica Vertica

Hadoop + Hadoop +


9

By the numbers

Records Processed Per Day* 2-3 Billion

Volume Processed Per Day 2-3 TB

Total Data Size 120 TB

Nodes - Production 40

Nodes - All Environments 200 +

Vertica Hadoop

* Searches, Impressions, Clicks …

4-5 Billion

5 + TB

3.5 PB

150

300 +


10

Fully Realized Data Hub

• LWES• Hadoop• HBase / APIs• Map Reduce • Pig / Hive• Impala

• Operational systems• Programmatic access• SAS / R• Interactive tools

100 + Data Sources

The Data Hub centralizes ETL for all data sources. EDW is one of many “clients” of the Data Hub.

Other Clients

EDW (Vertica)

Data Hub

Reporting Semantic

Layer (BO/IBI)


11

Semantic layers are a key enabler of our Enterprise BI strategy.

Business Objects Universe

• Formalize business logic• Designed for usability• Self-service• Controlled namespace• Data dictionary• Enable unlimited reports

Benefits


TWO MODELS OF DATA USAGE

12


13

Our Data Hub strategy enables two distinct models of data usage.

Domain-SpecificEnterprise

• Standardization of key corporate metrics or “outcomes”

• Capabilities: general-purpose base data for broad usage

• Pervasive use of data throughout the organization

• Applying data to specific business or technology challenges

1 2

• Cross-functional• Holistic view of business• Centralized, structured

• Task or department specific• Distributed, decentralized• Evolves organically

Gov

erna

nce

Def

initi

onD

ata/

Org • Aligned with traditional BI

• Powered by EDW• Aligned with Data Hub• Proliferation of data sources


14

Two models of data usage - characteristics

• Data quality• Precision / accuracy• Consistency / Robustness

Domain-Specific

• Speed !!!• Directionally correct • Discovery / Experimentation

Enterprise

Driv

ers

21

• Data Scientists• Technical analysts• Operational Systems

• Executives / BOD• Business Managers• Business AnalystsU

sers

The Domain-Specific model will proliferate rapidly. What are the implications for Enterprise BI?

Vect

or • Evolving • Accelerating


IMPLICATIONS FOR ENTERPRISE BICHALLENGES & OPPORTUNITIES

15


16

A Data Hub Strategy Creates Opportunities for the Enterprise Model.

Accelerates EDW Build-Out.

Promotes EDW Data Quality via ETL Centralization & Specialization.

Enables a positive feedback loop between the two models of data usage.

1

2

3


17

New Dimension

Examples of positive feedback loop.

Domain-SpecificEnterprise1 2

Revenue, Canceled Customers

• Evaluate effectiveness of retention programs

• Segment customers by propensity to churnR

eten

tion

Ana

lysi

sVa

lue

Man

agem

ent

Data Inputs

Customer Segmentation

• Model if advertisers are getting good value

• Develop targets for click and call delivery

RevenueClicksCalls

Customer Value Buckets

New Dimension

Data Inputs


18

A Data Hub Strategy also Creates Challenges for the Enterprise Model.

• EDW is one of many clients of Data Hub

• Tradeoff investment in EDW vs Data Hub

Competition for Dev Resources

• Using non-standard inputs or outputs in models.

• Wrong conclusions, “shooting from the hip”

• Gaming the system

Risk of Increased Data Chaos or Going Rogue

1

2

There can be tension in an organization between the two models of data usage, which has to be actively managed.


19

Examples of tension between the two models of data usage.

Domain-SpecificEnterprise1 2

Wanted to change the definition of churn to make retention programs look betterR

eten

tion

Ana

lysi

sLe

ad G

en Revenue

New Advertisers

Focused on metrics liked leads delivered versus ultimate “outcomes”

AdvertiserChurn

BI needs to play a key role in mitigating these tensions.


20

Summary

• A Data Hub strategy creates both opportunities and challenges for your Enterprise BI strategy.

• BI needs to actively promote alignment between the two models of data usage we reviewed.

• The traditional BI function becomes even more critical in an environment of increased data chaos and potential to “go rogue”.

• It is critical for BI to have a comprehensive, long-term vision to navigate this changing landscape.


THANK YOU!

21

Eric StetzlerExecutive Director, Analytics & BI Product [email protected]

mailto:[email protected]

BI Innovation Summit_Stetzler_2014_11_12

Documents

Transcript of BI Innovation Summit_Stetzler_2014_11_12