Making Big Data a First Class citizen in the enterprise

© Copyright Ovum. All rights reserved. Ovum is an Informa business.1

Making Big Data a First-class Citizen in the Enterprise

Tony Baer

[email protected]

IT014-002860

January 24, 2014

mailto:[email protected]


Contents

The premise

What we mean

Scope – focus on Hadoop, the most popular emerging Big Data platform

Addressing the enterprise

IT organization

Data center infrastructure, processes, policies, and practices

Creating value; competitive benefits to the business

The endgame


The premise: Voting Big Data off the island!

Big Data must become a first-class citizen in the enterprise and cannot exist on its own island.


Hadoop user base and use cases are changing

Users are changing from Internet companies to mainstream enterprises.

Use cases are changing from Internet search, ad optimization to customer churn analysis, sales and promotions, and operations.


What does it mean for Big Data to become a first-class citizen?

IT organization

No more SWAT teams! Must map to existing people and skills

Data center

Must map to existing infrastructure, subject to same constraints

Enterprise

Must address real business problems, not abstract data science research


Contents

The premise


IT organization



The endgame


Mapping Big Data skills to the IT organization

Enrich, don’t replace, your existing app developers, DBAs, system administrators

Huge existing SQL skills base – you’re not going to replace them

Large Java developer base, lots of scripting language diversity

Popularity of JavaScript/JSON

Skills:

Technology – the easy part

Domain and data science – not so easy

Don’t forget the people part!


Extending the IT organization for Big Data

SQL and NoSQL/Hadoop platforms are converging

SQL access to Hadoop

Hadoop platform SQL support

BI tool Hadoop support

MapReduce approaches to Advanced SQL platforms

MongoDB, CouchDB, Riak

Empowering web JavaScript developers with familiar JSON

Data science?

The apps are coming…

SQL on Hadoop and Big Data apps are works in progress…


Addressing the data center

Like most Internet technologies, Hadoop conceived in zone of trust

Small, elite band of practitioners

Big concern? Getting access to available cluster resources elsewhere inside the firewall

Enterprise?

Security

Data stewardship

Coping with finite resource

Availability and reliability


Data center: Big Data must be secure like any database management system or data warehouse

AAA enforced for access, authentication, authorization

Must become more granular by user, data

Must become more unified

Integrate with LDAP/Active Directory

Data privacy mandates

This is a policy, not a technology, issue

“Don’t be creepy” – don’t blindside your customers based on knowledge they didn't know you have

Regulation plays driving role for some sectors


Data center: Big Data platforms must behave like any database management system or data warehouse

Data stewardship/lifecycle

Data quality, protection, lifecycle management, retention

Resource management

Capacity utilization critical

Availability/reliability

Performance management essential for large clusters

Major change from early Internet adopters


Address the business

Good business cases count!

Do:

Focus on existing problems (the problems are often more obvious than you think…)

Identify key points of pain, like any new IT solution

Don’t

Concoct “interesting” data science problems for the heck of it

Get carried away with data (with lots of data, there are lots of chances for detecting irrelevant trends)

Give up after a few tries …. iterate!

Don’t get caught up in a data science project


Hadoop benefits: Solving familiar problems in new ways

Customer holistic view

Predictive churn analysis

upsell/cross-sell, next-best-offer,

cross-channel ID resolution

Risk mitigation

Fraud detection, counter-party risk

management, credit scoring

Operational efficiency

Machine data for managing smart

grids, smart urban infrastructure, supply chain

logistics

Not arbitrary data science


Contents

The premise

What we mean

Scope – focus on Hadoop, the most popular emerging Big Data platform


IT organization



The endgame


The endgame: What becoming a first-class citizen really means

Big Data – and emerging platforms like Hadoop – originated as specialized IT systems requiring specially skilled practitioners.

This model is not sustainable as Big Data crosses over to the enterprise.

Big Data must get off its island.

Big Data must be accessible to the IT organization, fit into the data center, and address real business problems.


Big Data: Embrace and extend

IT organization

Embrace existing SQL, Java, and other programming language skills

Extend skills to understand handling of larger volumes and varieties of data and new analytic techniques to supplement SQL

Data center

Embrace existing policies and practices for data stewardship, resource management, security, performance management

Extend policies and practices to accommodate platform with different workload characteristics, and support of active archiving

Business

Embrace existing competitive problems; don’t look for new problems because the data and platform are different

Extend approaches to problem solving by incorporating new data types and new forms of analyses to deepen understanding and insights

Making Big Data a First Class citizen in the enterprise

Technology

Transcript of Making Big Data a First Class citizen in the enterprise