Big Data Analytics:Profiling the Use of Analytic Platforms in User Organizations

Wayne EckersonDirector of Research, Business Applications

and Architecture Media GroupTechTarget

Sponsors

Why Big Data?

1. Changing data types

2. Technology advances

3. Insourcing & outsourcing

4. Developers discover data

Analytics against Big Data

• Patterns• Real-time• Complex calculations• Sustainable advantage

Framework for success

Culture

Business Executives

Perfo

rman

ce M

easu

rem

en

tFa

ct-b

ase

d D

eci

sion

s

PeopleC

asu

al U

sers

Power Users

IT p

rofe

ssio

nals

Organization

BI G

overn

an

ceD

ata

Govern

an

ce

Analytics Center of Excellence

Architecture

Even

t-driv

en

Rep

ort

ing

Analytics

Analytic Platform

Analytic PlatformsAn analytic platform is a data management system

optimized for query processing and analytic that provides superior price-performance and availability compared with general purpose database management systems.

Yes72%

No28%Have you purchased or

implemented an analytic

platform as defined in this

survey?

Analytical Techniques

• MPP• Balanced configurations • Storage-level processing• Columnar storage and compression• Memory• Query optimizer• Plug-in analytics

Types of Analytic Platforms

Technology Vendor/ProductMassively Parallel Processing (MPP) Analytic Databases

Teradata Active Data Warehouse, Greenplum (EMC), Microsoft Parallel Data Warehouse, Aster Data (Teradata), Kognitio, Dataupia

Columnar Databases Paraccel, Infobright, Sand Technology, Sybase IQ (SAP), Vertica (Hewlett-Packard), 1010data, Exasol, Calpont

Analytic appliances Netezza (IBM), Teradata Appliances, Oracle Exadata, Greenplum Data Computing Appliance (EMC)

Analytic bundles IBM SmartAnalytic, Microsoft FastTrack

In-memory databases SAP HANA, Cognos TM1 (IBM), QlikView, Membase

Distributed file-based systems

Hadoop (Apache, Cloudera, MapR, IBM, HortonWorks), Apache Hive, Apache Pig,

Analytic services 1010data, Kognitio

Nonrelational MarkLogic Server, MongoDB, Splunk, Attivio, Endeca, Apache Cassandra, Apache Hbase

CEP/Streaming Engines IBM, Tibco, Streambase, Sybase (Aleri), Opalma, Vitria, Informatica

Which type of analytic platform have you purchased or implemented?

Analytic Database

Analytic Appliance

Analytic Service

File-based Analytic System

46%

49%

5%

10%

Purchase requirements Faster queries

Store more data

Reduced costs

More complex queries

Higher availability

Quicker to deploy

Easier maintenance

Faster load times

More diverse data

More flexible schema

More concurrent users

Built-in analytics

39%

27%

48%

35%

30%

46%

41%

23%

22%

30%

22%

43%

70%

46%

33%

64%

47%

33%

53%

63%

24%

25%

47%

35%

56%

67%

56%

44%

67%

44%

44%

44%

33%

22%

56%

22%

36%

50%

50%

36%

21%

43%

7%

14%

64%

64%

7%

14%

File-based Analytic SystemAnalytic Service Analytic ApplianceAnalytic Database

Explicitly looking for this option?

Analytic Database

Analytic Appliance

Analytic Service

File-based Analytic System

38%

48%

0%

60%

Business Intelligence

12Analytics Intelligence

Con

tinu

ou

s In

tellig

en

ce

Con

ten

t In

tellig

en

ce

Data Warehousing

Ad hoc query, Spreadsheets, OLAP, Visual

Analysis, Analytic Workbenches, Hadoop

Analytic Sandboxes

Even

t-driv

en

Reports and Dashboards

MAD Dashboards

Data Ware-

housing

End-User Tools

Even

t-Driv

en

Ale

rts an

d D

ash

board

s

BI Delivery Framework 2020

Excel, Access, SAS, Visual Analysis

Ad hoc exploration

Dash

board

Ale

rts

Even

t dete

ction

an

d

corre

latio

n

CE

P, S

tream

s

Analytic Sandboxes

Design Framework

Architecture

Searc

h,

NoS

QL,

Java

Un

ivers

al In

form

ati

on

Acc

ess

Had

oop

, M

ap

Red

uce

Key-

valu

e

pair

in

dexe

sReporting &

Analysis

Reporting & Monitoring (Casual Users)

Predefined

Metrics

Corporate Objectives and Strategy

TOP DOWN- “Business Intelligence”

Processes and Projects

Analysis and Prediction (Power Users)

Ad hoc

queries

BOTTOM UP – “Analytics Intelligence”

Analysis Begets

Reports

Reports

Beget

Analysis

Pros:

-Alignment

-Consistency

Cons:

-Hard to build

-Politically charged

-Hard to change

- Expensive

-“Schema Heavy”Pros:

-Quick to build

- Politically uncharged

- Easy to change

- Low cost

Cons:

-Alignment

-Consistency

--“Schema Light”

DW

Architecture

Non-volatile

data

Analytics

Architecture

Volatile

data

BI Architecture - 2020

Machine

Data

Web Data

Hadoop Cluster

Operational Systems

(Structured data)

Power User

BI

Server

Casual UserOperational

System

Operational

System

Documents & Text

Upload & query

Ad hoc query

Query & report

Free-

Standing

Sandbox

Dept

Data

Mart

Data Warehouse

Virtual SandboxesTop-down Architecture

Bottom-up Architecture

In-memory

BI Sandbox

External

Data

Alerts

Audio/video

Data

Streaming/ CEP

Engine

Extract, Transform, Load

(Batch, near real-time, or real-time)

Reports /Dashboards

Ad hoc query

Ad hoc query

Analytical platform or non-relational

database

BI Architecture - 2020

Machine

Data

Web Data

Hadoop Cluster

Operational Systems

(Structured data)

Power User

BI

Server

Casual UserOperational

System

Operational

System

Documents & Text

Upload & query

Ad hoc query

Query & report

Free-

Standing

Sandbox

Dept

Data

Mart

Data Warehouse

Virtual SandboxesTop-down Architecture

Bottom-up Architecture

In-memory

BI Sandbox

External

Data

Alerts

Audio/video

Data

Streaming/ CEP

Engine

Extract, Transform, Load

(Batch, near real-time, or real-time)

Reports Dashboards

Ad hoc query

Ad hoc query

Analytical platform or non-relational

database

Recommendations1. Harmonize top down and bottom up BI2. Implement a BI architecture that

supports multiple intelligences3. Create multiple types of analytic

sandboxes 4. Implement analytic platforms that

meet business and technical requirements