Smarter Analytics and Big DataBuilding The Next Generation Analytical insights

Joel WatermanRegional Director, Business Analytics Middle East & Africa

WATCH A MOVIE

DOWNLOAD AN APP

READ A PAPER

O’Reilly:27.4.2011, Got Iphone, Apple is recording your moves

TomTom Caught Selling Speed Data to Dutch PolicePcWorld 28. April 2011

Why Business Analytics Matter

CIOs rank analytics as the #1 factor contributing to an organization’s competitiveness.

– CIO study 2009

Financial outperformers are 64% more likely to use analytics to evaluate talent supply and demand on an ongoing basis.

– CHRO study 2010

Enterprises that apply advanced analytics have 33% more revenue growth and 12X more profit growth.

– CFO study 2010

Eight out of ten CEOs expect complexity to increase significantly in the next five years; insight and intelligence is ranked a top 3 priority.

- CEO study 2010

Top-performing enterprises use business analytics 5 times more than lower performers.

– 2010 joint study, MIT and IBM Institute for Business Value

Why Business Analytics MatterThe Need for Analytics is Pervasive Across Business and Industry

The healthcare industry spends $250 - $300 billion on healthcare fraud, per year. In the US alone this is a $650 million per day problem.1

One rogue trader at a leading global financial services firm created $2 billion worth of losses, almost bankrupting the company.

5 billion global subscribers in the telco industry are demanding unique and personalized offerings that match their individual lifestyles.2

$93 billion in total sales is missed each year because retailers don’t have the right products in stock to meet customer demand.

Source: 1. Harvard, Harvard Business Review, April 2010.

2. IBM Institute for Business Value, The Global CFO Study, 2010.

IBM is Making Significant Investments in Smarter Analytics

Investment

Technical Expertise

Technology

Research

Business Partners

$ 16B for 30 acquisitions since 2005 8 Analytic Solutions Centers worldwide

More than 10,000 technical professionals 9,000 consultants delivering IBM analytics solutions

Power7: workload optimized analytic processing Smart Analytics cloud

Comprehensive, heterogeneous Big Data platform for the Enterprise

World’s largest math department in private industry FOAK breakthrough innovations including IBM Watson

Number 1 in patent ranking for 19 years and more than 500 analytics-related patents / year for last two years

More than 27,000 Business Partner certifications

Imagine the Possibilities of Analyzing All Available Data

Real-time Traffic Flow Optimization

Fraud & risk detection

Accurate and timely threat detection

Predict and act on intent to purchase

Faster, More Comprehensive, Less Expensive

Understand and act on customer sentiment

Low-latency network analysis

• Optimize building energy consumption with centralized monitoring

• Automate preventive and corrective maintenance

Capabilities Utilized:• Streaming Analytics• Hadoop System• Business Intelligence

Applications:• Log Analytics• Energy Bill Forecasting• Energy consumption optimization• Detection of anomalous usage• Presence-aware energy mgt.• Policy enforcement

Cisco turns to IBM big data for intelligent infrastructure

management

Harnessing the Largest Predictive Focus Group in the World

Purpose

• Understand public sentiment towards an event: movie trailers

• Deeply understand the potential customer profile: gender, occupation, intent to watch

• Alter marketing launch plans based on insight

Background

• 1.1 Billion Tweets analyzed

• 5.7 Million blogs/forum posts

• 3.5 million messages

• Also: Facebook, Google+, Tumblr, Flickr

Conclusion – Actionable Insight

Adjust the marketing launch plan before execution begins

• Creative – adjust messaging

• Trailers – alter scenes shown

• Budget – re-direct, increase, or decrease

• Execution – theatre placement, advertisement

placement

IBM Big Data Platform

IBM Big Data Strategy: Move the Analytics Closer to the Data

BI /Reporting Exploration / Visualization

FunctionalApp

IndustryApp

Predictive Analytics

Content Analytics

Analytic Applications

IBM Big Data PlatformSystems

ManagementApplication

DevelopmentVisualization & Discovery

Accelerators

Information Integration & Governance

HadoopSystem

Stream Computing

Data Warehouse

New analytic applications drive the requirements for a big data platform

• Visualize all available data for ad-hoc analysis

• Development environment for building new analytic applications

• And a whole lot good solid IT stuff...

Big Data Platform - Hadoop System

• Manages a wide variety and huge volume of data

• Augments open source Hadoop with enterprise capabilities

• Performance Optimization

• Development tooling

• Enterprise integration

• Analytic Accelerators

• Application and industry accelerators

• Visualization

• Security

DC WATER

IBM’s Hadoop System provides unique business value

• Integration with enterprise systems

• Industry & application accelerators

• Templates for quick starting app dev

• Visualization tools to help business users to explore big data

Big Data Platform - Stream Computing

Built to analyze data in motion

• Multiple concurrent input streams

• Massive scalability

Process and analyze a variety of data

• Structured, unstructured content, video, audio

• Advanced analytic operators

Asian telco reduces billing costs and improves customer satisfaction

Capabilities:

Stream ComputingAnalytic Accelerators

Real-time mediation and analysis of

6B CDRs per day

Data processing time reduced from

12 hrs to 1 sec

Hardware cost reduced to 1/8th

Proactively address issues (e.g. dropped calls) impacting customer satisfaction.

Stream Computing provides unique business value

• Real-time answers = Better outcomes for time sensitive applications (e.g. fraud detection, network management)

• Solution when data is too large or expensive to store

• Analyze data as it comes to you

• Keep data of interest for deeper analysis

Big Data Platform - Data Warehousing

Workload optimized systems

• Deep analytics appliance

• Configurable operational analytics appliance

• Data warehousing software

Capabilities

• Massive parallel processing engine

• High performance (OLAP)

• Mixed operational and analytic workloads

Deep Analytics Appliance – Revolutionizing Analytics

Purpose-built analytics appliance

Speed: 10-100x faster than traditional systemsSimplicity: Minimal administration and tuning

Scalability: Peta-scale user data capacity

Smart: High-performance advanced analytics

Dedicated HighPerformanceDisk Storage

Blades With Custom FPGAAccelerators

Pacific Northwest Smart Grid Demonstration Project

Capabilities:

Stream Computing – real-time control system

Deep Analytics Appliance – analyze massive data sets

Demonstrates scalability from 100 to 500K homes while retaining 10 years’ historical data

60k metered customers in 5 states

Accommodates ad hoc analysis of price fluctuation, energy consumption profiles, risk, fraud detection, grid health, etc.

Data Warehousing provides unique business value

• Consolidate, manage and reconcile data for enterprise business intelligence

• Establish trust, quality and governance where necessary

• Financial data

• Credit card data

• Healthcare

• Combine deep and operational analytics

• Maintain history for trending and historical reporting

Image: David Castillo Dominici

Big Data Platform - Accelerators

Analytic accelerators

• Analytics, operators, rule sets

Industry and Horizontal Application Accelerators

• Analytics

• Models

• Visualization / user interfaces

• Adapters

Mining in Microseconds

Image & Video

Simple & Advanced Text

Text(listen, verb), (radio, noun)

Acoustic

GeoSpatial

Predictive

Advanced Mathematical Models

Statisticspopulation

tt asR ),(

Analytic Accelerators Designed for Variety

Accelerators Improve Time to Value

Data miningStreaming statistical analysis

FinanceStreaming options tradingInsurance and banking DW models

Public transportationReal-time monitoring and routing optimization

TelecommunicationsCDR streaming analyticsDeep Network Analytics

Over 100 sample applications

User Defined Toolkits Standard Toolkits

Social Media AnalyticsSentiment Analytics, Intent to purchase

Retail Customer IntelligenceCustomer Behavior and Lifetime Value Analysis

Industry Data Models Banking, Insurance, Telco, Healthcare, Retail

Telecommunications CDR Analytic Accelerator

Analyze Call Detail Records in real timeStreaming Analytic Accelerators

• Dropped call analysis

• Who are VIP customers with service issues – proactive alerts

• Analytic Operators – CDR de-duplication, dropped call detection, termination reason, customer importance

• Visualization – real-time KPI dashboard

Data Warehouse Appliance• Integrated network, devices, customer, and

services model

• Telecom model, KPIs, and KQIs Image: nokhoog_buchachon

Big Data Platform – Information Integration and Governance

Integrate any type of data to the big data platform

• Structured

• Unstructured

• Streaming

Govern big data

• Secure sensitive data

• Lifecycle management to control data growth

• Master data to establish single version of the truth

Information and Governance for Big Data

32

32

Leaders integrate and govern massive Big Data with InfoSphereMarketing Services Leader

integrates big data for customer intelligence

Capabilities Utilized:Information Integration – data quality, ETLDeep Analytics Appliance

Complex customer data integration for

54M records/hour

Processing

5B simultaneous records

Manages 2 petabytes of data, 20B records

Big Data Platform - User Interfaces

Business Users

•Visualization of a large volume and wide variety of data

Developers

• Similarity in tooling and languages

• Mature open source tools with enterprise capabilities

• Integration among environments

Administrators

• Consoles to aid in systems management

Visualization - Spreadsheet-style user interface

• Ad-hoc analytics for LOB user

• Analyze a variety of data - unstructured and structured

• Browser-based

• Spreadsheet metaphor for exploring/ visualizing data

Gather Extract Explore IterateCrawl – gather statisticallyAdapter–gather dynamically

Document-level infoCleanse, normalize

Analyze, annotate, filterVisualize results

Iterate through any prior step

Big Data Platform - Analytic Applications

Big Data Platform is designed for analytic application development and integration

BI/Reporting – Cognos BI, AttivioPredictive Analytics – SPSS, G2, SAS Exploration/Visualization – BigSheets, Datameer Instrumentation Analytics – Brocade, IBM GBS Content Analytics – IBM Content AnalyticsFunctional Applications – Algorithmics, Cognos Consumer Insights, Clickfox, i2, IBM GBS Industry Applications – TerraEchos, Cisco, IBM GBS

IBM’s big data business partner ecosystem

CC&G Partners100th Big Data

Business Partner Signed

IBM’s unique strengths in Big Data

Big Data in Real-Time

Fit for purpose analytics

Enterprise Class

Integration

WATCH A MOVIE

DOWNLOAD AN APPvisit ibm.com/iibv and download the

IBM Institute for Business Value app at iTunes and Android Marketplace

READ A PAPER

http://tinyurl.com/c8866jx