TIBCO Spotfire: Data Science in the Enterprise

From Data to Insights

in Internet Time

Data Science in the Enterprise

Eric Novik Industry Analytics Group TIBCO Spotfire

21st CENTURY PLATFORM

RV, EMS, FTL

ActiveMatrix

BPM

MDM

BusinessEvents

ActiveSpaces

LogLogic

In-Memory

Spotfire

Visual Analytics

Data Discovery

Statistics

Silver Fabric

Private Cloud

Public Cloud

SAAS Apps

tibbr

Social Media Collaboration

© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.

Are Your Models Behaving Badly?


Data Mashups

Exploratory Data Analysis

Features Spotfire

Dashboards Model & Predict

Deploy Champion

Model

Test & Learn

Sales

Loyalty

Social Media

Behavior

Filter

Map

Merge

Shape

Social

Network

Members

Improve Guided -------- Deploy -------- In-Line Explore Data

Aggregate

Prepare Data Business Case

Increase

Productivity

Grow

Revenue

Ensemble

Forest

Regression

Additive

Models Transaction

Visualize

Geo

Location

Service

Challenger

Models

At Rest

In Motion

Value Theses

Reduce

Risk

ROI

Transaction

Dashboard Updates

Data Science 2.0: Guided and In-Line Analytics


Time

The TIBCO Spotfire Experience Reduces Latency

Business

Value

Business Event

Data Ready for Analysis

Analysis Completed

Decision Made

$$$$

$$$

$$

$ Action Taken

TIBCO Spotfire

Analytics

Speeds Action

and Increases

Business Value


Data a Information a Knowledge a Wisdom

Single BI Platform

Traditional BI and Discovery

Design Data Models

Source The Data

Build Reports

Business Users Joe Data-Modeler

CRM ERP Financials

ETL

OLAP Cube

Requests

Data Warehouse

Iterate the Loop


Business Analysts Joe Data-Modeler

OLAP Cube

BI Platform

Guided, Self-Service Analytics

Design Data Models

Source The Data CRM ERP Financials

ETL

Data Warehouse

The Spotfire Experience: Guided, Self-Service Analytics

Spotfire 5.x

In-Database Analytics

In-Memory Analytics

Hybrid On-demand Analytics

Data Virtualization

TIBCO Enterprise Runtime for R


Dynamic Visual Analytics: Data and Models


My Kind of Pie Chart….


“Numerical quantities focus on expected values, graphical summaries on unexpected values.” - John Tukey, Exploratory Data Analysis, 1977.

Visual Analytics


Visualizing Risk


Visualizing Correlations and Linear Models


Customer Acquisition/Retention ► Understand transactions and attractive

products for individual investors ► Identify prospects through data-mashups

of in-house and industry sources ► Understand client dissatisfaction

#1. Business and Customer Insight

Customer Growth ► Identify persuadable customers for

cross-sell / up-sell programs ► Drive campaign efficiencies ► Discover sensitivities of acceptance to

demographics and behavior ► Test and learn marketing and loyalty

Business leaders and marketing groups need insight in to prospects and customer behavior and to identify “persuadables” for growing the business


Customer Life Cycle

Cross-sell / Up-sell Sentiment & Intent Analytics Sales Force Detailing Optimization Test & Learn

Market Mix Analytics Price Elasticity Campaign Effectiveness Attrition & Social Churn

Customer Acquisition

Customer Retention

Relationship Growth

Customer Lifecycle


Customer Growth & Cross-Sell

• Response

• Acceptance of product (wealth management services, credit card)

• Analysis Unit

• Customer

• Features

• Bank transactions: ATM, checking; deposit, withdrawal, balance, …

• Demographics: sex, age, profession, nationality

• Model

• Classification trees

• Association & machine learning

• Value

• Customers * Value * Conversion


Citibank saw a 21% increase in customer spend, and shortened time-to-market for new campaigns by 92% after implementing TIBCO Spotfire cross-sell programs

Citibank Asia

http://www.tibco.com/customers/#citibank-asia

http://www.tibco.com/customers/




Citibank COP: Delivered Business Functionality

COP360 Degree

Customer View

DWH

ACWS

AFEWS

CBOL

ECCM

GRB

ECS

Oracle

Master

data

download

Master

data upload

Offer

palette

Service Request &

Offer Feedbacks

Addr

change

Status

Changes

Offer

response

Financial &

Non-financial

transaction

Campaign

Definition, Offer

Request &

Feedbacks

Customer Citibank Phone Officer (CPO)

DM Staff Members

CTI

Offer

palette

Offers

Files

Campaign

Master &

Offer List

Business Rules

Definitions

KPIs, Performance

Metrics, Campaign

FeedbacksCampaign

Definitions &

Offer list

Status

Changes

Offer

Request &

Feedbacks

Offer Request &

Feedbacks

Offer

response

Offer

response

Offer

response

Offer

request

Login

Internet offers

Campaign

Definition,

Limited Master

data upload

FMS

Offer

Feedbacks

Campaign

Definition,

Offer

Feedback

Fulfillment

info

Intelligent Customer/Scenario Dialogue Real-time synchronization of customer contact / touch-points (inbound and outbound)

Dynamic Offer Management Real-time (re) scoring Real-time offer eligibility and prioritization

Conversion Tracking and Analysis Real-time dash boarding / tracking

Dynamic Channel Synchronization and Management Infrastructure to address time-sensitive offers (e.g.. real-time cross-offers on SMS based on card transactions at specific locations)

Sales Process Tracking Close-loop tracking of intermediate fulfillment processes

On-Board Measures, Statistics and Visibility Reliability, Availability, Scalability Hot – Dynamic Configuration


BMO saw a 15% increase in customer spend on other products after implementing TIBCO Spotfire cross-sell programs.

Bank of Montreal

http://www.bmo.com/home


BE Discover pattern in data

…

Build predictive statistical model

Deposits correlate with opening wealth account in some demographics

Predict new wealth mgt customer from account transfers and demographics

Export optimal rules/thresholds

and deploy with BE

Make predictions and act on them

at every customer interaction

Measure campaign success

Business Analyst Data Scientist

Manager

5-10% increase in wealth account conversion rate

Male 28-42; median deposit; $ transfers; then extend offer

Refine/Extend

Extend selective offers to open new wealth account based on real-time predictions of propensity to accept offer at time of events e.g. transfer/deposit

Analyst

Event Cloud

Teradata Oracle BPM

Tibbr

Analytics-to-Action: Cross-Sell / Up-Sell

TERR


Customer Insight: Social Media & Sentiment Analytics

• Response

• Customer sentiment / intent

• Analysis Unit

• Tweets

• Features

• Favorability

• Recommendation

• Influencers / Followers

• Models

• tm, twitteR, wordcloud

• Value

• Attribution : contribution of channel

• Optimization : scenario planning

• Allocation : media mix


Sentiment and Social Media Analytics


Influencers and Followers


Business Insights: Procter and Gamble Business Sphere

Source: McKinsey Quarterly, Nov 2011


Business, Customer and Consumer Insights

• Analyses

• Business Insights

• Customer Insights

• Consumer Insights

• Analysis Unit

• Product * Market

• Customer behavior

• Features

• Product/channel attributes

• Promotions, pricing

• Value

• Focus on product * market combinations

• Identify managers / attributes with issues


Enterprise Risk Aggregation ► Aggregated view of multiple risk positions by

geography, product, … ► Assess loss given default for products and

counterparties

#2. Risk Management

Credit and Counterparty Risk ► Handle unexpected changes in markets with

rapid rebalancing ► Incorporate predictive analytics e.g. stress

tests, economic capital ► Collaborate between traders and managers

Operational Risk ► Track desk in near real time ► Identify violations, patterns, rule-brushing

traders

Risk managers need interactive and aggregate views on risk; and ability to visualize complex risk analyses with management

Market Risk ► Mark to market, VaR, scenario and sensitivity

analysis ► Interest rate and FX-risk ► Collaborate across organization for rapid

response to market conditions


Royal Bank of Scotland


Equity Trading ► Analyze price / volume across holdings with

drill down into individual securities ► Calculate statistical and technical indicators,

benchmark comparisons ► Incorporate indicators and financial data e.g.

Bloomberg, Thomson, FactSet ► Collaborate across trading teams

#3. Trading Analytics

Fixed Income Analysis ► Identify unknowns: bond prices, interest,

durations, convexity and yield ► Compare fixed income instruments with

economic indicators ► Incorporate statistical methods applied to

new and existing instruments

Trade Fraud and Compliance ► Track desk in near real time ► Identify violations, patterns, rule-brushing

traders

Traders need rapid and comprehensive view of risk and sensitivities


Portfolio Optimization ► Analyze risk-return sensitivities ► Efficient frontier statistics ► Drill to efficient portfolios of interest,

incorporating predictive analytics ► Rebalance on –demand

#4. Portfolio Management

Performance and Attribution ► Dimension-free exploration of assets across

time ► Real-time analysis and review of fund

strategies impact on performance ► Rapid transaction reconciliation with

exceptions analysis ► Integrate with CRM data for customer insight

Managers require multiple assets and funds in single view; need to prove value and comply with new regulations

Regulation Compliance ► Comply with new regulations that require

thorough understanding of trading across assets and liabilities


Anti-Money Laundering ► Identify / trace unknown and suspect

transactions across context e.g. business lines and geographies

► Set thresholds, define segments and automate alert scoring

► Comply with OCC guidelines

#5. Fraud and Compliance

Credit Card Fraud ► Identify fraudulent transactions by vendor,

location ► Highlight patterns, context and identify root

cause

Trade Fraud and Compliance ► Track desk in near real time ► Identify violations, patterns, rule-brushing

traders

Regulators expect banks to have powerful and robust analytics for compliance and Anti Money Laundering (AML)


TERR, Open Source R


Sources: NY Times : “The Age of Big Data” (Feb 11) NY Times: “How Companies Learn Your Secrets” (Feb 16)

TIBCO Enterprise Runtime for R (TERR)

• TIBCO owns the S language and have built an R engine

• TIBCO engineers created and support S+

• TIBCO engineers have created the world’s only commercial R engine: TIBCO Enterprise Runtime for R (TERR)

• TERR built from ground up in C++

• Faster than R; better memory management

• Optimization for known deficiencies in R constructs (e.g. looping)

• Clean room: no code from R

• MKL, lapack, linpack, eispack inside

• Licensing

• IP owned, supported and maintained product

• Scalable, embeddable, redistributable

• TERR is now inside Spotfire and TIBCO Products

• Spotfire Professional

• Spotfire Statistics Services

• TIBCO Business Events, TIBCO Grid Server

Engine Pool

Service dbase

Spotfire

Statistics

Services

TERR

SAS® MATLAB®

C# Java

TERR


Speed Comparison

Model Fitting: 5 Million Rows Model Scoring: 20 Million Rows

TERR

AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP


Data Viz Challenge


People Who Write Longer Emails Earn Higher Salaries!


Sala

ry x

10

,00

0

Staff

Senior Managers


Thank You!

Eric Novik Spotfire Industry Analytics Group

Twitter: @ericnovik

[email protected] :: www.ericnovik.com

mailto:[email protected]

http://www.ericnovik.com/

TIBCO Spotfire: Data Science in the Enterprise

Business

Transcript of TIBCO Spotfire: Data Science in the Enterprise