Simplifying Big Data Analytics for the Business

Bringing Data to the Business in a BIG Way

Tasso Argyros, Co-President and Founder Teradata Aster

Confidential and proprietary. Copyright © 2012 Teradata Corporation. 2

“Enterprises often address new information management challenges with one-off solutions, and the big-data challenge could unfortunately follow the same pattern.”

Gartner, April 2011

Is Big Data for Business?

“By 2017 the CMO will be spending more on IT than the CIO.”

Gartner, Jan 2012

“Barnes & Noble has made a profound transformation from being a physical seller of books to a digital technology company. A key component of that is the ability we have gained to leverage 'big data' to derive consumer insights that are deployed multi-channel for better personalization.”

Marc Parrish, VP, Customer Retention, Barnes & Noble

Presenter

Presentation Notes

There is huge potential and huge challenges in big data. But aster’s customers have found a way to get the value w/o the challenge


What is New About Big Data?

Source: CEO Advisory: ‘Big Data’ Equals Big Opportunity, Gartner, 31 March 2011.

Size Data Analytics Big

Data

Presenter

Presentation Notes

Explain that Big Data is More data (But that’s not new) Diverse data More (Beyond SQL) analytics


What Else is New About Big Data?

“Capture only what’s needed”

IT delivers a platform for storing, refining, and

analyzing all data sources Business explores data for questions worth answering

Big Data Analytics Multi-structured & Iterative Analysis

IT structures the data to answer those questions

Business determines what questions to ask

Classic BI Structured & Repeatable Analysis

“Capture only what’s needed”


Cost and Complexity Kill Enterprise Adoption

Challenges with Emerging Big Data Technologies

Cost Complexity

Enterprise Adoption

“Sophisticated tools for data integration and analysis on this scale are largely lacking today. There are opportunities to create tools and applications for Big Data.”

- IDC Market Analysis, Worldwide Big Data Technology and Services, 20102-2012 Forecast

Presenter

Presentation Notes

A lot of new technologies have complexity and cost. This requires small teams of specialized people which kills adoption and the business value that’s extracted


Is Big Data Solving Your Problems Today? 3 Key Questions to Ask your CIO

How many people in your organization can directly ask Big Data questions? 1

How much time does it take to answer a new business question with Big Data? 2

Is it hard to find the right people and budget to tackle new Big Data problems? 3

Need right technologies to realize business value of Big Data


Making Big Analytics Work: The Discovery Cycle Most Effective Way to Create Business Insight from Big Data

Analytical Idea

Evaluate Results

Zero-ETL Data Load/Integration

SQL & non-SQL Analysis

Operational DB

or EDW Operationalize

or Move On

Presenter

Presentation Notes

Explain how a data scientist goes through this process. Question: where does the business analyst fit here?


Key Requirements of a Discovery Platform

Highly Efficient & Performant Big Data Platform That Allows Quick Iterations 1

Hybrid Capabilities that Provide both Legacy (SQL, BI) and New (MapReduce) Interfaces 2

Significant Out-of-the-Box Analytical Apps that Minimize Development 3

Democratize Big Data & Maximize Enterprise Adoption


Your Analytic & Advanced Reporting Applications

Store

Process

Rapid Analytics Development

Embedded Analytic Processing

Massively Parallel Data Storage

• Commodity-hardware based • Software-only; Appliance; Cloud • Fault-tolerant & one-click expansion

• SQL-MapReduce framework • Analyze both non-relational +

relational data • Top performance for both SQL & MR

• 50+ pre-built analytical apps • Visual IDE: custom apps in hours • Several programming languages

Analysts Data Scientists Business Users Customers

Develop

Aster’s Discovery Platform Democratizes Big Data


SQL & MapReduce: the Gap

But what if you need both?

• Easy and Fast • Integrates well

with BI/Viz. tools

• But…not always powerful enough

SQL • Powerful, but… • Batch-oriented • Requires lots of

coding

MapReduce


Filling the Gap: SQL-MapReduce®

Industry’s Only Standard SQL + MR Combination

Standard SQL Support for Business Analysts Integrates Seamlessly with Most BI Tools 10x MapReduce Performance Advantage

1

2

3

Presenter

Presentation Notes

Explain advantages of SQL-MR


Some of the 50 out-of-the-box analytical apps

Analytic Foundation: the App Store of Big Data

Path Analysis Discover patterns in rows of sequential data

Text Analysis Derive patterns and extract features in textual data

Statistical Analysis High-performance processing of common statistical calculations

Segmentation Discover natural groupings of data points

Marketing Analytics Analyze customer interactions to optimize marketing decisions

Data Transformation Transform data for more advanced analysis


How SQL-MapReduce® Connects Business Analysts To MapReduce Processing

Invoke Pre-Bulid SQL-MapReduce® App Through SQL and Visualize Directly in Tableau®

Presenter

Presentation Notes

Tableau running directly mapreduce is revolutionary. Also, you don’t need to write java code if you are using our analytical foundation.


Beyond the Discovery Platform: Unified Big Data Architecture Enabling All Users for Any Data Type from Data Capture to Analysis

Discover and Explore Reporting and Execution in the Enterprise

Capture, Store and Refine

Audio/ Video Images Docs Text Web &

Social Machine

Logs CRM SCM ERP

Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualization, etc.


ANALYTICS

Unified Big Data Architecture for the Enterprise

Discovery Platform Active Data Warehouse

Audio/ Video Images Text Web &

Social Machine

Logs CRM SCM ERP

Engineers Business Analysts Quants Data Scientists

Java, C/C++, Pig, Python, R, SAS, SQL, Excel, BI, Visualization, etc.

Capture, Store, Refine

Presenter

Presentation Notes

This is the base case which is “best of breed” and recommended for enterprises. Depending on customer or use case, there will be iterations of this.


Aster SQL-H™ A Business User’s Bridge to Analyze Hadoop Data

Aster SQL-H gives analysts and data scientists a better way to analyze data stored cheaply in Hadoop

• Allow standard ANSI SQL to Hadoop data

• Leverage existing BI tool investments

• Enable 50+ prebuilt SQL-MapReduce Apps and IDE

• Lower costs by making data analysts self-sufficient

Announced June 12th, 2012


Cross-Channel Customer Interactions 17,000 Customers, 1 Month

Challenge • Know the “last mile” of a decision • Data Mining tools predict probability

but do not ID the “last mile”

With Aster • SQL-MapReduce listens and predicts

the “last mile” - Identifies all interaction patterns

prior to acquisition or attrition

Business Impact • 10-300x less effort to pinpoint a

customer in the “last mile”

Aster in Retail Banking: “Last Mile” Marketing

92k Online Sessions

25k ATM Sessions 34k Branch Visits

5,000 Call Center Sessions

43k E-mails

Presenter

Presentation Notes

Generalize numbers


Aster MapReduce: Understanding the “Last Mile”

What if I knew that this customer was likely to leave? I could… • Apologize • Offer an explanation • Reverse the $5 fee

“It takes 3x more to acquire a customer than to retain one”

Jan 5: Reverse Fee Request Jan 10: Request Made Again

Jan 15: Request Made Again Jan 7: Request Made Again

Jan 20: Account Closed


Multi-Channel Customer Analysis Iterative Discovery Analytics

STORE DATA

Business Question(s): • Is there any identifiable pattern of behavior prior to account closure? • Prior to new product additions? • If so, what does this pattern look like?

Presenter

Presentation Notes

Here is aPOC that we did for a financial institution and I’d like to point a couple of things out first. Many times “Big Data” is focused on unstructured or new data that isn’t being analyzed today. Here is an example of Big Data Analytics on existing structured or relational data. All of the data was pulled from existing database systems, Teradata and Oracle, and loaded directly into Aster. Some of the data was unstructured in its original form but preprocessing was done on the data and then modeled into a relational database. So Big Data Analytics or MapReduce analytics can also analyze existing data as well as new data in the enterprise. The business questions that customer wanted to solve that they can’t solve easily today in SQL is “What events led up to a product purchase?” or “What events led up to a customer defection”. So we combined all these data subject areas and interactions and used our pattern and path analytics to answer these questions.


Events Preceding Account Closure

Presenter

Presentation Notes

So the analysis is around finding the path of interactions that lead up to an event, with the event being a product purchase, a customer defection, or an appointment with a loan officer. The event analyzed in this example was an “Account Closure” and displayed using visualization tool like Tableau reading our Aster database tables. What you see are the list of interactions on the left that include branch activity, call center and web activity. I won’t go into detail but these interactions can be across multiple channels. On the bottom going right to left are the number of interactions that lead up to the event of “Account Closure”. One of the challenges of Big Data is that it does generate a lot of data and often it is hard to see the signal that lead to the event in all the noise of the data. So this is why Big Data Analytics is a very iterative process with the Data Scientist applying part science and part art while understanding their companies data and business problems.



SELECT * FROM npath ( ON ( SELECT … WHERE u.event_description IN ( SELECT aper.event FROM attrition_paths_event_rank aper ORDER BY aper.count DESC LIMIT 10) ) … PATTERN ('(OTHER|EVENT){1,20}$') SYMBOLS (…) RESULT (…) ) ) n;

Interactive Analytics

Reducing the “Noise” to find the “Signal”

Presenter

Presentation Notes

As mentioned earlier, the value of SQL-MapReduce is the ease by which a data scientist can easily manipulate the SQL-MapReduce to start to reduce the “noise” to find the “signal” or a clear patter than leads up to an event. So by simply changing the parameters of the SQL-MapReduce nPath operator that you see there on the left, the business analyst can start to investigate the data to get a clearer picture of what interactions typically happen that leads up to an Account Closure, again reducing the “noise” to find the “signal”. (like on a radio, fine tuning the noise to get to a clear channel)



SELECT * FROM nPath ( ON (…) PARTITION BY sba_id ORDER BY datestamp MODE (NONOVERLAPPING) PATTERN ('(OTHER_EVENT|FEE_EVENT)+') SYMBOLS ( event LIKE '%REVERSE FEE%' AS FEE_EVENT, event NOT LIKE '%REVERSE FEE%' AS OTHER_EVENT) RESULT (…) ) n;

Closed Accounts

Fee reversal seems to be a “Signal”

Presenter

Presentation Notes

Again, here we see that the data analyst iterated again and wanted to focus on “fee events” or customers who had asked for a fee reversal and got denied. This is a small sample set but it looks like 3 of 128 had a request for a fee reversal prior to closing their account at the bank. So in general the idea is that customers who request a fee reversal on the web or call center for instance, not a face to face interaction, and then come into the branch, a face to face interaction, and are denied they typically close their account. Makes sense in that they’ve already asked for a fee reversal, been denied, but want to give the bank one more chance. When denied this time they close their account. So how would you operationalize this? Is 3 account closures out of say 300 fee reversal requests something that you should act upon? Probably not. Banks charge fees to generate revenue and if they start reversing fees for everyone that asked they might was well not charge a fee in the first place. 297 out of 300 asked, got rejected and didn’t close their account. What the heck, at least try. However, now that I know this pattern of a fee reversal request at one or two alternate channels within 72 hours of asking at the branch leads to a high likelihood of account closure is there anything that I can do? We can either run daily processing in Aster to identify all customers who requested a fee reversal at the Call Center or Web or potentially in Teradata. We can then update their profile so that when the walk up to the teller, the teller terminal has the customer profile along with a note on their fee reversal requests in the past 72 hours. The teller can take at look at their profitability score and decide to honor the request from profitable customers and not honor the request from non-profitable and risk losing the customer to a competitor, which actually isn’t a bad strategy of pushing your non-profitable customers to competition. So this is a good example of the need to have a very flexible and iterative discovery platform to find the signal from all the noise in Big Data.


Collect & analyze not only customer transactions, but also customer interactions

Use SQL-MapReduce pre-built operators to identify behavioral patterns and uncover business insight

Utilize standard BI tools to ensure insights are consumed by the right business analysts and acted upon

Big Data for Business = use more data and more analytics to achieve a competitive edge

How did Big Data Help?

1

2

3


3

2

Summary

1 Huge Business Value in Big Data

Paradigms Like Discovery Platform Reduce Cost, Time-to-Market and Maximize Adoption

Teradata Aster Enables Business Users to Access Directly Big Data Technologies

Simplifying Big Data Analytics for the Business

Technology

Transcript of Simplifying Big Data Analytics for the Business