Pervasive Integrated Analytics

Pervasive Analytics Building Analytics into Everyday Business ActivitiesDavid Loshin Knowledge Integrity, Inc. June 5, 2012

Sponsor

Speakers

David LoshinPresident, Knowledge Integrity, Inc.

Amit PatelProgram Director, Data Warehouse Solutions Marketing, IBM

3

Building Analytics into Everyday Business ActivitiesDavid Loshin Knowledge Integrity, Inc. www.knowledge-integrity.com

Pervasive Analytics

2012 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350

4

Actionable Knowledge Drives Business Value

The corporate mission and strategic performance objectives reflect creation of value Performance improvement is informed though actionable knowledge How do the results of business intelligence and analytics feed business performance improvement?

Risk

Revenue

Customer Experience

Expenses

Performance5


Pervasive, Integrated Analytics

IDC:

when organizational culture, business processes, and technologies are designed and implemented with the goal of improving the strategic and operational decision-making capabilities of a wide range of internal and external stakeholders. By using BI to monitor and manage core processes, including those that extend beyond organizational boundaries to encompass customers and suppliers, mature BI deployments succeed in making BI a pervasive resource.

TDWI:

Other terms: integrated analytics, embedded analytics, operational BI, active data warehousing, real-time BI, right-time BI 2012 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 6

Democratization of Business Intelligence

Integrated Analytics empowers users at all levels of decisionmaking across the management hierarchy:Organizational strategic decisions impacting setting and achieving corporate objectives Tactical decisions impacting operations such as supplier management, logistics, inventory, customer service, marketing & sales Team-level decisions driving collaboration, efficiency Individual decisions informing the way people do Delivery their daily jobs Level of Data Aggregation UsersDetailed Operational Data Aggregated Management Data Front line employees Mid-level and senior managers Alerts, KPIs, queries, drilldown (on demand) Summary stats, alerts, queries, and scorecards

Summarized Internal & External DataStructured analytic data Aggregate values

Executive staffSpecial purpose marketing, business process analysis Individual contributors 2012 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350

DashboardsData mining, OLAP, Analytics, etc. Alerts, messaging7

The Generic Use Case for Integrated Analytics

A business process with distinct performance objectives The business process involves decision points by one or more actor Performance can be impaired by absence of information Performance can be impaired by ill-informed decisions Process can be improved with well-informed decision-making Participants do not need to be tech-savvy to be informed

Example: Order fulfillment and delivery


8

Goal: Enable Decision-Making Across the Enterprise

Decision-making spans the organization, encompassing the scope of high-level strategic decisions to immediate operational decisions


9

Integrating Analytics

Data WarehouseCustomer profiles Product data Affinity patterns

Query/Reporting Dashboards Alerts

Analytic Applications

Business Intelligence Services


10

ExamplesScenarioCustomer Service

Performance MeasuresFirst call resolution Reduced attrition Up-sells Speed of delivery Driver safety Reduced spoilage Increased sales Increased margin Reduced out-ofstocks Reduced outages Increased customer satisfaction

Decision MakerCall Center Representative

Actionable KnowledgeBlend customer profiles, probability of positive responses, product bundling, pricing sensitivity for real-time script modifications for call-center staff Blend spatial data, traffic, weather, emergency data, product data, packaging data for interactive routing instructions to drivers Blend POS sales rates, spatial data, customer profiles, calendar data, supplier performance, pricing sensitivity for real-time price adjustments and reduced overages Blend network status information, smart grid data, failure patterns, weather data, commodity data, asset failure data for monitoring and alerts for faster reaction to potential network events Blend clinical history data, patient details, diagnostic data, outcome data to guide Blend customer profiles, product data, and real-time social data feeds to identify opportunities to react to exposure to brand erosion11

Logistics

Driver

Retail

Floor Manager

Energy

Event Responders

Health care Brand Protection

Improved outcomes Reduced bad press

Physicians Marketing Manager


Expectations

Completeness of information necessary to make the right decisions Real-time integration of data from multiple sources Analytic + operational data Improved operational performance Seamless presentation suited to business operations Seamless integration with desktop tools Event-driven notification Analytics embedded within operational processes and supporting applications Multi-channel delivery 2012 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 12

Some Technical Challenges

Mixed workload business intelligence Integrating big data analytics Event stream processing Managing data access latency Variety of methods for knowledge delivery


13

Mixed Workload Business Intelligence

Different user archetypes require different service at different times

Power users with high demand for interactive dimensional analysis Data scientists developing and running big data analytics Business managers monitoring real-time dashboards and scorecards Integrated machine-to-machine clients streaming into operational applications

The Business Intelligence Platform must support predictable performance for all user types


14

Linking Big Data and the Data Warehouse

Results of algorithmic analyses from big data applications must be integrated with existing

Report Senior Manager

Data warehouse Data marts Big Data Data Warehouse Business intelligence tools Reports Dimensional analysis Packaging and assembly Web-based knowledge delivery Pervasive BI Visualization 2012 Knowledge Integrity, Inc. www.knowledge-integrity.com Etc. (301)754-6350

OLAPBusiness Analyst

Visualization

15

Event Stream ProcessingActive capturing, monitoring, correlating, calculating of data + information


16

Operational Reality: Managing Data Latency

More data can be

Accessed, Absorbed, Transformed, Manipulated, Combined, and Analyzed

Without understanding the characteristics of data access methods, volumes, and latency, your performance will be artificially throttled

from

Operational systems External sources Operational data stores Data warehouses and data marts Dynamic models

Data Warehouse


17

Knowledge Delivery: Scorecards, Dashboards, Alerts

Simplified presentation of summarized key performance metrics Customized by information consumer Continuously monitored and updated Can be pushed to a variety of channels and devices

Mash-ups incorporate additional information streams and applications to interpret and explore delivered intelligence


18

Considerations

Widespread adoption of business intelligence services coupled with lowered barrier to deployment opens new opportunities for integrating BI results Techniques for rich text analytics improves ability to incorporate social media data in a timely manner Workload demand will drive adoption of techniques for improved data access performance:

Data replication Continuous data feeds Improved compression methods Data distribution optimizations Differentiation of delivery by BI/analytics demand by platform


19

Questions and Open Discussion

www.knowledge-integrity.com

www.dataqualitybook.com

If you have questions, comments, or suggestions, please contact meDavid Loshin 301-754-6350 [email protected]

www.mdmbook.com

2012 Knowledge Integrity, Inc. 2011 www.knowledge-integrity.com (301)754-6350

20

Amit Patel Program Director, Data Warehouse Solutions Marketing [email protected]

Enabling Faster, More Accurate Business Decisions NEW! IBM InfoSphere Warehouse 10

2012 IBM Corporation

Information Management

Simplicity, Flexibility, Choice IBM Data Warehouse & Analytics SolutionsIBM Netezza IBM Smart Analytics System IBM Warehouse Software

Deep Analytics Appliances

Operational Analytics Optimized Systems

Custom Solutions

Deep Analytics

Operational AnalyticsWarehouse Accelerators

Information Management Portfolio(Information Server, MDM, Streams, etc)

The right mix of simplicity and flexibility22 2012 IBM Corporation


IBM InfoSphere Warehouse 10: New feature highlights

Adaptive Compression Multi-Temperature Data Management Faster and More Reliable Query Performance Continuous Ingest Time Travel Query Row and Column Access Control

23



Adaptive CompressionVersion 9.7

Version 10 Multiple page-level dictionaries in addition to a single table-level dictionary Compression dictionary contains locally frequent patterns, with one dictionary stored on every pageWhen a page becomes full, page-level compression is applied, immediately freeing up more storage in that page

Uses a single, static compression dictionaryCompresses data based on recurring patterns that appear in the table

A classic table reorg is necessary to improve compression ratios if a significant number of records in a table have been updated, or if a large amount of new data has been inserted

Reduced need for table reorgs

24

24



NEW! Multi-Temperature Data ManagementIncrease Ability to Meet SLAs; Postpone Hardware Upgrades Storage pools for different tiers of storage For range partitions, policy-based automated movement of data

HOTSSD RAID

WARMSAS RAID

COLDSATA RAID

DORMANTArchive(e.g. Optim Data Growth)

DB2 Workload Manager support Higher performance Improved ability to meet SLAs

Lower costs Gracefully extend lifespan of current storage

Using SSDs for indexes and logs and a SATA array for the data, we noticed fantastic improvements in I/O speeds, especially for synchronous reads. Additionally, the background movement of data between the storages groups is very fast. Thomas Kalb, CEO ITGAIN GmbH 2012 IBM Corporation


Faster and More Reliable Query Performance

InfoSphere Warehouse delivers 3x performance on BI workloads.

26

26



New! Real-Time Data WarehousingFaster Business Decisions; More Accurate Business Decisions Continuous feed of data

Parallel processing Supports multiple connections Higher performance Faster availability of data Minimal impact on query performance No downtime (even for large volumes of data) Lower costs Costs less than solutions outside databaseYou can now continuously feed data into your data warehouse at a high rate even whilst you are running queries against the tables in your data warehouse. InfoSphere Warehouse 10 represents a greatly strengthened offering for the data warehouse market. Ivo Grodtke, LIS.TEC GmbH 2012 IBM Corporation


Continuous Feeding of Data NewTraditional Data WarehousesData Sources Data Sources

InfoSphere Warehouse 10

DW

vs

DW

ETL- Extract, Transform, Load Frequency: Daily Loads, Slower Loads

ETL- Extract, Transform, LoadUp to the Second Loads, Faster Loads

28



NEW! Time Travel QueryEasily Analyze Historical Trends and Predict Future Demand Temporal logic & analysis Built deep into the database engine Valid time, transaction time, AS OF queries Higher performance Native support for fast performance Lower costs Eliminate need to maintain and update custom temporal implementations Easy to administer (simply turn on for any table)

The use of standardized SQL syntax for temporal operations and the integration deep into the database engine, make DB2 a leader in second generation bitemporal data management - Bitemp 2.0! Craig Baumunk, Principal at BitemporalData.comTime Travel Query is a big leap in helping customers easily implement time-aware applications in cost effective way. Shanmukhaiah D, Cognizant Technology Solutions 2012 IBM Corporation


NEW! Row and Column Access ControlEasy Compliance with Privacy and Sensitive Data Requirements Fine-grained access control Hide rows from unauthorized users Mask the value of columns for unauthorized users

Policy-driven security, with flexible policies Does not require classificationAccount Name Ana Bob Celia Dinesh Income 22,000 71,000 123,000 172,000 Branch A B B C

Teller Amy sees

1111-2222-3333-4444 2222-3333-4444-5555 3333-4444-5555-6666 4444-5555-6666-7777

Telemarketer Pat sees

Account 2222-3333-4444-5555 3333-4444-5555-6666

Name Bob Celia

Income 71,000 123,000

Branch B B

Account xxxx-xxxx-xxxx-4444 xxxx-xxxx-xxxx-5555 xxxx-xxxx-xxxx-6666 xxxx-xxxx-xxxx-7777

Name Ana Bob Celia Dinesh

Income 22,000 71,000 123,000 172,000

Branch A B B C 2012 IBM Corporation


And that is on top of what is already includedPerformanceMassively Parallel Processing Compression In-Database Mining High-Speed Query Performance Mixed Workload Mgmt Business Intelligence

Sophisticated AnalyticsOLAP

Flexibility

Autonomics High Availability Security

Native XML Support External Data Access Virtualized Deployment

Tools Business Models

31

Now pay for only what you analyze with the Terabyte Pricing Option! 2012 IBM Corporation


In Closing, to Recap.InfoSphere Warehouse 10: Real-time operational analytics empowering organizations to make active, timely and informed decisions as business events occur.

Improved Cost Efficiencies

Higher Performance

Increased Team Productivity

Adaptive Data Compression Multi-Temperature Storage

3x increase in performanceon BI workloads. Continuous Ingest of data

Built-In Time Travel Query Row and Column accesscontrols

Operational Analytics: Analytics over a large volume of data combined with high-scale operational access to the data and insights - delivering real time insights to improve each business decision.32 2012 IBM Corporation


For all things IBM Data Warehousing, just remember:

www.datawarehousing.com

Amit Patel [email protected]

33


Questions?

34

Contacting Speakers If you have further questions or comments:David Loshin [email protected] Amit Patel [email protected]

Pervasive Integrated Analytics

Documents

Transcript of Pervasive Integrated Analytics