Business Analytics and Netezza -Final - John...
Transcript of Business Analytics and Netezza -Final - John...
© 2010 IBM Corporation
Netezza and Business Analytics Synergy
Shimon Nir, IBM
Netezza Business Partner Update: November 17, 2011
© 2010 IBM Corporation2
Business Analytics software
Agenda
� Business Analytics / Netezza Synergy Overview
� Netezza overview
� Enabling the Business with Cognos 10 BI
� Cognos 10 BI / Netezza Optimization
© 2010 IBM Corporation3
Business Analytics software
Analyze
Integrate
Transactional &
Collaborative
Applications
Manage
Business Analytics
Applications
External
Information
Sources
Cubes
Streams
Big Data
Master Data
Content
Data
Streaming
Information Govern
QualitySecurity & PrivacyLifecycle
Data Warehouses
Standards
Foundation for Business Analytics and Optimization
© 2010 IBM Corporation4
Business Analytics software
IBM Business Analytics Portfolio
© 2010 IBM Corporation5
Business Analytics software
Netezza and IBM Business Analytics
� Breadth of Analytical Capabilities
� Revolutionary Way to work with BI
� Cost Effective Scale
IBM Business Analytics
Driving better business outcomes
Optimized foundation for analytics
� Speed
� Scalability
� Simplicity
Netezza Solutions
Integrated & Optimized
© 2010 IBM Corporation6
Business Analytics software
• Use raw, transaction level data
• Infuse data with outside data
• Exploit more predictor variables
• Go from reporting to action
• Take action with on-demand
analytics
• Deploy analytics into production
• Move from batch to near-real time
• Shift from months to minutes
123
Predict with more
accuracy
Deliver predictions
faster
Respond rapidly to
changes
New business value with in-database analytics
Netezza customers use
in-database analytics to:
© 2010 IBM Corporation7
Business Analytics software
IBM Netezza In-Database Analytics Version 2.0
� No data movement
� Analyze deep and wide data
� High performance, parallel
computation
Netezza
In-Database
Analytics
Netezza
In-Database
Analytics
TransformationsTransformations
MathematicalMathematical
GeospatialGeospatial
PredictivePredictive
StatisticsStatistics
Time SeriesTime Series
Data MiningData Mining
© 2010 IBM Corporation8
Business Analytics software
Agenda
� Business Analytics / Netezza Synergy Overview
� NETEZZA overview
� Enabling the Business with Cognos 10 BI
� Cognos 10 BI / Netezza Optimization
© 2010 IBM Corporation9
Business Analytics software
IBM Netezza Appliance
� Purpose-built analytics engine
� Integrated database, server and storage
� Standard interfaces
� Low total cost of ownership
� Speed: 10-100x faster than traditional system
� Simplicity: Minimal administration and tuning
� Scalability: Peta-scale user data capacity
� Smart: High-performance advanced analytics
.
© 2010 IBM Corporation10
Business Analytics software
Simplicity
IP 1..5
• no Tablespaces
• no Partitioning
• no Indexes
• no Log-Management
• 1x DB Server Cluster (active/passive)
• Connectors:
© 2010 IBM Corporation11
Business Analytics softwareInformation Management
Inside IBM Netezza Appliance
Hardware + Netezza Database
Red Hat Linux 5 64-bit
11
Motor MPP – Data streaming
IBM HS22 S-BladeIntel Xeon Processor
x2 Quad core - 2.4 GHz Westmere24 GB Memory
Host SMP –Administration
IBM x3650 M2 x2 Quad-Core - Intel 2.6 GHz CPUs
16 GB Memory
Storage
IBM 1TB Disk Drive 7.2K SAS
© 2010 IBM Corporation12
Business Analytics software
The Netezza TwinFin™ Appliance
12
High-performance database
engine streaming joins,
aggregations, sorts, etc.
SQL Compiler
Query Plan
Optimize
Admin
Processor &
streaming DB logic
Slice of User Data
Swap and Mirror partitions
High speed data streaming
SMP Hosts
S- Blades™
(with FPGA-based
Database Accelerator )
Disk Enclosures
© 2010 IBM Corporation13
Business Analytics software
The Netezza S-Blade™
13
IBM BladeCenter Server Netezza DB Accelerator
Intel Quad-Core
2+ GHz CPUDual-Core FPGA
125 MHz
24 GB DRAM SAS Expander
Module
© 2010 IBM Corporation14
Business Analytics software
Asymmetric Massively Parallel Processing™
Massively Parallel
Intelligent Storage
1
2
3
920
�
�
�
Network
FabricSMP Host
Front End
Netezza TwinFin Appliance
High-Speed
Loader/Unloader
ODBC 3.X
JDBC Type 4
OLE-DB
SQL/92
Execution
Engine
SQL
Compiler
Query
Plan
Optimize
Admin
Source
Systems
Client
High
Performance
Loader
3rd Party
Apps
DBA CLI
ETL Server
SOLARIS
LINUX
HP-UX
AIX
WINDOWS
TRU64
High-Performance
Database Engine
Streaming joins,
aggregations, sorts
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
© 2010 IBM Corporation15
Business Analytics software
High-Performance
Database Engine
Streaming joins,
aggregations, sorts
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
Execution
Engine
Asymmetric Massively Parallel Processing™
Massively Parallel
Intelligent Storage
1
2
3
920
�
�
�
Network
FabricSMP Host
Front End
Netezza TwinFin Appliance
High-Speed
Loader/Unloader
SQL
Compiler
Query
Plan
Optimize
Admin
SQL
1 2 3
1 2 3
1 2 3
1 2 3
Snippets
1 2 31 2 3
SQL
Source
Systems
Client
High
Performance
Loader
3rd Party
Apps
DBA CLI
ETL Server
SOLARIS
LINUX
HP-UX
AIX
WINDOWS
TRU64
© 2010 IBM Corporation16
Business Analytics software
Data Streaming
FPGA Core CPU Core
Uncompress ProjectionRestriction
VisibilityComplex ∑
Joins, Aggs, etc.
select DISTRICT,
PRODUCTGRP,
sum(NRX)
from MTHLY_RX_TERR_DATA
where MONTH = '20091201'
and MARKET = 509123
and SPECIALTY = 'GASTRO'
Portion of the tabla (ZoneMaps)
MTHLY_RX_TERR_DATA
(compressed)
where MONTH = '20091201'
and MARKET = 509123
and SPECIALTY = 'GASTRO'
sum(NRX)
select DISTRICT,
PRODUCTGRP,
sum(NRX)
Netezza DB
AcceleratorIBM Blade
Server
© 2010 IBM Corporation17
Business Analytics software
High-Performance
Database Engine
Streaming joins,
aggregations, sorts, etc.
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
S-Blade
Processor &
streaming DB logic
Asymmetric Massively Parallel Processing™
Massively Parallel
Intelligent Storage
1
2
3
920
�
�
�
Network
FabricSMP Host
Front End
Netezza TwinFin Appliance
High-Speed
Loader/Unloader
SQL
Compiler
Query
Plan
Optimize
Admin
1 2 3
1 2 3
1 2 3
1 2 3
Consolidate
Execution
Engine
ODBC 3.X
JDBC Type 4
OLE-DB
SQL/92
Source
Systems
Client
High
Performance
Loader
3rd Party
Apps
DBA CLI
ETL Server
SOLARIS
LINUX
HP-UX
AIX
WINDOWS
TRU64
© 2010 IBM Corporation18
Business Analytics software
Agenda
� Business Analytics / Netezza Synergy Overview
� Netezza overview
� Enabling the Business with Cognos 10 BI
� Cognos 10 BI / Netezza Optimization
© 2010 IBM Corporation19
Business Analytics software
• Full breadth of BI for every decision-maker• Query, reporting, analysis, scorecards and dashboards
• Delivered where, when and how needed
• Revolutionary way to work with BI• Unified workspace with built-in collaboration
• View of all time horizons: past, present & future
• Easy progression from viewing to exploration to more advanced
analysis
• Free to answer critical business questions• Answer “how am I doing”, “why”, and “what should I be doing?”
• Ensure organization-wide insight and alignment
© 2010 IBM Corporation20
Business Analytics software
Cognos BI Business Insight – Unified Workspace
Intuitive and easy to use�
�from simple viewing to advanced analytics
�faster and higher end user adoption rates
Highlights
� Limitless and
interactive
� All time horizons
� Drag and drop
assembly
� Seamless graduated
experience
Revolutionary Self-Service Business Intelligence
© 2010 IBM Corporation21
Business Analytics software
Cognos BI Business Insight – Unified Workspace
Cognos Real Time
•Call Center Data
Cognos BI
•Historical Trends
SPSS Predictive Analytics
•Churn Prediction
•Customer Segmentation
•Risk Analysis
Cognos Analytical Apps
•Workforce Management
© 2010 IBM Corporation22
Business Analytics software
Cognos BI Business Insight Advanced
Highlights
� Easily Author BI
Content
� Drag and drop
assembly
� Interactive Data
Analysis
Author Business Intelligence Content for Business users
© 2010 IBM Corporation23
Business Analytics software
Health Care Cost Spike – Cognos BI / SPSS & Netezza
Find similar Physicians
that have occurred cost
spikes.
Analyze details
Modeler Server
Modeler Server
NetezzaNetezza
Cognos 10 BICognos 10 BI
© 2010 IBM Corporation24
Business Analytics software
Agenda
� Business Analytics / Netezza Synergy Overview
� Netezza overview
� Enabling the Business with Cognos 10 BI
� Cognos 10 BI / Netezza Optimization
© 2010 IBM Corporation25
Business Analytics software
Cognos BI 10.1.1 is Optimized for Netezza
� Optimized SQL for Netezza
� 64-bit in-Memory Dynamic Cache
� Optimized for aggregation
� Dedicated Cognos Engineering resources
Fastest Query Response
Transformation
engineTransformation Libs
MDX
Execution
engineSQL HResult
ReposPlan Tree
Run Tree
Cache
Cache
Logs
Transformation
engine
Transformation
engineTransformation Libs
MDXMDX
Execution
engineSQLSQL HResultHResult
ReposPlan Tree
Run Tree
CacheCache
CacheCache
Logs
Cognos BI Dynamic Query Mode
4X Faster
Linear Data
Scale
© 2010 IBM Corporation26
Business Analytics software
Cognos BI V10.1.1: Dynamic Query Cache
Drill on Men6 SQL Queries: Total 7 sec
3 SQL Queries: Total 3 sec
3 Cache Hits
Dynamic Query Cache
64-bit Memory Cache
In-Memory Calculations
© 2010 IBM Corporation27
Business Analytics software
Cognos BI 10.1.1 - Faster for Analysis
* Based on findings in the lab using Cognos Business Intelligence V10.1.1 with Dynamic Query Mode compared
to previous versions of Cognos Business Intelligence. TPC-DS sample data was used.
• Cognos BI V10.1.1 scales significantly better with larger
data in an analysis scenario
• Cognos BI V10.1.1 reduces the SQL load on the Netezza
server due to caching
• We optimized the SQL as part of DQM so that the
SQL in Netezza runs by a factor of 5 faster.
• Example : against a 2.8B row fact table we were
able to execute the report in 14s and 8s of that
was SQL compared to CQM where it took 50 sec
and 42 sec for the SQL10:09:32 PM
© 2010 IBM Corporation28
Business Analytics software
Faster Performance under Concurrency
# Q
ue
ries
• Under concurrency Cognos BI V10.1.1 reduces the number of queries
going back to the database
• Cognos BI V10.1.1 reduces the load on the database so that query times
are consistent with increased user concurrency
• We optimized the connections with DQM which means that we can
increase concurrency on Netezza
© 2010 IBM Corporation29
Business Analytics software
An end to end Business Analytics solution drives
lightning fast analytics
Shashi Vangala, Manager & Chief Solutions Architect of Business Intelligence
“O“OPerformance is the mantra for Performance is the mantra for usus……embed larger extents of embed larger extents of
datadata……enabled extended visibility enabled extended visibility into the trendsinto the trends…”…”
© 2010 IBM Corporation30
Business Analytics software
30