© 2013 IBM Corporation1
Fred Ho | Informix Development
Sept 4, 2013
IBM Informix Warehouse Accelerator (IWA)
© 2013 IBM Corporation
Informix 12.1
Agenda
• Data Warehouse Trends
• IWA Technology Overview
• IWA Customers and Partners
• IWA Reference Architecture and Competition
• IWA Roadmap and 12.10 Features
© 2013 IBM Corporation
Informix 12.1
TRENDSDatabase and Data Warehousing Industry
3
© 2013 IBM Corporation
Informix 12.1
Data Warehousing Workload & Optimizations
� Data Warehousing/OLAP workload
are inherently more complex than
OLTP transactions and reasons are well-documented.
� Ways to overcome that include:– Building Indexes– Partitioning of data– Building cubes, MOLAP / ROLAP / HOLAP– Query tuning– Appliances that add a new layer of hardware
to perform I/O for DBMS
� Mixed-Workload always a challenge
� DBMS needs to be built to handle such a workload
4
© 2013 IBM Corporation
Informix 12.1
Third Generation of Database Technology
According to IDC’s Article (Carl Olofson) – Feb. 2010
� 1st Generation:
– Vendor proprietary databases of IMS, IDMS, Datacom
� 2nd Generation:– RDBMS for Open Systems– Dependent on disk layout, limitations in scalability and disk I/O
� 3rd Generation: IDC Predicts that within 5 years:– Most data warehouses will be stored in a columnar fashion– Most OLTP database will either be augmented by an in-memory database
(IMDB) or reside entirely in memory
– Most large-scale database servers will achieve horizontal scalability through
clustering
5
© 2013 IBM Corporation
Informix 12.1
Data Warehouse Trends for the CIO, 2011-2012
� Data Warehouse Appliances:– DW appliances are not a new concept.
– Most vendors have developed an appliance offering or promote certified configurations.
– Main reason for consideration is simplicity.
� The Resurgence of Data Marts:– Data marts can be used to optimize DW by offloading part of the workload, returning
greater performance to the warehousing environment.
� Column-Store DBMSs:– CIOs should be aware that their current DBMS vendor may offer a column-store solution.
– Don’t just buy a column-store-only DBMS because a column store was recommended by
your team.
� In-Memory DBMSs:– IMDBMS technology also introduces a higher probability that analytics and transactional
systems can share the same database.
Source:TheState of DataWarehousing in 2011; 1/31/2011; GartnerMarkBeyer,Roxane Edjlali,Donald Feinberg(IDNumber: G00209643)
6
© 2013 IBM Corporation
Informix 12.1
In-Memory DBMSs (IMDBMS) Have Plenty of Promises
� Promise that they will transform the way data is provisioned and consumed
� The extreme performance enable true real-time decision making on any shape
and volume of data
� Powers unique business innovation and competitive differentiation in today’s
high-speed business environment
� Gartner estimates that IMDBMS will replace 25% of traditional data warehouse
and OLTP systems by 2016
� Examples are: Real-time pricing for airlines or banks evaluating assets for rebalancing of portfolio
� Lower TCO by reducing the need for separate OLTP and data warehouse copiesof data and by eliminating need for cubes, aggregates and indexes.
Gartner ID Number: G002215735, Publication Date: 8 Sept 2011, Authors: Roxane Edjlali, Donald Feinberg
7
© 2013 IBM Corporation
Informix 12.1
IDC WW RDBMS 2012-2016 Forecast
� “ As a result of the near-ubiquitous adoption of 64-bit processors, precipitous declines in the price of main memory, and the need for greater transaction throughput, database technology is migrating from a disk-based paradigm to a memory-based one.
� Data is increasingly “stored” in memory, protected by redundant replication and asynchronous logging (for recovery), and organized for highly efficient retrieval and update.”
8
© 2013 IBM Corporation
Informix 12.1
Does the 21st-Century "Big Data" Warehouse Meanthe End of the Enterprise Data Warehouse?
Key Findings:
� Organizations that deploy an EDW almost all create second and third data warehouses or marts to support additional user needs (judging from up to 90% of the data warehouse inquiries received from Gartner clients), despite strict instructions to use the EDW.
� The architectural style of a data warehouse is usually determined by the available skills and tools, and secondarily by time-to-delivery.
Source: Gartner; Mark Beyer, Donald Feinberg (ID Number G00213081
9
© 2013 IBM Corporation
Informix 12.1
Market Research on IWA: What do the Analysts Say?
� White Paper by Bloor Research: “IBM Informix in Hybrid
Workload Environments”, August 2012“…for hybrid environments, Informix has a number of unique capabilities that
cannot be matched by either conventional data warehouse vendors or traditional data warehouses…”
� White Paper by Ovum Research: “Informix Accelerates
Analytic Integration into OLTP”, July 2012“…Supporting both operational and analytic workloads with the same system is
a relatively unique idea that is also being pursued by rivals such as Oracle. Ovum believes that IBM now has a strong story to tell with IWA”.
� Magic Quadrant for Data Warehouse Database Management
Systems by Gartner, Jan 2013“IBM has a vast number of products that use in-memory computing, including
solidDB, but its only in-memory solution for the data warehousing and analytical market is Informix Warehouse Edition.”
10
To download these and other papers, go to: http://www-01.ibm.com/software/data/informix/warehouse/
© 2013 IBM Corporation
Informix 12.1
IWA TECHNOLOGY OVERVIEWIBM Informix Warehouse Accelerator (IWA)
11
© 2013 IBM Corporation
Informix 12.1
TC
P/IP
Query
Processor
CompressedDB partition
InformixQuery Router
SQL
Results
(via DRDA)
IDS Database
Informix:
Routes SQL queries to the Accelerator
User need not change SQL or applications
Can always run query in Informix if not accelerated
Bulk Loader
Informix Warehouse Accelerator:
Connects to Informix via TCP/IP
Analizes, compresses and loads In-Memory…
…a copy of (portion of) Informix warehouse
Proceses routed SQL queries with extraorinary speed
Returns results/answer back to Informix/IDS
SQL Queries (from apps)
Informix Warehouse Accelerator
Informix Warehouse Accelerator (IWA):Overview and Seamlessly Integration with Informix/IDS
Informix Warehouse Accelerator (IWA) transparentlyaccelerates Informix warehouse/analytic queries up to
100 times or more!
Linux x86_6464-bit
© 2013 IBM Corporation
Informix 12.1
Intelligent Frequence Paritioning
64 bit Intel/AMD
Prcessors
TB of RAM Memoria
Predicates Evaluationon Compressed Data
CommonValues
RareValues
Nu
mb
er
of
occu
rren
ces
SIMD
No Need for Aggregate Tables
Row and Column Store
Compresion
IWA Technology Innovations provide:Extreme unparallel analytics speed for fast business decisions
© 2013 IBM Corporation
Informix 12.1
IWA: Breakthrough Technologies for Extreme Performance
14
1
2
34
5
6
7 1
2
34
5
6
7
Row & Columnar DatabaseRow format within IDS for transactional workloads
and columnar data access via accelerator for OLAP queries.
Extreme CompressionRequired because RAM is the limiting factor.
Massive ParallelismAll cores are used within used for queries
Predicate evaluation on compressed data
Often scans w/o decompression during evaluation
Frequency PartitioningEnabler for the effective parallel access of
the compressed data for scanning. Horizontal and Vertical Partition
Elimination.
In Memory Database3rd generation database technology avoids I/O. Compression allows huge databases
to be completely memory resident
Multi-core and Vector Optimized Algorithms
Avoiding locking or synchronization
© 2013 IBM Corporation
Informix 12.1
How Fast is IWA?Columnar In-Memory Analytics with Unprecedented Performance
© 2013 IBM Corporation15
© 2013 IBM Corporation
Informix 12.1
IBM Informix 12.10 editions: New Value-Added Software Bundles
16
© 2013 IBM Corporation
Informix 12.1
CUSTOMERS AND PARTNERSInformix Warehouse Accelerator (IWA)
17
© 2013 IBM Corporation
Informix 12.1
Some IWA Customers by Sector:Retail, Government, Transportation, E&U
18
© 2013 IBM Corporation
Informix 12.1
Some IWA Customers by Sector:Telecommunications, Insurance, Financial, IT Services
19
© 2013 IBM Corporation
Informix 12.1
Real-Life Productive IWAin Government Agency in LATAM
20
© 2013 IBM Corporation
Informix 12.1
IWA Architecture and Use (same Government Agency in LATAM)
21
Real-time Analytics
IWA
37x to 456x faster !!
IDS 10.00 -> 11.70.FCxAIX on pSeries
Cubes built in
Microsoft (MSAS)IWA 11.70.FCxLinux x86_64
BI Tools
ETL: Consolidate and aggregate data in IDS to later source and build MSAS cubes
ETL
Sources/OLTP
© 2013 IBM Corporation
Informix 12.1
Europe’s Largest Power company tackles the Smart Meter Big Data challenge with Informix TimeSeries + In-Memory Accelerator (IWA)
� E.ON Metering (EMTG) is the centre of excellence for the development and
commercialization of smart energy solutions and technologies and part of Europe’s largest
Power and Gas company E.ON
� EMTG operates a sophisticated Smart Meter data infrastructure based on IBM Informix
TimeSeries technology in combination with Informix In-Memory Warehouse Accelerator
� IBM Information Management products currently used:
– Informix 11.70 Ultimate Warehouse Edition
– Cognos Business Intelligence 10
22
© 2013 IBM Corporation
Informix 12.1
Some of our IWA Business Partners
23
© 2013 IBM Corporation
Informix 12.1
REFERENCE ARCHITECTURE & COMPETITION
Informix Warehouse Accelerator (IWA)
24
© 2013 IBM Corporation
Informix 12.1
25
Current Customer BI Architecture with Informix
Source If Applicable
IBM Presentation Template Full Version
ETL
Sun T3
Solaris
IUE v11.50
Intel serversMS Windows
MS SQLserver
Cubes
BIETL
IUE : Informix Ultimate Edition
Informix
DW
Data Mart 1
Data Mart 3
Data Mart 2
Prod A
Prod E
Prod D
Prod C
Prod B
© 2013 IBM Corporation
Informix 12.1
26
Target Referenced Architecture with IWA
Source If Applicable
IBM Presentation Template Full Version
ETL
Sun T3Solaris
IUWE v11.70
BI
IUWE : Informix Ultimate Warehouse Edition
Informix
DW
Prod A
Prod E
Prod D
Prod C
Prod B
Warehouse
Accelerator
© 2013 IBM Corporation
Informix 12.1
IWA’s Industry Positioning and Competitors
27
DW Appliance
DataAllegro (Microsoft)
Dataupia
Greenplum (EMC)
Kognito
Netezza (IBM)
Columnar Database
Calpont
Exasol
Infobright
ParAccel
Sand Technology
Vertica (HP)
Sybase IQ (SAP)
In-Memory OLAP Tools
QlikTech/QlikView
Applix TM-1 (IBM-Cognos)
Exalytics (Oracle)
PALO
In-Memory DataWarehouse
HANA (SAP)
IWA (IBM)
© 2013 IBM Corporation
Informix 12.1
ROADMAP & NEW FEATURESInformix Warehouse Accelerator (IWA)
28
© 2013 IBM Corporation
Informix 12.1
Informix 11.70.xC2
� IWA 1st Release
� On SMP
Informix 11.70.xC3
� Workload Analysis Tool
� More Locales
� Data Currency
Informix 11.70.xC4
� IGWE
� IWA on Blade Server
IBM Informix Warehouse/Analytics Present Roadmap
20112011 20122012
Informix 11.70.xC5
� Partition Refresh
� Load from Secondary
� Solaris on Intel
Informix 11.70.xC6
� Partition Refresh
� Load from Secondary
� Solaris on Intel
Informix 11.70.xC7
� Partition Refresh
� Load from Secondary node in Cluster
� Solaris on Intel
Informix 12.10.xC1
� Bundled Cognos BI & SPSS
� Automatic incremental refresh
� Trickle feed (continuous refresh)
� Accelerate new SQL & OLAP queries
� Admin IWA using OAT & built-in functions
� Right/Real-Time In-Memory Analytics
� Big Data on Sensor data (TimeSeries+IWA)
� Informix 12.10.xC2
� Coming soon…
20132013
© 2013 IBM Corporation
Informix 12.1
Summary: Key Value Propositions for the Informix Market
• State-of-the-art query accelerator for current OLTP customers
• Ideal for an embedded database in a single machine environment
•Leverage partner-based solutions & sales
• Fully compatible with existing Informix architecture
•Deploy in HA configurations
•Integration with OAT
• Ideal for SMB
•Commodity based hardware, configurable memory size, no expensive interconnect required
•Low cost entry, scaling via cluster
•Appliance not always a fit
• Seamless integration with TimeSeries for Big Machine data
© 2013 IBM Corporation
Informix 12.1
Informix 12.10: Simply Powerful
28
© 2013 IBM Corporation
Informix 12.1
SAPs
0
10000
20000
30000
40000
50000
60000
70000
80000
3850
(CRANFORD,SC)
3850 (Paxville, DC) 3850 (TULSA DC) 3850 (TIGERTON
QC)
3850 DUNNINGTON
(HC)
3850 x5 (NEHALEM-
EX, 8C)
SAPs
SAPs 7.0 UNICODE
SAP ECC 6.0 Basic
IWA HW & SW Breakthrough Technology Innovations
� Row and Column storage
� Compression & queryprocessing on compressed data
� Intelligent Partitioning
� No Aggregate Tables, No Indexes
� Insert Only on Delta
� Multi-Core Parallelism & Intel 64 SIMD tech.
� Massive Parallel Scaling for Loads & Queries
� In-Memory storage & query
� Extreme performance (10x up to 200x faster complex queries) using Low cost commodity HW, transparent integration with Informix ORDBMS
� Fast Storage Backup for Recoverability
In 12.10, we made IWA even more accessible by providing:
• Automatic incremental (partition-level) refresh and trickle feed (continuous refresh)
•Support for smart sensors/meters data (Time Series) –Big Data (on sensors’ data)
•Additional SQL capabilities for common OLAP queries
• Integrated administration via Open Admin Tool (OAT) and SQL API functions
IWA 12.10: Highlights of New Features and Benefits
© 2013 IBM Corporation
Informix 12.1
IBM Informix 12.10: Performance for Real-Time Operational Analytics and Big Data on Sensor Data
� NEW! IWA Support for UNION queries and additional SQL support
� NEW! Much faster operational analytics and enhanced OLAP capabilities
– Enhanced integration with Cognos for much faster Cognos BI
– IDS and IWA support for OLAP functions and windowed aggregates
33
© 2013 IBM Corporation
Informix 12.1
IBM Informix 12.10: Performance for Real-Time Operational Analytics and Big Data on Sensor Data
� NEW! Trickle Feed (Continuous Refresh)
– Automated, continuous updates/refresh of changed data from Informix into IWA for
speed-of-thought analysis of real-time data
34
ifx_setupTrickleFeed
Tracks changes in Dimensions
Tracks inserts in Fact tables
Automated updates in IWA
datamart
© 2013 IBM Corporation
Informix 12.1
IBM Informix 12.10: Performance for Real-Time Operational Analytics and Big Data on Sensor Data
� NEW! Automatic Partition-Level Refresh
� NEW! IWA administration through OpenAdmin Tool (OAT) and SQL API functions
35
© 2013 IBM Corporation
Informix 12.1
IBM Informix 12.10: Performance for Real-Time Operational Analytics and Big Data on Sensor Data
� NEW! Informix TimeSeries + Informix Warehouse Accelerator Integration
– Provides real-time analytics of stored sensor data (Big Data for sensors/meters)
36
© 2013 IBM Corporation
Informix 12.1
Top Related