Doag 2014 konrad-haefeli_in-memory-option-on-exadata
-
Upload
trivadis -
Category
Technology
-
view
143 -
download
3
description
Transcript of Doag 2014 konrad-haefeli_in-memory-option-on-exadata
2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
2014 © Trivadis
Exadata with In-Memory Option – the
best of all ?!?
Konrad HÄFELI
Senior Solution Manager
Infrastructure Engineering
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
1
2014 © Trivadis
Trivadis ist führend bei der IT-Beratung, der Systemintegration,
dem Solution-Engineering und der Erbringung von IT-Services
mit Fokussierung auf und Technologien
im D-A-CH-Raum.
Unsere Leistungen erbringen wir aus den strategischen Geschäftsfeldern:
Trivadis Services übernimmt den korrespondierenden Betrieb
Ihrer IT Systeme.
Unser Unternehmen
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
B E T R I E B
2
2014 © Trivadis
Mit über 600 IT- und Fachexperten bei Ihnen vor Ort
3
12 Trivadis Niederlassungen mit
über 600 Mitarbeitenden
200 Service Level Agreements
Mehr als 4'000 Trainingsteilnehmer
Forschungs- und Entwicklungs-
budget: CHF 5.0 Mio. / EUR 4.0
Mio.
Finanziell unabhängig und
nachhaltig profitabel
Erfahrung aus mehr als 1'900
Projekten pro Jahr bei über 800
Kunden
Stand 12/2013
Hamburg
Düsseldorf
Frankfurt
FreiburgMünchen
Wien
Basel
ZürichBern
Lausanne
3
Stuttgart
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
3
Brugg
2014 © Trivadis
In Memory Option on Exadata - the best of all?!?
Trivadis an der DOAG
Ebene 3 - gleich neben der Rolltreppe
Wir freuen uns auf Ihren Besuch.
Denn mit Trivadis gewinnen Sie immer.
DOAG-Konferenz 2014
4
2014 © Trivadis
A year ago DOAG 2013…
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
5
2014 © Trivadis
Chose another topic… Flash Cache
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
6
2014 © Trivadis
… finaly in July 2014
Previews…
Betas Versions
Announcments: 10th of June
Releasing 22nd of July
12.1.0.2 full release
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
7
2014 © Trivadis
It’s an option, is it worth it?
On a normal Database Server?
Hopefully, if requirements accordingly…
But what about Exadata Database Machine?
Are the «expensive» Storage Server Feature
Smart Scan
Flash Cache
a benefit for IMCS (In Memory Column Store) ?
Resp. vice versa?
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
8
2014 © Trivadis
AGENDA
1. Exadata Feature
2. In-Memory Overview
3. In-Memory Configuration
4. In-Memory Management
5. In-Memory with Exadata Tests
6. Conclusion
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
9
2014 © Trivadis
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
Exadata Feature
10
2014 © Trivadis
Architecture Features Overview
11
Source: Oracle Corporation
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
Smart Scan Capabilities
Predicate Filtering
Column Filtering
Join Processing
Scans on encrypted Data
Scans on compressed Data
Create Tablespace –
Storage Server formats
blocks
Backup and Recovery – I/O
for incremental Backup is
much more efficient, only
changed Blocks are
returned to Server
2014 © Trivadis
Exadata Hybrid Columnar Compression (HCC)
12
A compression unit is a logical structure spanning multiple DB blocks
Each row is self-contained within a compression unit
Data is organized by column during data load
Each column is compressed separately
Smart Scan is supported
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
2014 © Trivadis
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
In-Memory Overview
13
2014 © Trivadis
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
14
Columnar Storage: Example for Columnorientation
2014 © Trivadis
In-Memory Option – What is it ?
Oracle Databases traditionally stores Data in Row Format
Oracle Database In-Memory Option uses an In-Memory Column Store
A new memory component in the SGA called In-Memory Area
Data are stored in column format in the In-Memory Column Store (IMCS)
Both Memory Structure (Buffer Cache and IMCS) will be populated
Dual Format Architecture does not double the Memory requirement
The Optimizer is fully aware about the In-Memory Column Store
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
15
Source: Oracle
2014 © Trivadis
In-Memory Option – Database Tiering
With the In-Memory Area not all data must fit in Memory
Less performance sensitive can reside on lower cost Disk
IM Option acts transparently against the applications
On Exadata acts on the top of all existing Storage tiers
It is also fully compatible with the new Multitenant architecture
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
16
Source: Oracle
2014 © Trivadis
In-Memory Option – SIMD
In-Memory Column Store are scanned with SIMD Vector Processing
SIMD = Single Instruction Multiple Data
SIMD Processing are evaluated in a single CPU Instruction
IM Column Store are specifically designed to maximize the number of
column entries that can be loaded into Vector Registers
Theoretically Scan of billions of rows per
second are possible
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
17
2014 © Trivadis
Summary: Oracle Database In-Memory
Extreme Performance: Analytics & OLTP
Extreme Scale-Out & Scale-Up
Extreme Availability
Extreme Simplicity
Powering the Real-Time Enterprise
All In-Memory Benefits With No Application Changes
In Memory Option on Exadata - the best of all?!?
DOAG-Konferenz 2014
18
2014 © Trivadis
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
In-Memory Configuration
19
2014 © Trivadis
In-Memory Option – The way to use it…
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
20
On Exadata there are some prerequisites…
Storage Server
Min. 12.1.1.1.0*
RDBMS
12.1.0.2
Database Server
Min. 12.1.1.1.0
Grid Infrastructure
12.1.0.2
* Full Offload Support for 12.1.x RDBMS
2014 © Trivadis
In-Memory Option – Configuration
Configuration is very simple
The Initializations Parameters INMEMORY_SIZE controls the In-
Memory Area
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
21
SQL> alter system set inmemory_size=16G scope=spfile sid=‘*’;
SQL> select * from v$sga;
NAME VALUE CON_ID
-------------------- --------------- ----------
Fixed Size 5304248 0
Variable Size 11274289224 0
Database Buffers 108716359680 0
Redo Buffers 263131136 0
In-Memory Area 17179869184 0
2014 © Trivadis
In-Memory Option – Configuration
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
22
Initialization Parameter Comment
inmemory_clause_defaultenables you to specify a default In-Memory
Column Store (IM column store) clause for new
tables and materialized views
inmemory_force
allows you to specify whether tables and
materialized view that are specified as
INMEMORY are populated into the In-Memory
Column Store or not
inmemory_max_populate_serversspecifies the maximum number of background
populaserverste
inmemory_queryINMEMORY_QUERY is used to enable or
disable in-memory queries for the entire
database at the session or system level
inmemory_sizeINMEMORY_SIZE sets the size of the In-
Memory Column Store (IM column store) on a
database instance
inmemory_trickle_repopulate_ser
vers_percent
limits the maximum number of background
populate servers used for In-Memory Column
Store (IM column store) repopulation
optimizer_inmemory_awareenables or disables all of the optimizer cost
model enhancements for in-memory
2014 © Trivadis
In-Memory Option – Configuration
Tables or Tablespace must be enabled to use IM Columnn Store
Depending on configuration the IMCS will be populated
Activation at Tablespace and Table Level
At Table Level configuration for compression and priority are possible
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
23
2014 © Trivadis
In-Memory Option – Configuration
Compression Types
Priority Levels
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
24
Priority Level Description
Critical Objects are populated immediately
High Objects are populated after all critical objects are populated, if remains space
MediumObjects are populated after all critical and high objects are populated, if remains
space
LowObjects are populated after all critical, high and medium objects are populated, if
remains space
None Objects are populated only after they are scanned
Compression Type Description
No Compress No Compression
Memcompress for DML Compression optimized for DML
Memcompress for Query Low Compression optimized for Query (default)
Memcompress for Query High Compression optimized for Query and Space saving
Memcompress for Capacity Low Compression optimized with orientation towards Space saving
Memcompress for Capacity High Compression optimized for Space saving
2014 © Trivadis
In-Memory Option – Configuration
Activation at Tablespace Level
At Table Level configuration for compression and priority are possible
memcompress for query low and priority none are the default
settings
Table activation with compression and priority settings
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
25
SQL> alter tablespace SOE inmemory;
SQL> alter table orders inmemory;
SQL> alter table orders inmemory memcompress for query low priority
critical;
2014 © Trivadis
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
In-Memory Management
26
2014 © Trivadis
In-Memory Option – Licensing
License is need for using the In-Memory Option
~ 18000 € / per Processor
The Option is installed with the RDBMS Installation, however not
activated
Bug 19308780 – Wrong Feature Tracking, fixed in PSU October 2014
Very “lovely” discussion between Maria Colgan (Oracle IM Option Product
Manager) and Kevin Closson (“independent” Oracle/Exadata specialist)
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
27
2014 © Trivadis
In-Memory Option – Management
Few v$ Views are in place for In-Memory Column Store management
Not all v$ Views are documented (only red ones)
New IM Memory Option PL/SQL Package
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
28
SQL> desc dbms_inmemory
..
PROCEDURE DISABLEFASTSTART
PROCEDURE ENABLEFASTSTART
PROCEDURE POPULATE
PROCEDURE REPOPULATE
SQL> select table_name from dict where table_name like 'V$IM_%';
TABLE_NAME
------------------------------
V$IM_COLUMN_LEVEL
V$IM_COL_CU
V$IM_HEADER
V$IM_SEGMENTS
V$IM_SEGMENTS_DETAIL
V$IM_SEG_EXT_MAP
V$IM_SMU_CHUNK
V$IM_SMU_HEAD
V$IM_TBS_EXT_MAP
V$IM_USER_SEGMENTS Only V$ Views in red are documented
Only Procedure in red are documented
2014 © Trivadis
In-Memory Option – Management
New dba_/all_/user_ views attribute are been implemented
Monitoring of In-Memory Segment
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
29
SQL> desc user_tables
Name Null? Type
----------------------------------------- -------- ----------------------------
TABLE_NAME NOT NULL VARCHAR2(128)
.
INMEMORY VARCHAR2(8)
INMEMORY_PRIORITY VARCHAR2(8)
INMEMORY_DISTRIBUTE VARCHAR2(15)
INMEMORY_COMPRESSION VARCHAR2(17)
INMEMORY_DUPLICATE VARCHAR2(13
SQL> select owner, segment_name, segment_type, inmemory_size, bytes_not_populated,
populate_status from v$im_segments;
OWNER SEGMENT_NAME SEGMENT_TYPE INMEMORY_SIZE BYTES_NOT_POPULATED POPULATE_
---------- -------------- --------------- ------------- ------------------- ---------
EXA IM2 TABLE 1179648 0 COMPLETED
EXA IM4 TABLE 8519680 0 COMPLETED
2014 © Trivadis
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
In-Memory with Exadata Tests
30
2014 © Trivadis
147 Mio Rows, 10GB
IM is factor 10 faster then jus Buffer cache
First population of IM data factor 4 faster with offloading
80% offload efficiency
Dedicated Query
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
31
SQL> SELECT SUM(amount_sold) FROM sales;
Cell
offload_processing
FALSE TRUE CACHE FALSE TRUE
INMEMORY_QUERY DISABLE DISABLE DISABLE ENABLE ENABLE
Elapsed time 56.82 13.51 1.17 0.13 013
Factor 1 4 48 437 437
2014 © Trivadis
Dedicated Query
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
32
SQL> SELECT t.time_id,
TO_CHAR(SUM(amount_sold), '9,999,999,999') AS sales,
TO_CHAR(AVG(SUM(amount_sold))
over(ORDER BY t.time_id range BETWEEN INTERVAL '2' DAY
preceding AND INTERVAL '2' DAY following), '9,999,999,999')
AS centered_5_day_avg
FROM sales s, times t
WHERE s.time_id = t.time_id
GROUP BY t.time_id
ORDER BY t.time_id;
Cell
offload_processing
FALSE TRUE CACHE FALSE TRUE
INMEMORY_QUERY DISABLE DISABLE DISABLE ENABLE ENABLE
Elapsed time 01:30.14 29.49 05.95 05.49 05.48
Factor 1 3 15 16 16
2014 © Trivadis
In-Memory Option – Exadata Test Enviroment
Eighth Rack (2 DB Server – 256 Gb RAM, 3 Storage Server)
Swingbench (charbench) used as Benchmarking Tool (2 Min)
OLTP Profile with 100 concurrent Users
- 60/40% Read/Write ratio
DSS Profile with 16 concurrent Users
- 100/0% Read/Write ratio (Rollup and Cube Function)
In-Memory Area with 16G Memory (SGA 128GB)
All Columns of involved Tables (full Schema) are stored in the IM
Column Store (~ 10GB) and populated
Duplicate Feature intended for RAC Environment was not used
This Feature should be considered in HA or Scalability Environments
Our first steps in this way, was not so encouraging in terms of scalability
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
33
2014 © Trivadis
In-Memory Option – Test Cases
1. Smart Scan off , no In-Memory
2. Smart Scan on, no In-Memory
3. Smart Scan off, In-Memory compression “for DML”
4. Smart Scan on, In-Memory compression “for DML”
5. Smart Scan on, In-Memory compression “for query low”
6. Smart Scan on, In-Memory compression “for query high”
7. Smart Scan on, In-Memory compression “for capacity low”
8. Smart Scan on, In-Memory compression “for capacity high”
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
34
2014 © Trivadis
In-Memory Option – Test Results OLTP
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
35
Total completed Transaction
0
20000
40000
60000
80000
100000
120000
140000
160000
180000
Total Completed Transaction
2014 © Trivadis
In-Memory Option – Test Results OLTP
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
36
Average CPU Utilization
0
10
20
30
40
50
60
70
80
90
Avg User CPU
Avg Sys CPU
CPU Usage
2014 © Trivadis
Total completed Transaction
0
20000
40000
60000
80000
100000
120000
140000
160000
180000
0
10
20
30
40
50
60
70
80
90
In-Memory Option – Test Results OLTP
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
37
2014 © Trivadis
In-Memory Option – Test Results DSS
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
38
Total completed Transaction (Reports)
0
20
40
60
80
100
120
Total Completed Transactions
2014 © Trivadis
In-Memory Option – Test Results DSS
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
39
Average CPU Utilization
0
10
20
30
40
50
60
Avg User CPU
Avg Sys CPU
CPU Usage
2014 © Trivadis
0
20
40
60
80
100
120
0
10
20
30
40
50
60
In-Memory Option – Test Results DSS
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
40
Total completed Transaction (Reports)
2014 © Trivadis
In-Memory Option on Exadata – Result discussion
OLTP
factor 12 :-o
No offloading due to no direct path functionality
All in cache with high CPU load
40% read profit of IMCS
OLTP less CPU with IMCS due to SIMD benefit
DSS
No Offloading/No IMCS as fast but a lot of CPU Buffer Cache
Offloading, as fast but less CPU work done in Storage Server
IMCS 25% plus work, less CPU
The dataset and ressources are not in a good balance to test
Much more data needed
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
41
2014 © Trivadis
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
Conclusion
42
2014 © Trivadis
Exadata Database Machine works fine
Oracle IMCS Option works fine
Testing and interpretation is not trivial
The option costs 50% of EE core price
A serious evaluation is needed
PoC to find the best of all, unless
you need a:
In-Memory Option on Exadata – Conclusion
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
43
2014 © Trivadis
Questions and answers ...
2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
Konrad HÄFELI
Tel: +41 58 459 59 69
DOAG-Konferenz 2014
In Memory Option on Exadata - the best of all?!?
44