CON9528_PDF_9528_0001

77
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1

description

Oracle Best practices

Transcript of CON9528_PDF_9528_0001

Page 1: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1

Page 2: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 2

Oracle Exalytics Best Practices -- Getting the Most out of Your Oracle Exalytics Deployment

Deb Bhattacharjee Director

Lionel Hsu Consulting Member of Technical Staff

Saugata Chowdhury Consulting Member of Technical Staff

Oct. 3, 2012

Page 3: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 3

The following is intended to outline our general product direction. It is intended

for information purposes only, and may not be incorporated into any contract.

It is not a commitment to deliver any material, code, or functionality, and should

not be relied upon in making purchasing decisions. The development, release,

and timing of any features or functionality described for Oracle’s products

remains at the sole discretion of Oracle.

Page 4: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 4

Agenda

What is Exalytics?

Exalytics Performance Tuning

Exalytics Considerations

Page 5: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 5

WHAT IS EXALYTICS?

Page 6: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 6

Oracle Exalytics In-Memory Machine

Benefits • Speed-of-Thought Interactive Visual Analysis

• Faster Planning Cycles with Richer Models

• Quick to Deploy, Supports More Users

Unique Features • In-memory Analytics

• Accelerates BI and EPM Apps

• Fits with existing data sources, infrastructure

• Full-stack optimizations – hardware & software

• Optimized with Exadata – InfiniBand

Oracle Exalytics is the industry's first in-memory machine that

delivers the fastest performance for business intelligence and planning applications

AVAILABLE TODAY

Oracle Exalytics is the industry's first in-memory machine that

delivers the fastest performance for business intelligence and planning applications

Page 7: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 7

Exalytics Components

Memory Optimized

Hardware (1TB RAM, 40 cores)

BI Foundation Suite

Software (OS, middleware, BI)

In-Memory Software (TimesTen for Exalytics and

Optimized Essbase)

Page 8: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 8

Oracle Exalytics Hardware

Memory 1 TB RAM Compute 4 Intel® Xeon® E7-4870, 40 cores total Networking 40 Gbps InfiniBand – 2 ports 10 Gbps Ethernet – 2 ports 1 Gbps Ethernet – 4 ports 8 Gbps FibreChannel – 2 ports Storage 3.6 TB HDD Capacity

Page 9: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 9

Exalytics Storage Configuration

600 GB SAS - 0

600 GB SAS - 1

600 GB SAS - 2

600 GB SAS - 3

600 GB SAS - 4

600 GB SAS - 5

RAID 1

RAID 5 /u01(1.7TB) – software, TT data files

Single Disk Fault tolerant

Separate OS and Data disks

/boot (99MB) – boot partition

/ (447GB) – Operating system, logs, temp space

Page 10: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 10

Exalytics Software Components

Timesten DB

Exalytics

Data Warehouse

Webcat

OBIPS

(sawserver) OBIS

(nqserver)

WLS (bi_server)

(analytics app)

Repository

Browser/Mobile Javahost

(Chart Engine)

Essbase

Page 11: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 11

Exalytics – Some Performance Datapoints

• 2.7x throughput gain for on Exalytics when compared to 11.1.1.5 on same hardware (comparing peak throughput of 637 vs 235)

• Baseline’s throughput actually dropped under load due to lock contentions

Throughput Comparison - Baseline vs software optimization (datasource remaining constant) s

146

235

194

151121 127

71

205

368

493

563

605 619637

0

100

200

300

400

500

600

700

0 10 20 30 40 50 60 70 80

# of Virtual Users

Th

rou

gh

pu

t

Baseline - Throughput Exalytics - Throughput

Page 12: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 12

Exalytics – Some Performance Datapoints

• BI Query reporting w/ Exalytics on Oracle DB

• In-memory aggregates (Adaptive In-Memory Cache) vs. the baseline DW: 18X faster on the average

• Individual dashboard response time gain upto 45x (Filter Department)

Response Time Comparison (Slowest transactions) - Baseline vs. Exalytics (non-Exadata)s

Page 13: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 13

Exalytics – Some Performance Datapoints

• BI Query reporting w/ Exalytics on Exadata

• 23X average response time improvements thanks to in-memory aggregates, fast inter-connect and SQL optimizations for Exadata as compared to the base system

• Individual transaction gains upto 60X (Filter-Department)

Response Time Comparison (Slowest transactions) - Baseline vs. Exalytics w/ Exadata datawarehouses

Page 14: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 14

Exalytics – Some Performance Datapoints

• About 5x response time improvement under load

• 7x improvement in CPU Utilization by Essbase

Response Time improvement for end-to-end scenario for Planning PSB usecase with Essbase s

0

100

200

300

400

500

600

20 40 60 80 100

AvgResponseTime insec (Exalytics)

Avg ResponseTime in sec(11.1.1.2.1)

Page 15: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 15

Exalytics – Some Scalability Datapoints

• Up to 93x response time improvement with individual MDX Queries

• Up to 15x response time improvement for individual interactive calc scripts

• About 10x overall throughput improvement in mixed load scenarios accompanied by 1.8x CPU usage improvement

Response Time improvement for MDX Queries and interactive Calc under load with Essbase BSO Cubes

Page 16: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 16

Exalytics

OBIEE Runtime Performance for dashboard navigation

– Less than 5% overhead for dashboard navigation

– Setup

Bare metal : 40 cores , 1 TB RAM

OVM ver 3.0.3: Dom 0 pinned to 10 cores, 40 vcpu for OBIEE VM and 40 vcpu for TT

Bare Metal Virtualized

(40 Cores) OBII 40 vcpu

TT 40 vcpu No of users (0 thinktime) 60 60

Throughput (tps) 337 329 <5% overhead in virtualized

environment with same core

count

Avg response time (sec) 0.18 0.18

Total CPU % (BI + TT) 91% 93%

Metrices Comments

Performance characteristics with OVM

Page 17: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 17

Exalytics

Essbase BSO Interactive Calc and Query

– Less than 10 % overhead in virtualized environment compared to bare metal

– Setup

Bare metal : 40 cores , 1 TB RAM

OVM ver 3.0.3: Dom 0 pinned to 10 cores

9.1

16.3

22.6 25.5

8.4

15.7

21.7 22.9

0

5

10

15

20

25

30

1 2 3 4Th

rou

gh

pu

t T

xn

/se

c

# CPU Sockets and # of Essbase instances

Metal - 40 cores

OVM -- 40 cores

Performance characteristics with OVM

Page 18: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 18

EXALYTICS PERFORMANCE TUNING

Page 19: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 19

Timesten DB

Exalytics Performance Map • DB connection

•Log Level

Exalytics

Data Warehouse

Webcat

OBIPS

(sawserver) OBIS

(nqserver)

WLS (bi_server)

(analytics app)

Repository

Browser/Mobile

Javahost

(Chart Engine)

Essbase

•Hardware Acceleration flag

•Logical SQLs/Physical SQLs

•Init block

•Server Thread pool

• DB Gateway connection

pool

•OBIS Cache

•Temp/log folder location

•Log level

•JVM Heap Size for Trellis Charts

•Thread Pool size

•Javahost Clustering

•Temp/log folder location

•JVM Heap size

•Analytics app ConnectionPool

•Analytics app session timeout

•WLS LDAP connection pool

•Log folder location

•Hardware Acceleration flag

•Chart Thread Pool

•OBI PS Cache

•Session timeouts

•Cursor Wait Time

•Memory Quota

•Temp/log folder location

Page 20: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 20

Exalytics Set the right Log Level • Set the default log level to ‘Error’

• Log levels should be the first place to check while debugging performance issues

Page 21: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 21

Exalytics

Move temp files and log files to ramdisk for better scalability

– OBIPS

cd $BI_HOME/instances/instance1/tmp/OracleBIPresentationServicesComponent

mv coreapplication_obips1 coreapplication_obips1.localDisk

ln -s /dev/shm/coreapplication_obips1 .\

– Javahost

mkdir /dev/shm/coreapplication_obijh1

cd $BI_HOME/instances/instance1/tmp/OracleBIJavaHostComponent

mv coreapplication_obijh1 coreapplication_obijh1.localDisk

ln -s /dev/shm/coreapplication_obijh1 .\

– OBIS tmp folder

mkdir /dev/shm/coreapplication_obis1

cd $BI_HOME/instances/instance1/tmp/OracleBIServerComponent

mv coreapplication_obis1 coreapplication_obis1.orig

ln -s /dev/shm/coreapplication_obis1 .

– OBIS Sorting/Working directory: since temp folder is pointing the RAMDISK from above settings, use default for

sorting/working directory so it will be using RAMDISK as well

WORK_DIRECTORY_PATHS = ""; (blank means using default path)

Move tmp/log files to RAMDisks

Page 22: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 22

Exalytics OBIS file buffer size

– Increase file buffer size to 128KB to ensure good write to disk performance of sorting and cache

Population (28.5x improvement – in 11.1.1.7 )

CACHE_FILE_BUFFER_SIZE = 128KB; (in NQConfig.INI)

I/O buffer size

Page 23: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 23

Exalytics

– OBIPS Caches (instanceconfig.xml)

Leveraging large RAM to bump up OBI PS Caching

<Cache> <CatalogXml> <MaxAgeMinutes> : 240

<Cache> <CatalogXml> <MaxLastAccessedSeconds>: 14400

<Cache><Query><MaxEntries>: 5000

<Cache><Query><AbsoluteMaxEntries>: 20000

<Cache><Query><CruiseEntries> : 3000

<Cache><Query><ForceLRU> : true

<Cache><Accounts><CleanupFrequencyMinutes>:14400

<Cache><AccountIndex><CleanupFrequencyMinutes>:14400

<Cache><CatalogAttributes> <CleanupFrequencyMinutes>:14400

– OBIS Cache settings (NQSConfig.INI)

[CACHE]

ENABLE = YES;

MAX_CACHE_ENTRIES = 5000;

MAX_CACHE_ENTRY_SIZE = 256 MB;

MAX_ROWS_PER_CACHE_ENTRY = 100000 ; // Too many rows in cache can slow down performance

[SERVER]

INIT_BLOCK_CACHE_ENTRIES = 5000;

Caching

Page 24: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 24

Exalytics

OBIPS Chart Threadpool size(instanceconfig.xml)

– OBIPS has a thread pool for Javahosts. Its size needs to be the total number of threads allowed in all Javahosts.

– For example, there are two Javahost instances . Each has 300 threads defined in its own configuration file, and then the Max size for

OBIPS threadpool size needs to be 600 (300 + 300).

<ThreadPoolDefaults>

<ChartThreadPool>

<MinThreads>300</MinThreads>

<MaxThreads>600</MaxThreads>

<MaxQueue>4096</MaxQueue>

</ChartThreadPool>

</ThreadPoolDefaults>

Javahost Threadpool size (config.xml)

$BI_HOME/instances/instance1/config/OracleBIJavaHostComponent/coreapplication_obijh1/config.xml

<JobManager>

<MinThreads>300</MinThreads>

<MaxThreads>300</MaxThreads>

<MaxPendingJobs>300</MaxPendingJobs>

</JobManager>

OBIS Threadpool (in NQConfig.INI)

SERVER_THREAD_RANGE = 500-1000;

DB_GATEWAY_THREAD_RANGE = 500-1000;

AGGREGATE_THREAD_POOL_SIZE = 5;

Configure Right Threadpool Size

Page 25: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 25

Exalytics

OBIS RPD

– AdminTool: Connection Pool -> General Tab -> Maximum connections

WLS

– LDAP connection pool size

in WLS, needs to meet the concurrent login rate

Configure Right connection pool size

Page 26: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 26

Exalytics

• WLS /analytics plug-in Max connections – bridgeconfig.properties ($BI_HOME/user_projects/domains/bifoundation_domain/config/fmwconfig/biinstances/coreapplication/bridgeconfig.properties)

oracle.bi.presentation.sawconnect.ConnectionPool.MaxConnections=512

TimesTen max connection

– Make sure you have enough Semaphore

/sbin/sysctl -w kernel.sem=2048 (to set the value of kernel.sem)

/sbin/sysctl kernel.sem (to check the value of kernel.sem)

kernel.sem = 2048 64000 256 64

– In the Timesten server side DSN entry

For example

[crm_dwh2_adm]

Driver=/home/tt/TimesTen/dev/lib/libtten.so

DatabaseCharacterSet=US7ASCII

...

Connections=300

...

• OBIS Session LIMIT (NQSConfig.INI) [SERVER]

MAX_SESSION_LIMIT = 10000 ;

Configure Max connections/Sessions

Page 27: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 27

Exalytics

• WLS Analytics plug-in session timeout • Set the ‘Session timeout(in seconds) to control the timeout of the sessions in Analytics plug-in in WLS

• The timeout should be in sync with the session timeout in OBI PS.

OBIPS Session timeout

<ClientSessionExpireMinutes>5</ClientSessionExpireMinutes>

Check Session Timeouts

Page 28: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 28

Exalytics

OBIPS Memory Quota Checking

– Set the following shell variable for OBIPS in opmn.xml

SIEBEL_MEMORY_THREADALLOC_QUOTA=8388608 (e.g to set thread quota to 8MB)

– UI will throw an error when a user’s request consumes more than this memory

For very large TimesTen Datastore (>256GB), The following Linux kernel and Timesten

settings need to be changed. Please refer to Exalytics Best Practices doc for details.

– kernel.shmmax/kernel.shmall/kernel.sem

– /proc/sys/vm/nr_hugepages

– /proc/meminfo: Hugepagesize

– ttendaemon.options: linuxLargePageAlignment

Exalytics Memory Tuning

Page 29: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 29

Exalytics

• WLS BI Managed Server Heap sizes

• This needs to be modified based on the number of concurrent sessions and BIP reports.

SERVER_MEM_ARGS="-Xms256m -Xmx2048m -XX:MaxPermSize=512m“ (setDomainEnv.sh)

export SERVER_MEM_ARGS

Javahost heap sizes (opmn.xml) – For scenarios with Trellis charts , increase JVM heap size to 8GB (-xmx8192M )

<ias-component id="coreapplication_obijh1">

<data id="start-args" value="-server -Xms2048M -Xmx8192M … "

JVM heap size

Page 30: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 30

Exalytics

Clustering Javahosts can help scalability

– Use EM to increase the number of instances of Javahost

Javahost clustering

Page 31: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 31

Exalytics

Repository init-blocks : Called once during

server startup and refreshed after the

refresh interval as needed

Session init-blocks: called for each new

session; slow SQL can slow down login

Session init-blocks should be minimized

and lazily loaded (deferred)

– Previously it was not possible to defer row-

wise init-blocks ; restriction has been lifted in

11.1.1.6

Init-blocks should be associated with it’s

separate connection pool to reduce

contention

– Ensure enough connections are available

OBIS - Init Blocks

Page 32: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 32

Currently BI does not support issuing SQLs with bind variables to the database

– May result in high parse time in the database specially for very complex Queries

Work around: Add inititalization code in connection setting to change session parameter for

cursor_sharing (Oracle only)

– In the Execute on connect section,

click New and add:

alter session set cursor_sharing='FORCE';

– In the Execute on disconnect section,

click New and add:

alter session set cursor_sharing='EXACT';

OBIS using Bind Variables

Exalytics

Page 33: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 33

Exalytics OBIS Query Loglevel

– Loglevel 2 provided the logical query , physical

query and the response times

It has reasonable low overhead in low to

moderately busy system and can be kept at level 2

in production

Keep the log file in RAMDISK to reduce the IO

overhead

– Needs rpd access to change

Work-around: Have a session init-block defined

and read the value from a DB table

OBIS Log Analysis for Performance Tuning

Page 34: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 34

Exalytics Inspecting query log

– Provides the physical datasource name [username:ADMINISTRATOR] --- Sending query to database named FSCM_OLTP (id: <<966402>>), connection pool …

– Query Status Query Status: Successful Completion

– Row Count from the database [username:Administrator] --- Execution Node: <<27109>> DbGateway Exchange, Close Row Count = 1, Row Width = 10

bytes

– Physical SQL response time [username:ADMINISTRATOR] ----- Physical query response time 5 (seconds), id SQLBypass Gateway

Mapping a slow Query in database to a logical query in BI (11.1.1.6+)

– BI physical query will add a hash as a comment - “/* 2c16620a */ select T2005.Country as c…”

– BI Query log will have the corresponding hash -------------------- SQL Request, logical request hash: fdc49e9c

Sending query to database named FSCM_OLTP (id: <<1712462>>), connection pool named Connection Pool, logical request

hash fdc49e9c, physical request hash 2c16620a: [[

OBIS Log Analysis for Performance Tuning (cont’d)

Page 35: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 35

Exalytics

WLS Console

– # of sessions (/analytics) in WLS

EM (Enterprise manager)

– Sessions, requests/s…etc for WLS, OBIPS, OBIS & Javahost

OBIPS perfmon page

– http://<hostname>:<port>/analytics/saw.dll?perfmon

BIEE AdminTool

– Manage-Sessions: # of sessions, current request and session variables

Monitoring

Page 36: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 36

Exalytics Monitoring – WLS Console

Page 37: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 37

Exalytics

Monitoring – EM (Enterprise manager)

Page 38: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 38

Exalytics Monitoring - OBIPS perfmon page /analytics/saw.dll?perfmon

Page 39: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 39

Exalytics

Monitoring – AdminTool - Sessions

Page 40: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 40

Exalytics

OBIPS Cursor Wait Time

– There are two parameters that control how long OBI PS should wait before replying ‘refreshing page’

to the browser. NewCursorWaitSeconds is the time to wait on the initial callback from the UI for a

query to finish. OldCursor is for subsequent call backs for the query to finish.

– It is helpful to increase the ‘NewCursorWaitSecons’ for performance load testing, so the load test

scripts won’t error out because of not seeing the expected results

<Cursors>

<NewCursorWaitSeconds>1000</NewCursorWaitSeconds>

<OldCursorWaitSeconds> 1000</OldCursorWaitSeconds>

</Cursors>

Configuration changes for Load testing scripts

Page 41: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 41

EXALYTICS CONSIDERATIONS

Page 42: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 42

Exalytics Version Stripe Core components – optimized, tested together

Exalytics PS1

Exalytics V1

Page 43: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 43

Exalytics

OBIEE – Performance acceleration for Analytic Queries

Technique Expected Performance Improvement Considerations

Cached result set Consistently sub-second • For relatively static dashboard

• Pre-seeding cache, use IBOTs

• Purge cache on source data changes

• Use RAMDisk for storing the cache

Aggregate based acceleration Depends on row count reduction

• Aggregate Persistence for aggregate creation

• Summary Advisor for aggregate recommendation

• Build high-level KPI’s with summary level performance and

exception monitoring to tell a story

• Just raw data reports have no competitive advantage.

• OBIEE advantage is in multi-pass calculations, indexing,

comparisons

• Drill-down reports via Action Links to root cause analysis

Moving data into TimesTen Orders of magnitude faster depending on the queries

and the reference database

• Sizing and future data growth.

• Freeing up DB resources, in turn benefits other queries

Page 44: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 44

Exalytics

TimesTen database is supported

data source

BI Server connects to TimesTen v11

database using TT specific ODBC

3.5

BI Server – TimesTen Connection Architecture

Server daemon

TimesTen

In-Memory

Database

BI Server

TimesTen client driver

Data Store subdaemon(s)

Admin/Utility programs

Transaction Log files

Checkpoint files

Server proxies

client/server connections

within the same host

(shared memory IPC)

Page 45: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 45

OBIEE

Make sure the DB type is set

accurately.

– Typically more recent DB version will

support a wider range of functions.

BI Server internal execution

– Any function not supported by data

source

– Common Performance issue:

Using such function in filters

derived columns used in filters

Modeling – Function Shipping

Page 46: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 46

OBIEE

– Use foreign key joins wherever possible

– Complex joins

Avoid

Use simple equality joins

Set right cardinality ({0..1, 1, N} to {0..1, 1, N, Unknown}) –

important for join trimming

Outer joins

– Logical level

Dimension preserving behavior

– LTS source Mapping

For all other cases

– Outer join conversion technique

– Add dimension table row for N/a or Unknown records during ETL

– Replace Null foreign keys with the N/A value either during ETL or

via IfNull expressions in the RPD

Physical and Logical Joins

Page 47: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 47

OBIEE

defines the unique elements in each

logical level

– Portland (ME), Portland (OR) are distinct

– Basis of dimensional conformance across

sources

– Use unique, single column level keys

whenever possible

– Do not use derived columns

Consider LTS expression

Level Keys

Page 48: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 48

OBIEE

Use Descriptor ID Columns,

– Answers Filters / Dashboard Prompts on

description

Database filters in terms of Key Columns

– Index / Partition pruning friendly SQL

SELECT

0 s_0,

"Airlines Delay"."Carrier"."Carrier" s_1,

DESCRIPTOR_IDOF( "Airlines Delay"."Carrier"."Carrier") s_2

FROM "Airlines Delay“

WHERE

(DESCRIPTOR_IDOF( "Airlines Delay"."Carrier"."Carrier")='Q5')

ORDER BY 1, 2 ASC NULLS LAST

FETCH FIRST 100001 ROWS ONLY

OBIEE - Descriptor ID columns

Page 49: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 49

OBIEE

Runs in Online mode

Detects artifacts

– that can cause correctness issues

Uniqueness of level keys

Uniqueness of physical table keys

– that prevent BI to recommend/create

aggregates

Session Variables in LTS expression

Session Variables in LTS content filter

Generates warning

– Attributes not mapped to levels (impact on

aggregate navigation)

Model Checker

Page 50: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 50

AGGREGATES

Page 51: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 51

OBIEE

Ability to generate aggregate tables

based on logical star and map back

to the metadata repository

Wizard to allow the administrator to

identify measures and grains for

aggregate tables

Output is a set of scripts to create and

populate aggregate tables, as well as

automatically map sources to the rpd

Aggregate Persistence

Page 52: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 52

OBIEE

create aggregates

"ag_4183394923“ (aggregate name)

for "Airlines"."Flight Facts“ (logical fact table)

("Actual Elapsed Time (Min)","Air Time (Min)","Arr Lag (Min)",...) (measures)

at levels

("Airlines"."00 Time"."Time Year", "Airlines"."18 Route"."18 Route Detail", "Airlines"."11 Orig Airport Map"."11 Orig Division Map") (grain – level

combination)

using connection pool

"Aggregates (TT)"."Connection Pool" (connection pool)

in "Aggregates (TT)".."AIRLINES"; (schema)

Aggregate Generation Script

Page 53: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 53

Exalytics

Analyzes query workload over time

– Usage Tracking

– Summary Statistics table

– Cache hits are ignored.

Recommends minimal set of aggregates to

improve query response for the overall

system.

Does not actually create aggregates, but

outputs a script that is fed to the Aggregate

Persistence engine.

Summary Advisor

Page 54: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 54

Exalytics

Analyze facts

– Eliminate the ones not suitable for

aggregation

– Few facts at a time

Analyze over specific time window

OBIEE - Summary Advisor

• Accelerating Dashboard ?

• Select logical facts accessed

Page 55: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 55

Exalytics

Analyze facts frequently queried

Specify Aggregation Target

– Oracle

– TimesTen

OBIEE - Summary Advisor

• Workload sensitivity

• Cumulative time because of lots of small

queries or small number of time

consuming queries

• May need to modify usage tracking tables

manually to feed summary advisor right

input

Page 56: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 56

Exalytics Summary Advisor – Stopping criteria

Default stopping criteria 0%

Longer recommendation time

Longer Aggregate build/refresh time

Page 57: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 57

Exalytics Summary Advisor – Execute Aggregate Advisor

Page 58: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 58

Exalytics

Update database statistics (Oracle) before running Summary Advisor

Aggregate Recommendation depends on

– Database response times

Hardware, Table sizes, Indexes, tuning etc

– Workload

Subject area, dashboard

– BI Server cache settings

Rerun Summary Advisor in case of changes in the factors

No recommendation

– Acceleration by aggregation unlikely

Reporting at the detailed grain

Reporting at the existing aggregate grain

Summary Advisor

Page 59: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 59

OBIEE

AGO/TODATE functions will be navigated to the

detail source

– for lack of ordering information, even when the

aggregates are available

(Quarter, Sales) will hit the aggregates at

the month grain,

(Quarter, PeriodAgoSales) will not.

Specify Chronological keys for upper levels along

Time dimensions

Aggregate Navigation - Chronological keys

Page 60: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 60

OBIEE

– E.g. Inventory measure, Account Balance etc.

– Order of Aggregation is important for correct result

– For a given query grain, matching aggregate should

honor the order.

– Consider two LTS

1. Aggregate LTS: (city, month)

2. Detailed LTS: (store, day)

Consider two queries

1. Query1: (city, year) grain

2. Query2: (state, month) grain

Only query 1 can leverage Aggregate

– “Data is dense” – permits reordering of aggregation

Review the source data before setting

Both queries will leverage aggregate.

Aggregate Navigation – Measures w/ Dimension Specific Aggregation

Country<-Region<-State<-City<-Store

Year<-Quarter<-Month<-Day

Page 61: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 61

OBIEE

Count distinct aggregation

– BI Server uses the most detailed grain possible

– Unless you register additional LTSes explicitly

Aggregate Persistence

– Creates aggregates with the counts

– Not reusable except for Exact match with query grain

Consider expressing Distinct Count measure

as expression, referencing logical columns

– Summary Advisor can recommend suitable aggregates

Aggregate Navigation – Measures w/ Distinct Count aggregation

Page 62: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 62

Exalytics

Aggregates on a different target ?

– VPD enabled database

– Source specific SQL [Evaluate function]

Security implemented in LTS content filter

– Consider expressing security in terms of BI logical

columns, replace with BI security filter.

– Consider Role based security

LTS expressions:

– Space - Performance tradeoff

– More columns in aggregate table – compared to base

fact [ physical tables in the database]

– Aggregate build/refresh longer, query performance

faster

Aggregates – other considerations

Page 63: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 63

TIMESTEN

Page 64: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 64

Exalytics

Move Select Detail Data into TimesTen

– Use ODI to replicate from warehouse tables

– Use GoldenGate to replicate for Operational reporting

– Use ttBulkCp utility to load from text file

– Data volume considerations

TimesTen in-memory database

– Space for data (PermSize) – 400 GB

– Space for query processing (TempSize) – 400 GB

Use Columnar Compression to reduce the space requirement

Trim column sizes (Currency VARCHAR 15 VARCHAR 3)

Replicate only “query” columns

Summary Advisor can be used against detailed TimesTen tables as well.

TimesTen Data mart

Page 65: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 65

Exalytics

TimesTen implements native numeric types

– more efficient in query processing and improved space usage

BI Server automatically maps the BI NQS data types to TimesTen native data

types – INT maps to TT_INTEGER, DOUBLE maps to BINARY_DOUBLE etc.

Consider optimal data type mapping for replication to TT

– NUMBER without precision takes 22 bytes of storage

NUMBER (Max < 28) should be of type TT_TINYINT (1 Byte)

NUMBER (Max < 215) should be of type TT_SMALLINT (2 Bytes)

NUMBER (Max < 231) should be of type TT_INTEGER (4 Bytes)

TimesTen – Optimizing using native Data types

Page 66: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 66

Exalytics

Data load:

– Disable Redo log and use fuzzy checkpoints for faster load

Maintain up to date database statistics – Exact statistics: Use the ttOptUpdateStats built-in procedure

– Statistical sampling: Use the ttOptEstimateStats built-in procedure

Indexing

– Necessary for good query performance

– BI Server currently creates single-column indexes for aggregates stored in TimesTen

– Multi-column composite indexes can provide significant improvements for complex queries

– In case of replication, use the indexes on source table as reference

– Use TimesTen Index Advisor for recommendation on indexes, for specific SQL queries/workload (nqquery.log)

TimesTen – database tuning

Page 67: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 67

Exalytics

ttBackup creates a backup copy of a database that can be restored using the

ttRestore utility

– Full backup and incremental backup

ttBackup and ttRestore should be used to replicate across TimesTen instances

for HA/DR

TimesTen – database maintenance

Page 68: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 68

ESSBASE

Page 69: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 69

Exalytics

Parallel Data Load

– import database Sample.Basic using max_threads 12

Parallel Dense Restructure

– Set RESTRUCTURETHREADS in essbase.cfg for parallelism

Parallel Export

– PAREXPORT -threads 4 -in e:\data\input.txt 1 1;

– Default number of threads is specified using EXPORTTHREADS in essbase.cfg

Parallel Calculation in calculation scripts

Set CALCPARALLEL in essbase.cfg for parallelism and also set CALCTASKDIMS for the sparse

dimensions to be included in parallel tasks

ESSBASE – Block Storage - all things Parallel

Page 70: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 70

Exalytics

Multiple user concurrent queries and calculation script execution

– Design the application to allow for large degree of parallelism in queries and calculation script

execution.

Consolidation of Applications into single Exalytics box

– Exalytics has 4 sockets, 40 cores and large amount of memory and this allows us to consolidate

number of Essbase applications in different machines into a single box leading to efficient

management.

Set Socket Affinity for Each Application

– During consolidation, explicitly set the affinity to group of applications to a single socket so as to

efficiently utilize the memory associated with that socket.

Increase the memory foot print of index and block cache to take advantage of

in-memory calculation efficiencies

ESSBASE – Block Storage - considerations

Page 71: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 71

Exalytics

ASO scales linearly with the number of cores and memory

– Configure the ASO application to run on the entire box and assign as much memory and threads as possible

using essbase.cfg settings

Consolidation of Multiple ASO Applications into single Exalytics box

– Exalytics has 4 sockets, 40 cores and large amount of memory and this allows us to consolidate number of

Essbase ASO applications in different machines into a single box leading to efficient management.

Set Socket Affinity for Each Application

– During consolidation, explicitly set the affinity to group of applications to a single socket so as to efficiently utilize

the memory associated with that socket.

Consolidate multiple cubes into a single large cube in Exalytics box

– A large single cube is possible with the memory and CPU power of Exalytics and this avoids having to

consolidate the results from multiple cubes and re-aggregating and collating externally.

ESSBASE – Aggregate Storage - considerations

Page 72: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 72

Complete solution for Business Analytics

Pre-packaged hardware, software,

implementation, and support

Per User Per Month (PUPM)

promotional pricing

Ask us about this special offer……..

Analytics in a Box

Contact your Sales Consultant

today to learn more.

Page 73: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 73

Graphic Section Divider

Page 74: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 74

Page 75: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 75

Page 76: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 76

Exalytics

New flag ‘HardwareAccelarator’ for Exalytics – OBIPS (instanceconfig.xml)

<OracleHardwareAcceleration>true</OracleHardwareAcceleration>

– OBIS (NQConfig.INI)

ORACLEHARDWAREACCELERATION = YES

Tip 3: HardwareAccelarator flag

Page 77: CON9528_PDF_9528_0001

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 77

OBIEE Architecture