OBIEE Performance and RPD Optimization techniques Deb Bhatacharjee Sr. Director Anupam Bordia...

OBIEE Performance and RPD Optimization techniquesDeb BhatacharjeeSr. Director

Anupam BordiaConsulting Member of Technical Staff

Nathan ReynoldsArchitect

PSRSeptember, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 3

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

Best practices

Aggregate persistence

Caching

Case studies

Q&A

1

2

3

4

5


Prerequisites

• Good understanding of OBIEE architecture

• Some understanding of FA (Fusion Application) – BI Integration


Program Agenda

Best practices


Caching

Case studies

Q&A

1

2

3

4

5


Query governor – Global

• How do I prevent a BI Query from running more than 20 minutes and consuming most of the CPU?

• How do I prevent a BI Query from fetching more than a million rows?

•Order of precedence:

Global setting

Role based

Session based• Global setting in NQSConfig.ini:DEFAULT_DB_MAXEXECTIME=600; # in secondsDEFAULT_DB_MAXROWS=100000; # number of rows


Query governor – Role based

• Maximum governing value of all the available roles assigned to user

• To set, one can open user/role in AdminTool and set Max rows fetched or time after which query should be aborted if not completed.

Manage -> Identity -> Application Roles -> Permissions -> Query Limits

• Terminates query if it exceeds 10 minutes (not including fetch time) and/ or 100K rows

• Time granularity: 1 minute


Query governor – Role based


Query governor – Session based

Set below variables in query:OBIS_DB_MAXEXECTIME OBIS_DB_MAXROWS

SET VARIABLE OBIS_DB_MAXEXECTIME=60, OBIS_DB_MAXROWS=100000: SELECT…


Init Block - Defer

• Check if session init blocks are deferred:– Earlier row-wise init blocks could not be deferred

– Non deferred session init block are executed at each login delaying login

–Most of the session init blocks should be deferred. “Allow deferred execution” should be selected


Init Block - Defer


SQL Bypass

• SQL-bypass is the recommended approach for OTBI reports• Ensure all ADF queries are executed in SQL-bypass mode

• To verify, check nqquery.log for:<ADFQuery mode="SQLBypass" queryid="26500-19169" locale="en">

<ADFQuery queryid="26500-19169" locale="en"> # not using sql bypass

• System would be slow if missing sql-bypass database • 50x improvement observed if sql-bypass is used• Avoid projecting transient attributes, this results in BI-ADF broker skipping

sql-bypass mode and hence queries are slower


SQL Bypass


OBIS – Query logs

– In RPD, set session variable LOGLEVEL between 0 (no logging)-7 (extensive logging)– ‘2’ is recommended (on production) to get the logical and physical sqls

from the query logs, this setting has minimal performance impact–For detailed logging (not recommended for production instance), add

variables <variable id="LOGLEVEL" value="7"/> <variable id="NQUIRE_INTERNAL_LOGGING" value="1"/>under <ias-component id="coreapplication_obis1" inherit-environment="true"><environment> in file ~orainst\config\OPMN\opmn\opmn.xml


OBIS – Query logs


OBIS – Server logs

• Set system log level to 3 (to be set only during diagnostics)• Helps debugging issues related to init blocks• Time taken by LDAP server to authenticate• Time taken by each init block to execute• Example:

User ' [email protected] ' spent 14.000000 milliseconds getting http response when 'getAuthenticatedUserWithLanguageAndProperties'.User ‘[email protected]' spent 8833.000000 milliseconds at init block 'RPD Msgs’

mailto:[email protected]




OBIS – Server logs


OBIS Logs Setup Enterprise Manager (EM)

Log level of various BI components can be changed via EM. •Ensure that parameter centralConfigurationEnabled is set to “true” in file:~user_projects/domains/bifoundation_domain/config/fmwconfig/biee-domain.xml


OBIS Logs Setup Enterprise Manager (EM)


OBIS – Log inspection

• Line/Hash value after “SQL Request” denotes logical hash value for the query-------------------- SQL Request, logical request hash:42909950SET VARIABLE LOGLEVEL=7:SELECT, "Fact - Sales Invoice Lines"."Gross Sales” FROM "Sales …”

•Message to check if query is cacheable or not:The logical query block fail to hits or seed the cache in subrequest level due to session variable or config file disable subrequest cache

•Use Query log to verify cache hit:----------------- The logical query hits the plan cache, plan



•Find NQServer Execution plan (Do not confuse with SQL execution plan): ------------------- Execution plan:

•ADF query executed in bypass mode:<ADFQuery mode="SQLBypass" queryid="26500-19169" locale="en">SQLBypass mode is skipped if a transient attribute is being used in query

•Physical (Database) SQL Request:Physical query follows statement where query is being sent to physical datasource, “Oracle Data Warehouse Connection“ in this case. Logical and Physical hash values are highlighted Sending query to database named Oracle Data Warehouse (id: <<270614>>), connection pool named Oracle Data Warehouse Connection Pool, logical request hash 42909950, physical request hash 8245d55b:



•Query StatusQuery Status: Successful Completion

•Find number of rows returned by DB to NQServer:Execution Node: <<27109>> DbGateway Exchange, Close Row Count = 11, Row Width = 64 bytes

•Find response time for physical query execution:Physical query response time 1.898 (seconds), id <<270614>>


OBIS – Log inspection•Find summary stats for physical and logical query execution:Physical Query Summary Stats: Number of physical queries 1, Cumulative time 1.905, DB-connect time 0.000 (seconds)Logical Query Summary Stats: Elapsed time 2.305, Response time 2.304, Compilation time 0.398 (seconds), Logical hash 42909950

•Tie physical sql from nqquery.log to ORDBMS using physical hash value•Physical hash value is propagated to ORDBMS during query execution (it gets stored in action.v$sql)

select * from v$sql where action = 'c8dc78e8‘


Configurations for better performance inspection•Parallel init block execution

NUM_INIT_BLOCK_THREADS_PER_USER

•Parallel ADF VO query fetchingADF_SQL_BYPASS_THREAD_RANGE

Recommend to set to same value as DB_GATEWAY_THREAD_RANGE.

•Based on your workload, thread pools needs to be adjusted to capitalize on these features.

•Limit login timeout (in seconds)MAX_AUTHENTICATION_TIME = 120 INIT_BLOCK_LOG_TIME_THRESHOLD = 60


Program Agenda

Best practices


Caching

Case studies

Q&A

1

2

3

4

5


What Makes Queries Run Slowly?Problem

• Scanning too many records on the disk

• Returning too many records over the network

• Interface & processing bottlenecks

Causes• Large table

• Complex transformations

• Too many joins/ poor SQL

• Connection pool settings

• Logging

• Configuration settings

• Insufficient hardware

• Unsupported DB function

• Cross-DB join of large table

• Poor prompt design/ poor SQL


Why Aggregates?• Pros– Faster query response time usually by several order of magnitude– Efficient resource utilization– Aggregates can be in the same database as the source– Aggregates can be on a different relational source or Essbase

• Cons– Creating aggregates is a resource intense exercise– Cannot be used for real time data


Oracle BI EE “Architecture”

Tables

BI ServerSemantic Layer Metadata

SQL

RDBMS


Aggregates

RDBMS

DetailSummary aggregates

Semantic Layer Metadata

Metadata describes aggregate mappings

SQL SQ

L

At query time, BI Server queries the fastest source that has enough detail to satisfy the user request

Load time aggregation

Performance of highly summarized requests is dramatically improved

BI Server

Aggregates can reside in a different database


Steps using Admin ToolDesign aggregate facts and grains

Wizard to create metadata

1 Target database for create/load3

2


Program Agenda

Best practices


Caching

Case studies

Q&A

1

2

3

4

5


Query Plan Cache

•Saves CPU processing time by avoiding recompiling SQLMAX_QUERY_PLAN_CACHE_ENTRIES # recommended value 1024MAX_QUERY_PLAN_CACHE_ENTRY_SIZE # recommended 1 MB•Does not persist during server restart•Check query logs if query is being seeded in plan cache• The logical query seeds the plan cache•Check available RAM on the server and adjust above two parameters for your workload•Check compilation time from query logs

Logical Query Summary Stats: Elapsed time 9.958, Response time 9.958, Compilation time 0.956


Result Cache•Logical query results are cached by the server•Parameters to control number of rows per cache as well as number of cache entries allowed

[CACHE]ENABLE = NO;MAX_ROWS_PER_CACHE_ENTRY = 100000; MAX_CACHE_ENTRY_SIZE = 20 MB; MAX_CACHE_ENTRIES = 1000;

•Performance boost for repeating queries• Shared dashboards normally get high hit rates• Ad hoc and highly dynamic use cases have high miss rate• Not a substitute for database tuning and aggregation•Cached results are automatically shared by users with similar security


Result Cache

•Table level control (i.e., don’t cache real-time tables)•Use Cache Manager in Admin Tool (in online mode) to gather cache hit and miss number•You need to define seeding and purge strategy•Cache is persistent during server restart•May not be compatible from release to release, needs to be seeded after patching


•OBISAvailableDiagnostics();Lists all caches, result as well as plan cache

•OBISDiagnostics(‘$Name_of_cache_from_above_step')

Result cache statistics parameter list:•QUERIES PER SEC •FAILED QUERIES PER SEC •NEW PREPARES •ROWS PER SEC •KB PER SEC

Plan cache statistics parameter list: •CAPACITY •TOTAL REQUESTS •AVG REQUESTS •AVG HITS •AVG MISS

Cache – Statistics (programmatic)


Cache Hits•Caching is at the logical query layer, not physical layer•A query doesn’t need to be identical to get a hit•Cache results can be aggregated to satisfy a request•A more restrictive WHERE clause can hit•A subset of columns will hit•Expressions can hit if they use only cached columns•Cache entry currently contains• Database tables• Database Schemas • Connection pool parameters • Connection scripts • Physical Plan• Session variables (name/value) included in the plan

•Cache detector will not hit if the above variable values does not match for incoming query


Cache Strategy - Purge

• Purge to avoid stale data inconsistent with source• Event Polling Tables allow integration with ETL• Timed purges• Manual purges• Programmatic purges via SQL calls• Purge plan cache, nqSQL Statement: call sapurgeplancache();• Purge data cache, nqSQL Statement: call sapurgeallcache();


Cache Strategy - Seeding

• Seeding ensures good performance for the first users of the day• iBots can run standard dashboards to seed them• Run seed queries that are easy to hit (see Admin Guide)• Simple columns (no expressions)• No WHERE clause


Advanced Hit Detection• Multiple passes for more cache hits results in additional CPU usage• NQSConfig.INI Settings

USE_ADVANCED_HIT_DETECTION = YES;Consider the following 3 queriesQ1: SELECT Markets.Region, Markets.District, Markets.Market, Products.Brand,

Periods."Year", "Sales Measures".Dollars FROM Paint WHERE Periods."Year" IN ('1998', '1999')

Q2: SELECT Markets.District, Products.Brand, Periods."Year", "Sales Measures".DollarsFROM Paint WHERE Periods."Year" = '2000'

Q3:SELECT Markets.District, Products.Brand, Periods."Year", "Sales Measures".DollarsFROM Paint WHERE Periods."Year" = '1998‘

• Q1 is cached; Q3 gets a cache-hit from Q1 with aggregate-rewrite;• Q1 and Q2 is cached; Q3 now has a cache miss because exact-match algorithm doesn’t match Q2 ; • Can be fixed with advanced cache hit detection with multi-pass algorithm


Program Agenda

Best practices


Caching

Case studies

Q&A

1

2

3

4

5


Column size mismatch•Inefficient memory utilization• Ensure that data model is efficient. • Email column taking 2,000 characters is not obvious (10 such columns)• Alternatively, modify RPD by creating opaque table (not the desired option)


Use Init Block judiciously•Issue: BI Server memory is too high (40GB) in few minutes when large number of logins would occur•Cause:• Each login was invoking a init block which would fetch 68,000 rows adding up to 38 MB• OBIS memory would not come down after users have logged out or timed out• Large memory usage by OBIS was depriving other processes of memory. Only option to

reclaim memory was to restart OBIS•Fix:• Enable caching for row-wise init block such that it gets executed only once for the life of

the server. Cache lasts for the life of the server and cannot be refreshed like repository init block.• OBIS periodically calls MACompact() to release unwanted memory• Log Init Block query upon execution including its name• Consider: Cache lasts for the life of the server and cannot be periodically refreshed like

repository Init Block.


Use Init Block judiciously


Nested REPLACE•Nested replace statement results in padding large amount of data since for each replace it would allocate maximum size (1500 characters in this case)• Query returns 200+ MB for 3 rows• Projection column is VARCHAR(1500)• For each replace statement padded size grows exponentially

•This would occur with other concatenated functions like concat•‘cast’ the nested replace :• REPLACE(REPLACE(REPLACE(REPLACE("Sales - CRM Interactions and Customers Real Time"."Interaction

Extension"."Extension Attribute Character 006",'U','Undecided'),'S','Successful'),'N','Not-Successful'),'I','Incomplete')

• CAST(REPLACE(REPLACE(REPLACE(REPLACE("Sales - CRM Interactions and Customers Real Time"."Interaction Extension"."Extension Attribute Character 006",'U','Undecided'),'S','Successful'),'N','Not-Successful'),'I','Incomplete') AS VARCHAR(1500))

•Parameter OBIS_MAX_FIELD_SIZE was introduced to govern the field size (default: 32K)


Setting hints•Hints that you intend to pass to the end datasource (ORDBMS) can be set in RPD at connection or query level:• Execute on connect / disconnect• Execute before / after query•Upon disconnect if statement errors, connection is damaged and not returned to pool and counter not decremented•Eventually connections will be saturated and system will be frozen since no connections are available•Has been fixed


Setting hints


Table / VO Pruning•Use correct cardinality, 1:N results in more efficient SQL generation compared to (0,1):N

• Tables used in query with (0,1):N cardinalityPER_LOCATION_DETAILS_F_TL PER_LOCATION_DETAILS_F_VL POR_REQUISITION_LINES_ALL POR_REQUISITION_HEADERS_ALL PO_LINE_LOCATIONS_ALL PO_VERSIONS_INIT_SEQUENCE_V FUN_ALL_BUSINESS_UNITS_V GL_LEDGERS FND_CURRENCIES_B FND_CURRENCIES_TL


Table / VO Pruning


Table/ VO Pruning•Unnecessary VO’s being used (VO join pruning not occurring)


Filter rewriteOld style logical SQL generationSELECTIDOF("BIQA_AW_REL"."C1Channel"."Primary"."All Channels"),IDOF("BIQA_AW_REL"."C1 Product"."Standard"."Total"), "BIQA_AW_REL"."C1Channel"."All Channels","BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Sales", "BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Units"FROM "BIQA_AW_REL"UNION ALLSELECTCAST(NULL AS INTEGER),IDOF("BIQA_AW_REL"."C1 Product"."Standard"."Total"), CAST(NULL AS INTEGER),"BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Sales", "BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Units"FROM "BIQA_AW_REL"WHEREIDOF("BIQA_AW_REL"."C1Channel"."Primary"."Channel") IN (2, 4)UNION ALLSELECTCAST(NULL AS INTEGER),IDOF("BIQA_AW_REL"."C1 Product"."Standard"."Total"), CAST(NULL AS INTEGER),"BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Sales", "BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Units"FROM "BIQA_AW_REL"WHEREIDOF("BIQA_AW_REL"."C1Channel"."Primary"."Channel") IN (3, 4)ORDER BY 1

New style physical SQL generation (after filter rewrite)

SELECTIDOF("BIQA_AW_REL"."C1Channel"."Primary"."All Channels"),IDOF("BIQA_AW_REL"."C1 Product"."Standard"."Total"), "BIQA_AW_REL"."C1Channel"."All Channels","BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Sales", "BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Units", FILTER("BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Sales" USING IDOF("BIQA_AW_REL"."C1Channel"."Primary"."Channel") IN (2, 4)) as "SalesCG1", FILTER("BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Units" USING IDOF("BIQA_AW_REL"."C1Channel"."Primary"."Channel") IN (2, 4)) as "UnitsCG1", FILTER("BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Sales" USING IDOF("BIQA_AW_REL"."C1Channel"."Primary"."Channel") IN (3, 4)) as "SalesCG2", FILTER("BIQA_AW_REL"."C1Cube Units Sales Measures"."C1 Units" USING IDOF("BIQA_AW_REL"."C1Channel"."Primary"."Channel") IN (3, 4)) as "UnitsCG2"FROM "BIQA_AW_REL"ORDER BY 1


Filter rewriteWITH SAWITH0 AS (select sum(T7209.UNITS) as c1, sum(T7209.SALES) as c2, T6641.TOTAL_CHANNEL_ID as c3, T6716.TOTAL_CUSTOMER_ID as c4, T7121.TOTAL_PRODUCT_ID as c5from PRODUCT_DIM T7121, CHANNEL_DIM T6641, CUSTOMER_DIM T6716,

SALES_FACT T7209where ( T6641.CHANNEL_ID = T7209.CHANNEL_ID and T6716.SHIP_TO_ID = T7209.SHIP_TO_ID and

T7121.ITEM_ID = T7209.ITEM_ID ) group by T6641.TOTAL_CHANNEL_ID, T6716.TOTAL_CUSTOMER_ID, T7121.TOTAL_PRODUCT_ID),SAWITH1 AS (select sum(T7209.UNITS) as c1, sum(T7209.SALES) as c2, T6716.TOTAL_CUSTOMER_ID as c3, T7121.TOTAL_PRODUCT_ID as c4from PRODUCT_DIM T7121, CHANNEL_DIM T6641, CUSTOMER_DIM T6716,


T7121.ITEM_ID = T7209.ITEM_ID and (T6641.CHANNEL_ID in (2, 4)) and (T7209.CHANNEL_ID in (2, 4)) ) group by T6716.TOTAL_CUSTOMER_ID, T7121.TOTAL_PRODUCT_ID),SAWITH2 AS (select sum(T7209.UNITS) as c1, sum(T7209.SALES) as c2, T6716.TOTAL_CUSTOMER_ID as c3, T7121.TOTAL_PRODUCT_ID as c4from PRODUCT_DIM T7121, CHANNEL_DIM T6641, CUSTOMER_DIM T6716,


T7121.ITEM_ID = T7209.ITEM_ID and (T6641.CHANNEL_ID in (3, 4)) and (T7209.CHANNEL_ID in (3, 4)) ) group by T6716.TOTAL_CUSTOMER_ID, T7121.TOTAL_PRODUCT_ID),SAWITH3 AS (((select distinct 0 as c1,from … SAWITH3 D1order by c1, c6, c7, c8

More efficient sql, execution results in single pass on fact tableWITH SAWITH0 AS (select sum(T7209.UNITS) as c1, sum(T7209.SALES) as c2, sum(case when T6641.CHANNEL_ID in (2, 4) then T7209.SALES end ) as c3, sum(case when T6641.CHANNEL_ID in (2, 4) then T7209.UNITS end ) as c4, sum(case when T6641.CHANNEL_ID in (3, 4) then T7209.SALES end ) as c5, sum(case when T6641.CHANNEL_ID in (3, 4) then T7209.UNITS end ) as c6, T6641.TOTAL_CHANNEL_ID as c7, T6716.TOTAL_CUSTOMER_ID as c8, T7121.TOTAL_PRODUCT_ID as c9from PRODUCT_DIM T7121, CHANNEL_DIM T6641, CUSTOMER_DIM T6716,

SALES_FACT T7209where ( T6641.CHANNEL_ID = T7209.CHANNEL_ID and T6716.SHIP_TO_ID = T7209.SHIP_TO_ID and T7121.ITEM_ID = T7209.ITEM_ID ) group by T6641.TOTAL_CHANNEL_ID, T6716.TOTAL_CUSTOMER_ID, T7121.TOTAL_PRODUCT_ID)select distinct 0 as c1, …from SAWITH0 D1order by c1, c2, c4, c5


Filter rewriteConvert a physical column CASE statement into a logical column definition based on filter function

Logical Query:SELECT Time.Quarter saw_0, Time.Week saw_1, rsum("Fact - RMW Opportunity"."Expected Revenue (000)") saw_3 FROM "Sales - Sales Revenue and Pipeline" WHERE (Time.Quarter = VALUEOF(CURRENT_QUARTER)) ORDER BY saw_0, saw_1 Old Physical Query:select T264047."PER_NAME_QTR" as c1, T264047."PER_NAME_WEEK" as c2, sum(case when T346644."LOST_FLG" = 'N' and T346644."SUMMARY_FLG" = 'Y' then T346644."REVN" end ) as c5from "W_DAY_D" T264047, "W_REVN_F" T346644where ( T264047."ROW_WID" = T346644."CLOSE_DT_WID" and T264047."PER_NAME_QTR" = '2004 Q 2' ) group by T264047."PER_NAME_QTR", T264047."PER_NAME_WEEK“ order by c1, c2 New Physical Query:select T264047."PER_NAME_QTR" as c1, T264047."PER_NAME_WEEK" as c2, sum(T346644."REVN") as c5from "W_DAY_D" T264047, "W_REVN_F" T346644where ( T346644."LOST_FLG" = 'N' and T264047."PER_NAME_QTR" = '2004 Q 2' and T264047."ROW_WID" = T346644."CLOSE_DT_WID" and T346644."SUMMARY_FLG" = 'Y' ) group by T264047."PER_NAME_QTR", T264047."PER_NAME_WEEK“ order by c1, c2


Missing descriptor id column •Inefficient SQL Plan due to missing Descriptor Id Column when using lookup–With OBIS cache ON, queries which include lookup do not perform because

the plan generated by nqsserver is suboptimal. –OBIS executes each lookup query and then stitches it together. –For the query in question, we saw RT of 18+ sec with 7K+ rows being fetched

for each lookup. –Fix is to specify "Descriptor ID Column" for the lookup. Query generation

engine in that case creates a better plan by sending the complete query to database and RT is reduced to 7 sec.


Missing descriptor id column


Program Agenda

Best practices


Caching

Case studies

Q&A

1

2

3

4

5

OBIEE Performance and RPD Optimization techniques Deb Bhatacharjee Sr. Director Anupam Bordia...

Documents

Transcript of OBIEE Performance and RPD Optimization techniques Deb Bhatacharjee Sr. Director Anupam Bordia...