UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : [email protected] W: www.rittmanmead.com

Mark Rittman, Technical Director, Rittman MeadUKOUG Engineered Systems Summit, London April 2013

Oracle Exalytics - Tips and Tricks from the Field

mailto:[email protected]


Mark Rittman

•Mark Rittman, Co-Founder of Rittman Mead•Oracle ACE Director, specialising in Oracle BI&DW•14 Years Experience with Oracle Technology•Regular columnist for Oracle Magazine•Author of two Oracle Press Oracle BI books

‣Oracle Business Intelligence Developers Guide‣Oracle Exalytics Revealed

•Writer for Rittman Mead Blog :http://www.rittmanmead.com/blog

•Email : [email protected]•Twitter : @markrittman


http://www.rittmanmead.com/blog



About Rittman Mead

•Oracle BI and DW gold partner•World leading specialist partner for technical excellence, solutions delivery and

innovation in Oracle BI•Approximately 30 consultants worldwide•All expert in Oracle BI and DW•UK based•Offices in US, Europe (Belgium) and India•Skills in broad range of supporting Oracle tools:

‣OBIEE‣OBIA‣ODIEE‣Essbase, Oracle OLAP‣GoldenGate‣Exadata



Rittman Mead in the Developer/User Community



Agenda

•What is Oracle Exalytics (stop me if you’ve heard this before...)•How Exalytics works (again, stop me...)•Exalytics Best Practices, and Experiences from the Field, including...

‣Installation, Configuration for Multiple Environments, Management‣Choosing What Goes in the Adaptive Data Mart ‣Where Does Exalytics Add Value?‣Measuring Exalytics Performance and Metrics‣Maintaining and Refreshing the Adaptive Data Mart



What is Oracle Exalytics?

•Hardware Element‣Sun Fire X4470 M2 server‣1TB RAM, 40 Cores, 3.6TB HDD

•Software Element‣OBIEE 11.1.1.6 with Exalytics Enhancements‣Oracle Essbase 11.1.2 with Exalytics Enhancements‣Oracle TimesTen 11.2.2.2 for Exalytics‣New in v1.1 : Support for Oracle Endeca

Information Discovery, Golden Gate, ODI‣Runs on 64-bit Oracle Linux

(Exalogic distribution)•OBIEE and Essbase are licensed as

Oracle BI Foundation•Exalytics features can only be used in

conjunction with Exalytics hardware



Rittman Mead and Oracle Exalytics

•Rittman Mead are the first Oracle Partner in the UK to buy an Exalytics box•Ordered February 2012 through Arrow (reseller), delivered end-April 2012•Hosted in a secure datacenter in London Docklands, access via VPN•To be used for customer PoCs, training, testing and possible cloud hosting



Exalytics as the Exa-Machine for BI

•Runs the BI layer on a high-performance, multi-core, 1TB server•In-memory cache used to accelerate the BI part of the stack•If Exadata addresses 80% of the query performance,

Exalytics addresses the remaining 20%‣Consistent response times for queries‣In-memory caching of aggregates‣40 cores for high concurrency‣Re-engineered BI and OLAP software

that assumes 40 cores and 1TB RAM

ERP/Apps DW

Oracle BI

In-Memory DB/Cache



Some Example Exalytics Use-Cases

•Customers who do not have the time/ability/means to put in-place an aggregationstrategy for their dashboards, or who understand how labour-intensive this is

•Customers who have done what they can with summaries, indexes etc, but needthat extra 10% or so of performance, particularly around response-time consistency

•Customers who want to create rich, visual dashboards that would otherwisestretch the BI Presentation Server

•Customers who need to support high numbers of concurrent users•Customers running Hyperion Planning and Budgeting who want to reduce the planning

time cycle, re-calculate data faster and generally do more in less time•Customers who are moving to a single supplier, engineered-systems hardware

strategy, using Exadata for their databases, Exalogic for their Java application hosting, and now Exalytics for their BI

•Customers looking to consolidate multiple BI systems into a single large instance



Exalytics Sweet-Spots, in our Experience...

•Customers with BI systems that use an underlying data warehouse, withbatch-load refresh cycles that permit pre-aggregation of data

•Customers who have a large number of users running the same reports and dashboards as each other

•Customer with BI systems that aggregate data via predefined hierarchies•Customers with reasonably simple and well-tested / validated RPDs•Customers using Essbase for planning/budgeting applications, and who want to

move the cube into memory•Customers moving from v10 to v11 OBIEE, with little in the way of visual, interactions

etc, and use Exalytics as the platform for delivering these



Exalytics Under the Covers - How Does it Work?

• Exalytics brings together different technologies, which are still standalone products in their own right

• To harmonise and optimise their use within Exalytics, it utilises the following techniques:‣ In-Memory Adaptive Data Mart‣ In-Memory Intelligent Result Cache‣ In-Memory Cubes

• Some of these are genuine "secret sauce"‣ New functionality and algorithms‣ You can only get them through licensing Exalytics

• Others are descriptions of DW/BI strategies, or existing product functionality, extended to take advantage of the capacity for processing in memory that Exalytics has



In-Memory Adaptive Data Mart

•Commonly-used aggregates are copied into Oracle TimesTen for Exalytics•Past query patterns are analyzed and suitable aggregates recommended•Oracle BI Server then uses these aggregates to make queries run faster•Aggregates change over time in

response to changes in query patterns•Tools are provided for managing

and populating these aggregates

TimesTen BI Server

Exa

lyti

cs

Aggregates

Data Warehouse

Detail-levelData



Summary Advisor for Aggregate Recommendation & Creation

•Utility within Oracle BI Administrator tool that recommends aggregates•Bases recommendations on usage tracking and summary statistics data•Captured based on past activity•Runs an iterative algorithm that searches,

each iteration, for the best aggregate



Oracle Exalytics Best Practices, and Experiences from the Field

•Installation, Configuration for Multiple Environments, Management•Choosing What Goes in the Adaptive Data Mart •Where Does Exalytics Add Value?•Measuring Exalytics Performance and Metrics•Maintaining and Refreshing the Adaptive Data Mart



Configuring Multiple Environments (Consolidation)

•Exalytics main value proposition is around performance‣In-memory aggregates speeding up query performance

•However many customers wish to use Exalytics for consolidation‣Run multiple environments (test, dev, prod) on a single server‣Consolidate multiple applications on a single server

•Desire is usually down to cost (multiple physical servers, per-processor licensing)

DEV DEV TEST PROD

vs.

MWHOME1

DEV

MWHOME2

DEV

MWHOME3 MWHOME1 MWHOME1 MWHOME1

Single Exalytics Server Exalytics Svr 1 Exalytics Svr 2 Exalytics Svr 3



Best Practice for Consolidation : Oracle Virtual Machine

•Exalytics is now certified for use with Oracle Virtual Machine, Oracle’s hard-partitioning virtualization platform

•OVM is installed first, on Exalytics “bare metal”, with OVM Repository on external DB server (as with BIPLATFORM, MDS)

•Exalytics VM templates then used to create required environments (dev, test etc)

•Also allows for licensing in smaller per-processor license blocks (5, 10, 15 etc)

•OVM provides for resource caging, isolating environments from each other

DEV

MWHOME1

Single Exalytics Server

OVM Hypervisor (Oracle VM Server)

OVM Guest 1

DEV

MWHOME1

OVM Guest 2

DEV

MWHOME1

OVM Guest 3

Oracle VMManager

Oracle VMRepository



High Availability using Clustered Exalytics Servers

•Multiple Exalytics servers can be clustered for HA and performance (load balancing)‣Requires hardware load balancer above clustered servers

•Linked together via InfiniBand (one for clustering, one for connection to Exadata)•Note: does not double available RAM

- aggs are replicated in each TT database‣Note that Summary Advisor / nqcmd

does not replicate agg tables -manual replication required

•HA is between TT instances, and OBIEEmanaged server + system components



What Should the TimesTen Adaptive Data Mart be Used For?

•Standard approach is to store aggregates in the TimesTen datamart‣Aggregated by the source DB, aggregates then cached in TT database

•Other approaches could be used, however‣Store whole detail-level dataset in the TT database‣Store just recent detail-level data in TT, and use OBIEE’s fragmentation feature‣Use Essbase to hold the aggregates instead‣Use Essbase to hold detail-level + aggregates

•Where does Exalytics add value? •What should go in the TimesTen database?



Conclusions and Best Practices Around Detail/Aggregate Storage

•Best use of OBIEE + TT is how the product was intended to be used‣Use TimesTen for storing (caching) pre-created aggregates‣Aggregates created by source DB (Oracle, SQL Server etc)‣Aggregation not performed by TimesTen (as it would be if it held detail-level data)

•IMDB interesting approach, but relies on TT performing aggregation (slow)•Same goes for holding whole detail-level source in TimesTen•Also, bear in mind that majority of performance benefit may come from re-hosting

OBIEE application on the Exalytics hardware‣Particularly with Exadata sources, source DB may even be fastest host for aggs‣Test for your data, do not pre-suppose outcome



Refreshing Aggregates Within the TimesTen Data Mart

•Aggregates are built in TimesTen to support sub-second response times•Summary Advisor tool suggests aggregate dimensionality and grain, generates script•BI Server’s Aggregate Persistence executes script:

1. Create TimesTen aggregate table2. Populate TimesTen aggregate3. Update RPD online with new aggregate metadata

•Handles supporting dimensions too

Base data Aggregates

TimesTen

OBIEE Aggregate Persistence

RPD

Aggregate tables created

RPD updated with new aggregate mappings

Aggregate tables loaded from base data



Out-of-the-Box Summary Advisor Limitations

•To refresh an aggregate, Summary Advisor deletes and rebuilds from scratch•The RPD is edited directly on the BI Server each time an aggregate is created or rebuilt•Build failures can be difficult to debug•If a rebuild fails it can leave the RPD in an inconsistent state with TimesTen

After a failed build, the aggregates are still in the RPD, but no longer exist in TimesTen

nqquery.log suggests possible errors but no clear root cause

This is that the error the user sees



Alternatives to using Aggregate Persistence

•However aggregates are refreshed, they must be included in the RPD‣Can be done manually‣Aggregate Persistence is useful for this

•Complete refresh of aggregate data‣Extract the SQL that OBIEE generates in Aggregate

Persistence, run this through ODI‣Write bespoke aggregate refresh code in ODI

•Incremental refresh using GoldenGate and ODI‣Instead of rebuilding aggregates in their entirety each

time, only update the part of the aggregate that has changed on the base data



Incremental refresh of aggregates using GoldenGate and ODI

•Changes to base data are replicated by GoldenGate onto Staging database•ODI Interfaces apply deltas to the TimesTen aggregates

Aggregates

TimesTen

CDC staging data

Oracle

Base data

(any supported DB)

GoldenGate ODI

ODI agentGoldenGate GoldenGate

TimesTen JDBC client



Custom BI Server ETL Option : Incremental BI Server Refresh

•Uses BI Server to do the refresh, but invokes just part of it - the data refresh part•Avoids unnecessary aggregate table drop/rebuild, and online RPD edit•Uses BI Server logical ETL SQL features

to just refresh the TT aggregate table, using only the latest set of detail-level data‣Avoids use of additional ETL tools‣No additional license cost‣Works against any BI Server source



Custom BI Server ETL Option : Incremental BI Server Refresh

•Uses a number of “undocumented” but support BI Server ETL features‣POPULATE command for loading data into an RPD table‣WHERE clause for limiting the refresh to just the incremental load‣INACTIVE_SCHEMAS to stop BI Server refreshing using existing RPD agg table

•See http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics-aggregates-using-native-bi-server-capabilities/ for full details including scripts etc

SET VARIABLE DISABLE_CACHE_HIT=1, DISABLE_CACHE_SEED=1, DISABLE_SUMMARY_STATS_LOGGING=1, INACTIVE_SCHEMAS='"TimesTen Aggregates".."EXALYTICS"'; populate "ag_sales_month" mode ( append table connection pool "TimesTen aggregates"."TT_CP") as select_business_model "Sales"."Fact Sales"."Sale Amount" as "Sale_Amoun000000AD","Sales"."Dim Times"."Month YYYYMM" as "Month_YYYY000000D0" from "Sales" where "Dim Times"."Month YYYYMM" = VALUEOF("THIS_MONTH");


http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics-aggregates-using-native-bi-server-capabilities/

http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics-aggregates-using-native-bi-server-capabilities/


More Information

•Thank you for attending, & more info can be found at http://www.rittmanmead.com•Contact us at [email protected] or [email protected]•Look out for our books, “Oracle Business Intelligence Developers Guide” and

“Oracle Exalytics Revealed”•Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead)


http://www.rittmanmead.com/




Mark Rittman, Technical Director, Rittman MeadUKOUG Engineered Systems Summit, London April 2013

Oracle Exalytics - Tips and Tricks from the Field

UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field

Technology

Transcript of UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field