Oracle Exalytics - Tips and Experiences from the Field (Enkitec E4 Conference 2013, Dallas)

T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com

Oracle Exalytics - Tips and Experiences from the FieldMark Rittman & Stewart Bryson, Rittman MeadEnkitec Extreme Exadata Expo, Texas, August 2013


Sunday, 4 August 13

https://www.enkitec.com/apex/f?p=15000:3010:0::::P3010_SEARCH:Oracle%20Exalytics%20-%20Tips%20and%20Experiences%20from%20the%20Field




mailto:[email protected]


http://www.rittmanmead.com



Mark Rittman

• Mark Rittman, Co-Founder of Rittman Mead• Oracle ACE Director, specialising in Oracle BI&DW• 14 Years Experience with Oracle Technology• Regular columnist for Oracle Magazine• Author of two Oracle Press Oracle BI books• Oracle Business Intelligence Developers Guide• Oracle Exalytics Revealed• Writer for Rittman Mead Blog :

http://www.rittmanmead.com/blog• Email : [email protected]• Twitter : @markrittman

Sunday, 4 August 13





http://www.rittmanmead.com/blog




T : +1 (888) 631-1410 E : [email protected] W: www.rittmanmead.com

Stewart Bryson

• Twitter : @StewartBryson

• Oracle ACE in BI/DW

• Oracle BI/DW Architect and Delivery Specialist

• Community Speaker and Enthusiast

• Writer for Rittman Mead Blog:http://www.rittmanmead.com/blog

• US Conference Chair of the Rittman Mead BI Forum

• Developer of Transcend Framework

• Email : [email protected]

• Real Time BI with Kevin & Stewart‣ iTunes: http://bit.ly/realtimebi‣ YouTube: http://www.youtube.com/user/realtimebi

Sunday, 4 August 13





http://bit.ly/realtimebi

http://bit.ly/realtimebi

http://www.youtube.com/user/realtimebi

http://www.youtube.com/user/realtimebi


About Rittman Mead

• Oracle BI and DW platinum partner• World leading specialist partner for technical excellence, solutions delivery and innovation in Oracle BI• Approximately 50 consultants worldwide• All expert in Oracle BI and DW• Offices in US (Atlanta), Europe, Australia and India• Skills in broad range of supporting Oracle tools:‣OBIEE‣OBIA‣ODIEE‣Essbase, Oracle OLAP‣GoldenGate‣Exadata ‣Endeca

Sunday, 4 August 13






Main Exalytics Proposition

• In-Memory analytics - lightening-fast response, free-form analysis and aggregation• Rich, immersive dashboards powered by high-spec hardware• Extra OBIEE + other features only available on this platform• Enables fast development controlled by the business• Faster planning and budgeting

Sunday, 4 August 13






Oracle Exalytics In-Memory Machine

• Engineered system, complements Oracle Exadata Database Machine (though can work standalone)• Combination of high-end hardware (Sun x86_64 architecture, 3RU rack-mountable, 1-2TB RAM)

and optimized versions of Oracle’s BI, In-Memory Database and OLAP software• Delivers “in-memory analytics” focusing on analysis, aggregation and UI‣Rich, interactive dashboards with split-second response times‣ 1-2TB of RAM, to run your analysis in-memory‣ Infiniband connection to Exadata and Oracle Big Data Appliance‣ 40 CPU cores to support high-levels of user concurrency‣ Lower TCO through known configuration, combined patch sets‣Contains software features only licensable through

Exalytics package

Sunday, 4 August 13






Exalytics as the Exa-Machine for OBIEE

• Runs the BI layer on a high-performance, multi-core, 1-2TB server• In-memory cache used to accelerate the BI part of the stack• If Exadata addresses 80% of the query performance, Exalytics addresses

the remaining 20%‣Consistent response times for queries‣ In-memory caching of aggregates‣ 40 cores for high concurrency‣Re-engineered BI and OLAP software

that assumes 40 cores and 1TB RAM

ERP/Apps DW

Oracle BI

In-Memory DB/Cache

Sunday, 4 August 13






Also Supports Essbase, and Endeca Information Discovery

• In-Memory Essbase for planning, budgeting and sales analysis-style OLAP applications• Endeca Information Discovery for search/analytic applications against diverse data

In-Memory Cache

Essbase Planning Engine

Smart Storage Manager

Lock Manager

Unified Indexing

Data Mashup

Text Analysis

Unified Search

Faceted Navigation

Interactive Exploration

Information Discovery

OracleExalytics

In-Memory Machine

Sunday, 4 August 13






Exalytics history - software

Oct 11 Feb 12 Aug Sep Jan 13 Apr Jul

Exaly&cs announced at OOW2011

v1.0.0.0

OBI 11.1.1.5Essbase 11.1.2.2.0 TimesTen 11.2.2.2.1

PS1 (v1.0.0.1)OBI 11.1.1.6.2 BP1Essbase 11.1.2.2.100TimesTen 11.2.2.3.0

Cer&fied:Golden Gate 11.1.1ODI 11.1.1.5+Endeca 2.3OBIA 7.9.6.4

Exaly&cs available to license through Trusted Par22ons on OVM

PS2 (v1.0.0.2)OBI 11.1.1.6.5+Essbase 11.1.2.2.101+TimesTen 11.2.2.4.1+

Cer&fied: Endeca 3.0

PS3 (v1.0.0.3)Essbase 11.1.2.2.200OBIEE 11.1.1.7.0+TimesTen 11.2.2.5

Cer&fied:Essbase 11.1.2.3OBIA 11.1.1.7.1

Exaly<cs X2-‐4/V1 becomes GA

Hardware upgrade, new OBI features,

support for OBIA 11g

Non-‐core so7ware such as ODI and Golden Gate now

cer<fied

So7ware support updates

Flash PCIe added

Sunday, 4 August 13






Exalytics history - hardware

Oct 11 Feb 12 Aug Sep Jan 13 Apr Jul

Exaly&cs announced at OOW2011

X2-‐4• 40 cores (4x E7-‐4800)• 1TB RAM• 3.6TB raw disk (6x SAS-‐2 600GB)• Two 40Gb/s InfiniBand ports• Four 10/100/1000Base-‐T on-‐board Ethernet ports

Exaly&cs available to license through Trusted Par22ons on OVM

X2-‐4 Flash Upgrade Kit

Adds 2.4TB of Flash storage to exis&ng X2-‐4 machines, with 6x Sun F40 400GB eMLC Flash PCIe cards

X3-‐4• 40 cores (4x E7-‐4800)• 2TB RAM• 2.4TB eMLC Flash PCIe• 5.4TB raw disk (6x SAS-‐2 900GB)• Two 40Gb/s InfiniBand ports• Four 10/100/1000Base-‐T on-‐board Ethernet ports

X2-‐4 Memory and Flash Upgrade KitAdds 2.4TB Flash PCIe and addi&onal 1TB RAM to exis&ng X2-‐4 machines

Exaly<cs X2-‐4/V1 becomes GA

Hardware upgrade, new OBI features,

support for OBIA 11g

Non-‐core so7ware such as ODI and Golden Gate now

cer<fied

So7ware support updates

Flash PCIe added

Sunday, 4 August 13






Rittman Mead and Oracle Exalytics

• Rittman Mead were the first UK Oracle Partner to purchase an Exalytics server, back in Q1 2012• Intention was to use it for customer PoCs, internal and external training, R&D• Had additional support from the Oracle product development team as an “early adopter”• Run several customer PoCs since then, independently & with Oracle• Developed Exalytics training material• Written “Oracle Exalytics Revealed” ebook for Oracle Press• Most importantly - springboard for several customer projects‣UK Retailer‣US Pharmaceuticals Company‣UK/Worldwide Broker + Financial Information Company‣US / US / Asia Financial Asset Management‣UK National Health Service Hospital‣ etc

Sunday, 4 August 13






Findings from the Field

• So what have we found, after over a year of working with, and implementing, Exalytics?• How well does it work as a query accelerator (“analysis at the speed of thought”)?• How else has it been used (in-memory data warehouse, for example)?• What role has the Essbase software played (and, Endeca?)• How have customers been deploying it?

Sunday, 4 August 13






Some Example Exalytics Use-Cases

• Customers who do not have the time/ability/means to put in-place an aggregationstrategy for their dashboards, or who understand how labour-intensive this is

• Customers who have done what they can with summaries, indexes etc, but needthat extra 10% or so of performance, particularly around response-time consistency

• Customers who want to create rich, visual dashboards that would otherwisestretch the BI Presentation Server

• Customers who need to support high numbers of concurrent users• Customers running Hyperion Planning and Budgeting who want to reduce the planning time cycle, re-

calculate data faster and generally do more in less time• Customers who are moving to a single supplier, engineered-systems hardware strategy, using Exadata for

their databases, Exalogic for their Java application hosting, and now Exalytics for their BI• Customers looking to consolidate multiple BI systems into a single large instance

Sunday, 4 August 13






Exalytics Sweet-Spots, in our Experience...

• Customers with BI systems that use an underlying data warehouse, withbatch-load refresh cycles that permit pre-aggregation of data

• Customers who have a large number of users running the same reports and dashboards as each other• Customer with BI systems that aggregate data via predefined hierarchies• Customers with reasonably simple and well-tested / validated RPDs• Customers using Essbase for planning/budgeting applications,

and who want to move the cube into memory• Customers moving from v10 to v11 OBIEE, with little in the way of

visual, interactions etc, and use Exalytics as the platform for delivering these

Sunday, 4 August 13






Exalytics as the Query Performance Enhancer

• In conjunction with a well-tuned source data warehouse, Exalytics adds an in-memory analysis later• Based around Oracle TimesTen for Exalytics, Oracle’s In-Memory Database• Aggregates are recommended based on query patterns, and automatically created in TimesTen• Summary Advisor makes recommendations, which adapt as queries change• Meant to be “plug-and-play” - no need for

expensive data warehouse tuning• So how does it work in-practice? TimesTen BI Server

Exal

ytic

s

Aggregates

Data WarehouseDetail-level

Data

Sunday, 4 August 13






Summary Advisor for Aggregate Recommendation & Creation

• Utility within Oracle BI Administrator tool that recommends aggregates• Bases recommendations on usage tracking and summary statistics data• Captured based on past activity• Runs an iterative algorithm that searches,

each iteration, for the best aggregate

Sunday, 4 August 13






Recommendations Based on Enhanced Usage Tracking Data

• Historically, usage tracking has been tracked using S_NQ_ACCT‣Holds basic usage tracking statistics + logical SQL query

• Now supplemented by S_NQ_DB_ACCT ‣Extra usage tracking information, includes physical SQL

• Exalytics Summary Advisor uses S_NQ_SUMMARY_ADVISOR‣Contains summary statistics, execution time etc‣Gathered at same time as usage tracking when

Exalytics is enabed‣Contents can be derived from usage tracking if needed

Sunday, 4 August 13






Oracle TimesTen for Exalytics

• New version of TimesTen specifically for Exalytics (and only available with Exalytics)• Support for analytic functions‣Perform all the processing at source‣Combine with being in-memory = should be very fast

• Column compression‣Whitepaper cites 5x - possible for dimension tables, usually much lower for fact tables (20% compression)‣Given the hardware capacity, we could seriously contemplate loading the whole Data Warehouse into

memory - see techniques and limitations later on‣Opens up lots of interesting design potential

• We can load aggregates into TimesTen, leave base data at source, and use OBIEE’s Vertical Federation capability to seamlessly report across both‣All hidden from the end-user, all they will know is that their reports run fast!

Sunday, 4 August 13






TimesTen and OBIEE Architecture

• Oracle BI Server communicates with TimesTen through TimesTen Client• Summary Advisor, and nqcmd use Oracle BI Server to access TimesTen• Typical single TimesTen database per Exalyics machine‣Max TimesTen database size around 300MB

- Due to need to set aside equalTemp size for the Perm size selected

• Clustered Exalytics boxes can be daisy-chainedtogether using InfinBand connections‣ For HA scenarios, does not increase available RAM‣Summary advisor scripts write to both TimesTen

databases, replicating aggregates‣ TimesTen databases can be “wired together”

for failover/HA purposes

TimesTenMemory-ResidentDatabase

CheckpointFiles

LogFiles

ODBC

Oracle BIServer

nqcmdSummaryAdvisor

Sunday, 4 August 13






Example 1: Using Exalytics to Pre-Aggregate and Cache BI Apps Data

• Can we use the Summary Advisor to automatically cache commonly-used aggregates in-memory?‣Similar to regular OBIEE caching, relies on query repeatability + use of aggregation

• Could we copy all, or part, of the BI Apps data warehouse directly into TimesTen?• How would we update the RPD to point to the in-memory tables?• How fast would TimesTen be to load, and to query, vs. Oracle/SQL Server/DB2 etc?• Here’s our thoughts and R&D to date....

Sunday, 4 August 13






Using the Summary Advisor on BI Apps 7.9.6.4 Step 1 : Start Advisor

• Server has to be an Exalytics server, in this example is patched-up to 11.1.1.6.9• Workstation has the 11.1.1.6.9 BI Administration tool installed• Select Tools > Utilities, then Oracle BI Summary Advisor from utility list

12

Sunday, 4 August 13






Using the Summary Advisor on BI Apps 7.9.6.4 Step 2 : Scope Source Queries

• By default, all queries registered in the usage tracking and summary statistics tables are in-scope• Refine the recommendations by limiting timeframe, and setting minimum accumulated time threshold• Still an opportunity later on to pick and choose from recommended aggregates• Once selected, then select the TimesTen connection pool and database as the aggregate table target

34

Sunday, 4 August 13






Using the Summary Advisor on BI Apps 7.9.6.4 Step 3 : Create Aggregates

• Summary Advisor then recommends a set of “candidate” aggregates, which you can choose to implement• Select all, none or some of the recommended aggregates• Then run the resulting logical SQL script using the nqcmd utility• Note - may need to clean-up BI Apps DW data to remove duplicates etc before script completes OK

Sunday, 4 August 13






Using the Summary Advisor on BI Apps 7.9.6.4 Step 4 : Review RPD and Data

• Aggregate Persistence process called by the “create aggregates” process also maps tables in RPD• Physical layer contains entries for the TimesTen tables• Business Model and Mapping later contains vertically-federated LTSs for the new TT tables

Sunday, 4 August 13






Summary Advisor for BI Apps : Pros and Cons

• Pros‣Automatically analyzes query patterns and recommends aggregates to accelerate them‣Handles the registering of the TimesTen tables in the RPD, including mapping back into business model‣Supports any data source that the BI Server supports

• Cons‣Queries have to have run before they’ll be considered for loading

into TimesTen for Exalytics‣Relies on subsequent queries being able to use those aggregates‣Could get unwieldy if many aggregates are registered in the RPD‣Summary Advisor process does not automatically clear down

tables that don’t feature in future recommendations‣Refresh process for aggregates is inefficient

Sunday, 4 August 13






Refreshing Summary Advisor Aggregates Within the TimesTen Data Mart

• Aggregates are built in TimesTen to support sub-second response times• Summary Advisor tool suggests aggregate dimensionality and grain, generates script• BI Server’s Aggregate Persistence executes script:

1. Create TimesTen aggregate table2. Populate TimesTen aggregate3. Update RPD online with new aggregate metadata

• Handles supporting dimensions tooBase data Aggregates

TimesTen

OBIEE Aggregate Persistence

RPD

Aggregate tables created

RPD updated with new aggregate mappings

Aggregate tables loaded from base data

Sunday, 4 August 13






Out-of-the-Box Summary Advisor Limitations

• To refresh an aggregate, Summary Advisor deletes and rebuilds from scratch• The RPD is edited directly on the BI Server each time an aggregate is created or rebuilt• Build failures can be difficult to debug, if it fails can leave the RPD in an inconsistent state with TimesTen

After a failed build, the aggregates are still in the RPD, but no longer exist in TimesTen

nqquery.log suggests possible errors but no clear root cause

This is that the error the user sees

Sunday, 4 August 13






Alternatives to using Aggregate Persistence

• However aggregates are refreshed, they must be included in the RPD‣Can be done manually‣Aggregate Persistence is useful for this

• Complete refresh of aggregate data‣Extract the SQL that OBIEE generates in Aggregate Persistence, run

this through ODI‣Write bespoke aggregate refresh code in ODI

• Incremental refresh using GoldenGate and ODI‣ Instead of rebuilding aggregates in their entirety each time, only update

the part of the aggregate that has changed on the base data

Sunday, 4 August 13






Custom BI Server ETL Option : Incremental BI Server Refresh

• Uses BI Server to do the refresh, but invokes just part of it - the data refresh part• Avoids unnecessary aggregate table drop/rebuild, and online RPD edit• Uses BI Server logical ETL SQL features

to just refresh the TT aggregate table, using only the latest set of detail-level data‣Avoids use of additional ETL tools‣No additional license cost‣Works against any BI Server source

Sunday, 4 August 13






Custom BI Server ETL Option : Incremental BI Server Refresh

• Uses a number of “undocumented” but support BI Server ETL features‣ POPULATE command for loading data into an RPD table‣ WHERE clause for limiting the refresh to just the incremental load‣ INACTIVE_SCHEMAS to stop BI Server refreshing using existing RPD agg table

• See http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics-aggregates-using-native-bi-server-capabilities/ for full details including scripts etc

SET VARIABLE DISABLE_CACHE_HIT=1, DISABLE_CACHE_SEED=1, DISABLE_SUMMARY_STATS_LOGGING=1, INACTIVE_SCHEMAS='"TimesTen Aggregates".."EXALYTICS"'; populate "ag_sales_month" mode ( append table connection pool "TimesTen aggregates"."TT_CP") as select_business_model "Sales"."Fact Sales"."Sale Amount" as "Sale_Amoun000000AD","Sales"."Dim Times"."Month YYYYMM" as "Month_YYYY000000D0" from "Sales" where "Dim Times"."Month YYYYMM" = VALUEOF("THIS_MONTH");

Sunday, 4 August 13





http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics-aggregates-using-native-bi-server-capabilities/





But .. What About if I want to Put My Entire Data Warehouse In-Memory

• Actually a common request - many reporting datasets are under 500GB / 1TB in size• Why not put the whole dataset in-memory, rather than just expensively maintain aggregates?• Maybe put “hot data” - current month, popular products - into TT as a form of in-memory partitioning?• Use Exalytics as an analytics toolbox - independence from IT / DBAs• In-memory access to detail-level, as well as summary-level, data

Sunday, 4 August 13






Options for Analyzing Large Datasets “In-Memory”

• Summary Advisor only puts aggregates, not the whole dataset, in-memory‣May be the only practical solution for very large datasets, and comes with automation for build/maintainance

• TimesTen could potentially hold the full dataset, but has practical limitations on scale‣ TimesTen tables can be large, even with compression‣ Lacks PQ, partitioning, so could possibly struggle when analyzing large amounts of table data

• Essbase Aggregate Storage Option (ASO) is also a possibility, and also ships with Exalytics‣Stores data at detail level, uses indexing and fast aggregation to analyze large, dimensional datasets

vs.

Sunday, 4 August 13






What is Essbase Aggregate Storage Option?

• Essbase is Oracle’s Exalytics OLAP server, and has two storage types for databases: BSO and ASO• BSO (“Block Storage Option”) is traditional MOLAP, and stores data in multidimensional blocks and indexes‣Good for fast access and write-back at all levels, needs pre-calculating, but limited in scale

• ASO (“Aggregate Storage Option”) is more like relational storage, and can scale hugely‣ 100x scaleability over BSO, 10x faster load and calc, 20x smaller databases‣But can only write-back to level 0, more limited calculations, aimed at DW-style sales analysis applications‣Uses bitmaps based on cube metadata to quickly index the whole cube, plus optional stored pre-aggs‣Different approach to TimesTen, more “OLAP” than relational, but is it an option?

Sunday, 4 August 13






Example Two: Storing Detail-Level DW Data in Exalytics

• Examples are to establish which of TimesTen, or Essbase ASO, is best-suited to storing whole DW datasets• Two sample datasets were chosen:‣Sample Sales (SH) from the Oracle database - 900k rows of fact data + 5 dimensions‣Airline Delays from the Exalytics demo program - 130m+ rows of flight leg information + 4 dims

• Looking to assess three factors:‣Speed to load‣Speed to access detail data‣Speed to access summary data

SH.SALES900k+ rows PERFORMANCE

130m+ rows

Sunday, 4 August 13






Initial Data Load - TimesTen for Exalytics

• TimesTen tries to mirror Oracle RBDMS in terms of SQL, PL/SQL etc• Has its own optimized datatypes, plus in-memory optimized indexes‣ Ideally you would map Oracle datatypes to the most

optimal TT ones, not just VARCHAR, NUMBER etc• Column-based compression can also be used‣Creates query performance gain proportionate to

the space saving, i.e. 20% compression, 20% faster query‣Benefits dimension tables most, but some gain for facts

• ttimportfromOracle utility automatically creates optimal TT DDL, and replicates the Oracle data into TT‣Need to be on TimesTen 11.1.2.2.5 to be able to

load into compressed TT tables‣ From TT 11.1.2.2.5, can make use of parallel load into TT‣All tables loaded, took around 9GB space + 4GB temp

c:\TEMP\tt\tt_sh>C:\TimesTen\tt1122_64_3\support\ttImportFromOracle.exe -oraConn sh/password@orcl -compression 1 -tables sales promotions products customers channels timesBeginning processingResolving any tablename wildcardsEliminating any duplicate tablesGetting metadata from sourceGenerating database user listAssigning TimesTen datatypesAnalyzing source tables (this may take some time)Analyzing table 'SH.SALES' ...Analyzing table 'SH.PROMOTIONS' ...Analyzing table 'SH.PRODUCTS' ...Analyzing table 'SH.CUSTOMERS' ...Analyzing table 'SH.CHANNELS' ...Analyzing table 'SH.TIMES' ...Estimating compression ratiosGenerating output filesFinished processing

Sunday, 4 August 13






Initial Data Load - Essbase ASO

• Had to model the data in Essbase Studio, the cube design IDE for Essbase• Prefix and transform dimension member names to

ensure uniqueness• Can source data either from OBIEE repository,

or direct from source• Essbase database loaded in <1 min, around 250MB in size• Modeling was only for a single star - question how practical this

modeling in Essbase would be for a large, conformed DW‣Knowledge of Essbase required‣Dealing with data issues‣Not a trivial exercise

Sunday, 4 August 13






Initial Query Response Times for Large, Detail-Level Datasets

• Created dashboards against the Oracle DB, TimesTen and Essbase ASO databases• Enabled for Exalytics, removed Submit and Reset buttons from prompts• Results were surprising‣All were fast (<1 sec/query) for Sales History‣Oracle RDBMS was slow for Airline Delays

- 60 secs+ for individual analyses‣Essbase ASO was fast for Airline Delays

- 5-10 secs for individual analyses- But caused a few BI Server crashes

‣ TimesTen was also slow for Airline Delays- 30 secs+ for a query- Lots of timeouts

• Was this correct?

Sunday, 4 August 13






Analyzing the Slow TimesTen Query Performance with Large Datasets

• Queries against Oracle’s SH.SALES table can often benefit from partition elimination• Oracle database also has potential for parallel query - TT is limited to a single CPU/query• But is the TT execution plan optimal?• No - TmpHashScan operation indicates TT’s optimizer

has created an on-the-fly temporary index for the query‣ Impacts on query performance‣ Ideally, these indexes should already be there

• Solution - post-load index optimizationusing the TT Index Advisor utility

TmpHashScan indicates creation of temporary index

Sunday, 4 August 13






Optimizing the TimesTen for Exalytics Database

• TT Index Advisor uses a three-stage process‣ ttindexAdviceCaptureStart(1,0) - start capturing all queries across the database ‣ ttindexAdviceCaptureEnd(1) - stop capture of queries‣ ttindexAdviceCaptureOutput(1) - output index DDL for queries across database

• Then implement the DDL, and rinse-and-repeat• Impact - massive - query time fell to <1 sec‣But would need to run advisor regularly

Sunday, 4 August 13






Optimizing the Essbase ASO Database

• But - there’s a similar process also available for Essbase ASO• Focuses on aggregates, rather than indexing (indexing already covered by ASO database bitmaps)• Accessed from the Essbase Administration Services Console• Fast to aggregate - can recommend, or run manually• Also brought query response time

down to <1 sec

Sunday, 4 August 13






Refreshing the TimesTen and Essbase ASO Databases

• As well as the initial load, you will need to refresh (or rebuild) the in-memory structures with new data• TimesTen tables can be refreshed using various options‣Create custom ODI or GoldenGate routines - same as any other target, but “fragile”

- Beware table locking - TT compressed tables cannot be queried while loading- OBI also takes out full table locks when running queries- ttloadfromOracle can be used to automatically full-load TT tables, or edit script with WHERE clause- Either requires BI down-time, or use double-buffering solution

• ASO has features for incremental, trickle-refresh for databases‣Based around concept of “slices” - sub-cubes that contain the new/updated data‣Database is then dynamically aggregated to include the slices‣Slices can be incorporated back into the main database from time-to-time‣Designed to support incremental load

Sunday, 4 August 13






So Which Technology Best Suits Loading the Whole Dataset into RAM?

• Remember - primary use-case for Exalytics / OBIEE / Timesten is caching of regularly-used aggregates• Permits analysis across massive source datasets, and concentrates in-memory feature where most needed• But - space permitting - TimesTen could handle the whole dataset‣Remember to use compression‣Use the Index Advisor after each load‣Also factor in the refreshing of TT tables - is this a 24x7 system

• Essbase ASO also an interesting option‣Much smaller datasets - potential to load more source data‣Optimized for this task‣But requires Essbase modeling skills‣ Features exist for Essbase ASO real-time refresh,

but probably will need an Essbase admin on the teamvs.

Sunday, 4 August 13






Deployments of Exalytics “In The Field”

• Surprisingly few of our implementations have been Exadata + Exalytics‣Maybe Exadata provides enough of a performance boost?‣Maybe those customers are self-supporting?

• Most have bought Exalytics as the best way to host their BI + Analytics mid-tier‣ Typically with a non-Exadata (but Oracle) database underneath‣Sometimes as a way of avoiding DW query optimization

• For many, these are their first Sun (and Exa-) servers• Many have bought single Exalytics servers ‣ (but what about dev, test etc?)

• Some have bought 3+ (including one of our customers)

Sunday, 4 August 13






Deploying Multiple Environments

• Exalytics main value proposition is around performance‣ In-memory aggregates speeding up query performance

• However many customers wish to use Exalytics for consolidation‣Run multiple environments (test, dev, prod) on a single server‣Consolidate multiple applications on a single server

• Desire is usually down to cost (multiple physical servers, per-processor licensing)

DEV DEV TEST PROD

vs.

MWHOME1

DEV

MWHOME2

DEV

MWHOME3 MWHOME1 MWHOME1 MWHOME1

Single Exalytics Server Exalytics Svr 1 Exalytics Svr 2 Exalytics Svr 3

Sunday, 4 August 13






Oracle “Best Practice” for Consolidation : Oracle Virtual Machine

• Exalytics is now certified for use with Oracle Virtual Machine, Oracle’s hard-partitioning virtualization platform

• OVM is installed first, on Exalytics “bare metal”, with OVM Repository on external DB server (as with BIPLATFORM, MDS)

• Exalytics VM templates then used to create required environments (dev, test etc)

• Also allows for licensing in smaller per-processor license blocks (1 processor, 2 cores)

• OVM provides for resource caging, isolating environments‣But requires more infrastructure‣And has limitations around use of InfiniBand

• So many customers (and us) just do multiple FMW Home installson the same server, manually managing conflicts etc‣Supported by Oracle, now detailed in tech docs

DEV

MWHOME1

Single Exalytics Server

OVM Hypervisor (Oracle VM Server)

OVM Guest 1

DEV

MWHOME1

OVM Guest 2

DEV

MWHOME1

OVM Guest 3

Oracle VMManager

Oracle VMRepository

Sunday, 4 August 13






High Availability using Clustered Exalytics Servers

• Multiple Exalytics servers can be clustered for HA and performance (load balancing)‣Requires hardware load balancer above clustered servers

• Linked together via InfiniBand (one for clustering, one for connection to Exadata)• Note: does not double available RAM

- aggs are replicated in each TT database‣Note that Summary Advisor / nqcmd

does not replicate agg tables -manual replication required

• HA is between TT instances, and OBIEEmanaged server + system components

Sunday, 4 August 13






Summary

• 1 year on, we are starting to see Exalytics deployments “in the field”• Usage of the platform is much wider than just “aggregates into TimesTen”‣Customers want to get “full value” from their high-end hardware, esp. 1TB RAM

• Recent platform refresh to 2TB RAM, SSD, plus several software + certification updates• OBIEE + Summary Advisor + TT works as a way of caching commonly-used aggs in memory‣Recent updates make support for BI Apps etc possible‣But give consideration to summary update process - default solution can be inefficient

• Also possible to store whole datasets / datamarts / DW sub-sets in-memory‣Utilities exist for TT to automate the process - but don’t forget Index Advisor post-load‣Essbase ASO also an interesting solution for large, sparse datasets

• Many solutions exist for HA, multiple environments etc• Speak to Rittman Mead if you’re considering Exalytics -

experience from several real-world projects in UK, Europe, USA

Sunday, 4 August 13






Thank You for Attending!

•Thank you for attending this presentation, and more information can be found at http://www.rittmanmead.com•Contact us at [email protected] or [email protected]•Look out for our book, “Oracle Business Intelligence Developers Guide” out now!•Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead)

Sunday, 4 August 13












Oracle Exalytics - Tips and Experiences from the FieldMark Rittman & Stewart Bryson, Rittman MeadEnkitec Extreme Exadata Expo, Texas, August 2013


Sunday, 4 August 13









Oracle Exalytics - Tips and Experiences from the Field (Enkitec E4 Conference 2013, Dallas)

Technology

Transcript of Oracle Exalytics - Tips and Experiences from the Field (Enkitec E4 Conference 2013, Dallas)