#Kscope

To Slice or Not to Slice?

The Next Advancement in ASO Cubes

Angela WilcoxBI ArchitectMedAssets

#Kscope

Developing Essbase Applications

● Like the best, most advanced Essbase conference there ever could be

● Advanced content● Good practices● Written by some of the most well

known Essbase developers● Source code at

www.developingessbasebook.com● You should buy it

#Kscope

ASO Cube Slices

Only available in ASO cubes

Allows true incremental loads Allows true regional clears Real time and near-real time now possible

Greatest thing since sliced bread! Really!

#Kscope

Steps to Using Slices

STEP 1: Load data into the cube and aggregate using your normal processes

STEP 2: Define your new load and define it to load to a slice● Instead of loading in the main cube it will be situated

next to it● Aggregate views will be created automatically

#Kscope

Steps to Using Slices Cont…

STEP 3: Multiple slices can be added in the same set of loads

STEP 4: Eventually performance degradation will be noted and can be statistically proven in the statistics

STEP 5: Merge the slices together● Merge into a single slice or● Merge the slices into the larger historical slice

#Kscope

Steps to Using Slices, Cont.

STEP 6: Aggregations must be dropped when the merge occurs – so a re-aggregation of data will need to happen when the merge is completed.

The specifics of which options to deploy for an installation will be unique and need to be considered.

Look at a real-life case study from JCP

#Kscope

Use Case

Discuss and examine production code for each of the following tasks:● Creating the Main Database● A Week in the Life● Merging Slices● Updating Dimensions● Aggregating the Cube● Creating a Slice● Clearing Slices

#Kscope

Requirements

The requirements for this implementation had some unique characteristics that no previous implementation had:● Essbase = database of record● Data from twenty-five disparate sources● The staging tables would hold one week only and

then be cleared● Two years of history plus the one year building

#Kscope

Requirements, Cont

Granular Dimension● Six levels● Million plus members

Large weekly load volumes● Fifty million records to start● Three hundred million records when fully deployed

No downtime during Monday and Thursday updates

#Kscope

Requirements, Cont

Dimensional updates ● Weekends only● Cube allowed offline

Essbase Classic Add-in Templates● Template Only● No Ad-hoc● SLA less than 30 seconds

#Kscope

View Before Processing Starts

Statistics regarding the slices and incremental data are provided in EAS. ● Open EAS.● Drill down to the database level of the cube.● Right-click on the database and select Edit > Properties.● Select the Statistics Tab.

#Kscope

A Week in the Life

Cube 1 Saturday am

alter database ${APP_NAME}.${DB_NAME} clear aggregates ; export database ${APP_NAME}.${DB_NAME} level0 data to

data_file '${APP_NAME}.export.txt' ;

Dimensions of Cube1 Total Stored

Metrics 171 169

Dimension1 215 215

Dimension2 6 6

Dimension3 25 11

Dimension4 228 228

Dimension5 1159 1148

Dimension 6 1039504 843779

#Kscope

A Week - Merging Slices

alter database ${APP_NAME}.${DB_NAME} merge all data;

We did not use – but available:● alter database ${APP_NAME}.${DB_NAME} merge

all data remove_zero_values;● alter database ${APP_NAME}.${DB_NAME} merge

incremental data;

#Kscope

A Week Sunday - Updating Dimensions

import database ${APP_NAME}.${DB_NAME} dimensions connect as ${USER}identified by ‘${PWD}’ using server rules_file '${LD_RULE1}', connect as ${USER}identified by ‘${PWD}’ using server rules_file '${LD_RULE2}’on error append to '${ESS_ERROUT}' ;

#Kscope

A Week Sunday – Aggregating Data

execute aggregate build on database ${APP_NAME}.${DB_NAME} using view_file ‘${AGG_VIEW}’;

Views vs. Default Aggregation● Behavior Known?● Size of Cube?● SLA met with default?

#Kscope

A Week Sunday – Create Slices

alter database ${APP_NAME}.${DB_NAME} initialize load_buffer with buffer_id 1 resource_usage 1.0 property ignore_missing_values;import database ${APP_NAME}.${DB_NAME} data connect as ${USER} identified by '${PWD}' using server rules_file ${LD_RULE} to load_buffer with buffer_id 1 on error write to '${ESS_ERROR_DIR}${JOB_FILE_PREFIX}.${APP_NAME}.${LD_RULE}.err' ;import database ${APP_NAME}.${DB_NAME} data from load_buffer with buffer_id 1 create slice;

#Kscope

Slice Behavior

Behavior not always as expected

Load #Cell Count Result

1 75,000 1 Slice with 75,000 cells

2 400,0001 Slice with 475,00 cells (merged because slice one had less than 100,000 cells)

3 1,000,0002 Slices; 1 with 1,000,000 cells and 1 with 475,000 cells (nothing was merged because load #3 is greater than two times larger than the existing slice)

4 300,0002 Slices; 1 with 1,000,000 cells and 1 with 775,000 cells (load 4 is merged with loads 1 and 2 because that slice is less than 2 times the size of the new slice)

#Kscope

Weekend Setup Recap BIG ASO Saturday:

● Clear Aggregates● Backup Using Export● Drop Aggregates● Merge Slices

Sunday● Build Dimensions● Re-Aggregate● Load Data

SMALL ASO Saturday:

● Clear Aggregates● Backup Using Export● Drop Aggregates● Merge Slices● Re-Aggregate

Sunday● Build Dimensions

● Load Data

#Kscope

When Things Go Wrong - Clearing Slices import database ${APP_NAME}.${DB_NAME}

data from load_buffer with buffer_id 1 override all data;

import database ${APP_NAME}.${DB_NAME} data from load_buffer with buffer_id 1 override incremental data;

import database ${APP_NAME}.${DB_NAME} data from load_buffer with buffer_id 1 subtract values create slice;

#Kscope

Clearing Slices, Cont

alter database ${APP_NAME}.${DB_NAME} clear data in region ‘{2011W48}’ physical;

alter database ${APP_NAME}.${DB_NAME} clear data in region ‘{2011W48}’; ● Logical Clear - Creates a slice● Zeros

A number of things will automatically happen if there are multiple slices in the cube when this command is issued:● Merge the slices● Clear the specified region● Re-materialize all aggregate views

#Kscope

Questions?

Most of the Information presented here is available in the “Developing Essbase Applications” – see my chapter!