459A

Calc Scripts for Mere Mortals

Presented by:Edward Roske, interRel Consulting

Agenda

Review the Basics Calc Script Commands Using Functions Controlling Execution Creating Blocks Organizing Calc Scripts Real World Calculations Optimization

Types Of Calculations

Outline consolidations Unary operators + - * / % Relies on hierarchy

Member formulas Allows complex calculations Can use functions and conditional processing

Calculation scripts Override outline operations Offer most control

Calculation Order

Two basic categories Outline order

Dense Sparse

Dimension order

Calculation Order All Dimensions First, Accounts Second, Time Third, all remaining dense dimensions Forth, all remaining sparse dimensions Last, Two-pass calculations

Calc Scripts

Reasons to use calc scripts: Override outline operators Specific calculation commands available Iterative calculations Variables Optimization

Calc Script Editor

Choice of the Essbase XTD Analytic Services Calc Script Editor or another text editor

If you use another editor: Save file with .csc extension Easier to perform some functions

Find/Replace See more lines of the script Printing options

Can mix and match

Essbase XTD Analytic Services Calc Script Editor Has a number of functions built in Command buttons simplify entries Validation function to detect errors Limited work area

Calc Script Editor Functions

Most simply enter the character that are on the face of the button

The is a notable exception Provides a list of available functions Grouped by function type Can include parameters

Member Selection

Associate the outline Can select members from the member list

Automatically adds quotes if needed Frequently easier/faster to just type the

member names

Online Help

Use menu option F1 does not work

Functions and commands are included Can also use Essbase XTD Analytic Services

documentation Technical Reference

Calculation Commands

Limited number of true calculation commands Not functions they are in another chapter Not Set statements they are covered under

optimization

Agg

Syntax AGG(Dimname, Dimname,);

Summary Aggregates dimension(s) according to the outline

operators Member formulas are ignored Very fast

Fewer than 6 levels

Array

Syntax Array ArrayName[DimName] = {initial values };

Summary Declares an array variable with as many

members as the dimension name that follows it Can be initialized with values if desired

Optional

Calc All

Syntax Calc All;

Summary Calculates all dimensions in the database Uses default calculation order This is typically the default calc Very efficient Can add Except to remove

dimensions/members

Calc First

Syntax Calc First;

Summary Calculates members tagged as TB First Does not update Time dimension, only Accounts

Calc Last

Syntax Calc Last;

Summary Calculates members tagged as TB Last Does not update Time dimension, only Accounts

Calc Average

Syntax Calc Average;

Summary Calculates members tagged as TB Average Does not update Time dimension, only Accounts

Calc Dim

Syntax Calc Dim(DimName, DimName,)

Summary Calculates the dimensions that are listed All dense dimensions are calculated first in the order

they appear Sparse dimensions are calculated last - also in the

order they appear If you need a different order, use multiple Calc Dim

commands Includes member formulas

Calc TwoPass

Syntax Calc TwoPass;

Summary Calculates members tagged as Two-Pass in the

Accounts dimension

CCONV

Syntax CCONV type->Currency name

Summary Special command reserved for currency

conversion Converts values to the currency name specified Only usable if you have the currency conversion

module

ClearBlock

Syntax ClearBlock ALL | UPPER | NONINPUT |

DYNAMIC Summary

Designed to clear blocks by replacing values with #Missing

Within a FIX block, may only clear a portion of a block

Options for which types of block to clear

ClearData

Syntax ClearData MemberName;

Summary Designed to set a member value to #Missing Works within a fix

ClearBlock vs. ClearData

ClearBlock is designed to clear an entire block At least a large portion of the block

No Member name, just block type ClearData is designed to clear a member

Usually before it is recalculated In many cases either one will work

DataCopy

Syntax DataCopy MemberName To MemberName;

Summary Copies data from one member to another All intersections are included Great for copying versions Will create blocks if they do not exist

If/Else/ElseIf/EndIf

Syntax IF (condition) ElseIf/Else EndIf;

Summary Condition must be if parenthesis In a calc script, associate with a member name

Does not really matter which member name Each If must end with EndIf Most effective on dense members

Fix/EndFix

Synatx Fix(MemberNames, Functions Etc.) ENDFIX

Summary Used to focus or limit a calculation Most effective on sparse members Used extensively

Loop/EndLoop

Syntax Loop (integer, break) EndLoop;

Summary Causes a block of code to be repeatedly

executed The integer determines the number of times the

loop runs The break value allows early exit Although it is a loop, its not a true programming

loop

Using Commands

Need to combine commands with functions Execute in the order they are written None by itself is that useful Used together, they provide a powerful tool

set to apply business rules to the database

Not Commands But

A member name by itself Calculates the member formula if present Calculates outline rollup if there is no formula

Example East;

Multiple members can be listed Member set functions can also be used

Example @IDESCENDANTS(2003)

Assignment Statements

Assign a value to a member Example

Budget = 0; Budget = Budget * 0.95;

There are other math operators More later

Every command ends with a semicolon (;) Okay, there are exceptions but most do

More on Member Names

All the members are calculated at once if possible

To force multiple passes of the database, group the member names in parenthesis One pass for each group in parenthesis Not usually what you want, so use with caution

Function Categories

Conditional Boolean Relationship Calculation Operators Mathematical Member Set Range and Financial

Allocation Forecasting Statistical Date and Time Miscellaneous Custom Defined

Calculation Operators

Nothing too exciting + - * / % Use parenthesis to control order of

operations

Conditional

Typically used within an If command >, >=,

Boolean

More traditional functions@ISACCTYPE@ISANCEST@ISCHILD@ISDESC@ISGEN@ISIANCEST@ISICHILD@ISIDESC@ISIPARENT

Typically used in If Statement Returns True/False

@ISISIBLING@ISLEV@ISMBR@ISPARENT@ISSAMEGEN@ISSAMELEV@ISSIBLING@ISUDA

Relationship Functions

Returns Values Frequently used to get other values in

database @ANCESTVAL@ATTRIBUTEBVAL@ATTRIBUTESVAL@ATTRIBUTEVAL@CURGEN@CURLEV@GEN

@LEV@MDANCESTVAL@MDPARENTVAL@PARENTVAL@SANCESTVAL@SPARENTVAL@XREF

Mathematical Functions

Most functions available@ABS@AVG@EXP@FACTORIAL@INT@LN@LOG@LOG10@MAX@MAXS

@MIN@MINS@MOD@POWER@REMAINDER@ROUND@SUM@TRUNCATE@VAR@VARPER

Member Set Functions

Returns a List of Member Names

@ALLANCESTORS@ANCEST@ANCESTORS@ATTRIBUTE@CHILDREN@CURRMBR@DESCENDANTS@GENMBRS@IALLANCESTORS@IANCESTORS@ICHILDREN

@MATCH

@IDESCENDANTS@ILSIBLINGS@IRSIBLINGS@IRDESCENDANTS@ISIBLINGS@LEVMBRS@LIST@LSIBLINGS

@MEMBER

@RDESCENDANTS

@MERGE@PARENT@RANGE

@RELATIVE@REMOVE@RSIBLINGS@SIBLINGS@UDA@WITHATTR

Range and Financial Functions

Accepts a range of members instead of individual members

@MINRANGE@MINSRANGE@NEXT@NEXTS@NPV@PTD@PRIOR@PRIORS@SHIFT@SLN@SUMRANGE@SYD

@ACCUM@AVGRANGE@COMPOUND@COMPOUNDGROWTH@CURRMBRRANGE@DECLINE@DISCOUNT@GROWTH@INTEREST@IRR@MAXRANGE@MAXSRANGE@MDSHIFT

Allocations Functions

Only two functions in this category Allocate MDAllocate

Used in simple situations to allocate an amount down to lower levels

Forecasting Functions

Similar to what you may have used in Excel @MOVAVG@MOVMAX@MOVMED@MOVMIN@SPLINE@TREND

Statistical Functions

Most common are available@CORRELATION@COUNT@MEDIAN@MODE@RANK

@STDEV@STDEVP@STDEVRANGE@VARIANCE@VARIANCEP

Date and Time

Just one to convert strings to a date ToDate

Miscellaneous

The catch-all category@CALCMODE@CONCATENATE@SUBSTRING@NAME

Using Functions

Frequently, functions are nested or used as parameters for another function

Functions are prefixed with @ Parameters are separated by commas Used in many places

Right of = FIX statements IFs etc.

Focusing Calculations

Two most common methods Fix If

Other methods: CrossDim operator MemberSet functions Relationship functions Variables

FIX

FIX limits the members that are processed Most effective when used on members of

sparse dimensions Only blocks that meet the Fix criteria are

processed Remaining blocks are simply skipped

Typical uses Only calculate one scenario Limit calc to current year

More on FIX

FIX works properly on dense members May result in all blocks being processed All statements between the FIX and ENDFIX

are executed as a block FIX statements an be nested Multiple members from the same dimension

are treated as or In other words, all members will be processed

What You Cannot Do With FIX

You cannot assign values to a member that is included in the FIX statement Actually, values cannot be assigned to any

member from a dimension in the FIX statement Example

Fix(Sales)Sales = 100;

ENDFIX Would result in an error

FIX On

Members from one or more dimensions Can list specific member names Can include member set functions Can include AND and OR

Use AND to get the intersection of two or more functions

Caution on empty sets

IF Statements

Conditional processing is often needed to process business rules

Can include complex conditions And, Or, Not, etc.

What You Can Check For

Most common are Members

Specific members MemberSet comparisons Use boolean functions

Values >, < = etc. Use #Missing for test

True is treated as 1 and false is 0

Syntax for IF

Place condition in parenthesis Associate with a member

Doesnt matter which member MemberName (

If (condition) Statements

EndIf ) Not required for member formulas

More on IF

Can have ElseIf (condition) Can Have Else Can have both Still terminate with single EndIf

Be sure to include ; All ElseIF/Else conditions have to remain

within parenthesis for the member association

To If or FIX

Generally, If works best on dense dimension members Particularly when ElseIF or Else are

includedBlock is only brought into memory once and all related

conditions are processedFix would bring block in multiple times

Conversely, FIX works best on sparse dimension members

Old saying Fix on Sparse, If on dense If youre dense, fix on sparse?

There are always exceptions

Other Method to Focus Calculations Cross Dim operator

Allows reference to specific cells Frequently in another block Technique for complex allocations

Relationship functions Can be used to get a value from another part of the

database Example is @ParentVal()

MemberSet functions Return member names Can be used to calculate those members or in a Fix/If

Loops

Can use the Loop/EndLoop to repeatedly execute calc commands

Not designed to increment variables and reference different members

Used to solve goal seeking problems

Why Create Blocks

Often need blocks that do not exist Typically when performing allocations

Actually, one of the most challenging aspects of writing calc scripts If a block does not exist, not processed by a calc script

Except outline calculations Frequent cause of calculation problems

ExampleIF (Budget > 100)

Actual = 50;ENDIF Assumes Scenario is sparse

Ways to Create Blocks

There are basically four methods to create blocks: Load data Calculate the database Use the DataCopy command Place a sparse member on the left side of an

assignment statement A little more complex than this

Load Data

During a data load, a block is created if it doesnt already exist Can use this to create blocks Create data records based on source information Load the file One trick is to use the Replace function of the

load rule

Calculate the Database

An outline rollup will create upper level blocks

Both Calc All/Dim and AGG will cause blocks to be created

Only blocks containing data are created Empty blocks are not written

DataCopy Command

Easiest way to create blocks If an intersection for the To member does

not exist, it is created Frequently, the only practical method to

create the needed blocks Danger in that it can create excessive

numbers of blocks

Assignment Statements

If The member on the left of the assignment

statement is sparseAnd

The database setting Create bocks on equations is selected

Blocks are created Not always possible to have a sparse

member on the left side of the assignment statement Try fixing on a dense member and assigning to

a sparse memberE.g. Fix on the account member and assign to another

May need to change dense/sparse settings

Changing Database Settings

A quick diversion When you change database settings, you

should stop and then start the database (or application) for the settings to take effect

Organizing Calc Scripts

Most calc scripts have common parts or sections Housekeeping Initial Calculation Mainline Rollup Back Calc

Housekeeping

The calculation environment is established Intelligent calculation is turned on/off Other Set commands are issue

Variables are declared Comments describing the purpose of the calc

script are entered Use /* and */ for comments

Key Set Commands

Set AggMissg ON | OFF Controls if #Missing values are aggregated If all children and #Missing, parent is et to

#Missing Upper level step-on

Caution needed if values loaded at upper levels SET MSG ERROR | WARN | INFO | SUMMARY |

DETAIL | NONE; Detail lists each block being processed Good for debugging Generates a large log and is slow

Initial Calculation

This is where business rules are applied Frequently include allocations

This usually requires a partial database rollup There may be many calculation blocks with

FIX statements Or If statements

Typically the most complex portion of the calc script

Mainline Calculation

This is the rollup of the database May be focused

e.g. FIX(Actual) Regardless, this is the major rollup of the

database

Back Calc

This is where two-pass members and any other members requiring special treatment are handled

Often requires recalculation of all or most of the blocks

Corrects percentages and ratios that may have been summed

Testing Calc Scripts

The following methodology works well: Use a test database Create test data in Excel

Keep it on its own sheet Clear the database Use Lock & Send to load test data Run the calc from Excel Retrieve onto a test sheet

More on Testing

Best to know expected outcome before retrieving

Can create a calc script to clear the database Minimizes switching between Excel and App

Manager The key is small amounts of data which

allows short test cycles Many people test with large amounts of data

resulting in long cycles and inability to get many tests in per day

When initial testing is complete, add to the test data set to make sure everything is working properly

Typical Issues

Intelligent Calc Prevents blocks from being calculated

FIX on too few or too many members Nested FIXs are ORs, not ANDs

Dynamic Calc runs after Batch Calc Dynamic calc members do not make good counters

Referring to the wrong block Remember, every intersection is processed unless

the calculation is focused

Real World Calcs

These exercises are derived from calculations created for customers but modified for the Sample application

Exercise

Simple Allocation Complex Allocation Comparable Sales Compute LQ (last quarter) in the scenario

dimension Compute 12 month rolling average Compute Market Share and Product Mix Get data from another database

Exercise (Continued)

Putting it together This is for the Budget dimension only Clear upper level blocks Add the member Insurance to the Measures

dimension and load a value at the top level Allocate the insurance expense to all products based

on Budgeted Sales Calculate Market Share in the calc script (may need

to change storage settings); insure it is correct at all levels

Include values for Last Quarter and a 6 month rolling average

Perform a rollup of the database

Goal

Make the calculation run faster Variety of settings/changes available to make

things run faster Many limiting factors

Existing reports/processes Source systems Hardware

Parallel Calculation

Apply multiple processors to a calc Only straight forward calcs can use this

If order dependent portions, calculated in serial Maximum of 4 processors Recommend 1 less than number on server Available in Essbase XTD Analytic

Services 6.5

Cache Settings

More is not always better Sometime Hyperion Essbase seems to spend

more time maintaining caches than calculating Usually find a point where increasing cache

does not improve performance Typically a trial and error process Set uncommitted / 0 on Transaction tab

Dense/Sparse Settings

This will make the largest difference Other factors influence decisions

Attribute dimensions If only a portion of the database is calculated

ideally you want to Fix on it as sparse members Reduces the number of blocks processed

May need to try a number of combinations Smaller blocks almost always calc faster

Review Dimension Order

Largest dense dimension to smallest Smallest sparse dimension to largest Hourglass shape Dimension calc order is influenced by order

of dimensions in the outline Goal is to process fewest blocks possible

Eliminate Database Fragmentation Previously calculated database takes longer

to calculate Faster to create blocks than to update them

Non-adjacent blocks can take longer to access As data is added to blocks, they grow and cannot

be rewritten to same place Export, ClearData (All), Import OR

Restructure Old DeFrag Command no longer available

Want to calc everything in the block at same time

Review Blocks that are Processed Can use SET MSG DETAIL Each block is listed as it is calculated

See application log Can determine number of times each block is

processed Can also see if expected blocks are being

processed Often, extra blocks are being read Correct by updating FIX

Control Calculation Environment Use SET Commands

Set CACHE HIGH; New cache Tracks the block used in the calc

Set CALCHASHTBL ON; Good for flat dimensions

Set FromBottomUp ON; Faster than TopDown Make sure you get correct results

Other Methods

Remove dimensions from the cube Can make a huge difference Some companies keep multiple databases

One with all dimensions and other with limited Partition

Move data that does not need to be calculated to another database

Intelligent Calc

More SET Commands

SET LOCKBLOCK HIGH; Allows more blocks to be locked Not really optimization but

SET CALCTASKDIMS n; Determines how many dimensions are used to

generate opportunities for the parallel calculation

Simulate the Calculation

Method to predict the actual calc time Based on the number of blocks that would be

created SET MSG ONLY; SET NOTICE HIGH; CALC ALL; Does not actually calc the database but

provides timings Run the calc, and use ratio of actual time to

simulated Not perfect but

Dynamic Calc

Can improve calc times significantly Make entire dimension dynamic calc

Never needs to be calculated Especially effective for Time dimension

Normally calculates very slowly All upper level members of dense dimension

Keeps block smaller Upper 1-2 levels of sparse dimension(s)

Be aware of how many blocks are needed to calc highest level

Custom Functions

Can write functions in Java Compile, save on server, and register Call from calc scripts Run slower than same calc command May run faster than complex series of

database passes required to perform the same thing

Custom Macros

Can predefine combinations of commands SumRange is actually a Macro

Combines Sum and Range functions Not necessarily faster but might avoid errors Need to define parameters May be possible to save a pass of blocks

Database Statistics

Not a optimization technique but a tool Can monitor calculation impacts

Number of blocks created Can refresh while calc is running

Block creation rate Not necessarily best for the calc

Copy and paste into Excel

Comments / Questions ?

Edward Roske

Office: 972-735-8716

Email: eroske@interrel.com

Please Complete Your Session Evaluation.

459ACalc Scripts for Mere Mortals

Presented by:Edward Roske,interRel Consulting

459AAgendaTypes Of CalculationsCalculation OrderCalculation Order All DimensionsCalc ScriptsCalc Script EditorEssbase XTD Analytic Services Calc Script EditorCalc Script Editor FunctionsMember SelectionOnline HelpCalculation CommandsAggArrayCalc AllCalc FirstCalc LastCalc AverageCalc DimCalc TwoPassCCONVClearBlockClearDataClearBlock vs. ClearDataDataCopyIf/Else/ElseIf/EndIfFix/EndFixLoop/EndLoopUsing CommandsNot Commands ButAssignment StatementsMore on Member NamesFunction CategoriesCalculation OperatorsConditionalBooleanRelationship FunctionsMathematical FunctionsMember Set FunctionsRange and Financial FunctionsAllocations FunctionsForecasting FunctionsStatistical FunctionsDate and TimeMiscellaneousUsing FunctionsFocusing CalculationsFIXMore on FIXWhat You Cannot Do With FIXFIX OnIF StatementsWhat You Can Check ForSyntax for IFMore on IFTo If or FIXOther Method to Focus CalculationsLoopsWhy Create BlocksWays to Create BlocksLoad DataCalculate the DatabaseDataCopy CommandAssignment StatementsChanging Database SettingsOrganizing Calc ScriptsHousekeepingKey Set CommandsInitial CalculationMainline CalculationBack CalcTesting Calc ScriptsMore on TestingTypical IssuesReal World CalcsExerciseExercise (Continued)GoalParallel CalculationCache SettingsDense/Sparse SettingsReview Dimension OrderEliminate Database FragmentationReview Blocks that are ProcessedControl Calculation EnvironmentOther MethodsMore SET CommandsSimulate the CalculationDynamic CalcCustom FunctionsCustom MacrosDatabase StatisticsComments / Questions ?Please Complete Your Session Evaluation.