hyperion.com

preliminary analysisThe first step in your analysis is to identify measures and

dimensions for the model. Even with increased dimensionalscalability of the ASO, it is preferable to include a dimensioninto the model only if the analysis along that dimension isrequired.

In addition, dependent dimensions should be modeled asattribute dimensions or alternative hierarchies if possible.

A dimension is dependent if you can always determine themember within that dimension when the member withinsome other (primary) dimension is specified. For example,consider a Product dimension with individual SKUs as thelowest level. The packaging type for a product can bedetermined by the SKU of that product. Thus, it isrecommended that you model the Packaging Typedimension as an attribute or alternative hierarchy on theproduct dimension, and not as a stand-alone dimension.

Whether you choose to use attributes or alternativehierarchies depends on the following factors:

If members of different levels need to be included intogroupings for the dependent dimension, choose alternativehierarchy modeling. For example, if you want to classifymarkets by size, and Large Market group includes several

large individual cities and several states, the Market Sizedependent dimension cannot be modeled as an attributedimension. Attributes can be associated only with membersof the same level.

If a requirement exists for querying the members of theprimary dimension across members of the dependentdimension, the dependent dimension should be imple-mented as an attribute dimension. For example, if carscan be classified by their color and model, and therequirement is to be able to ask queries, such as Give methe sales of all red Civics in San Francisco, the Colorshould be an attribute dimension.

outline sizingAfter the dimensions are identified, the following metricsshould be gathered, if possible:

For every dimension, determine the number of stored levels within every hierarchy on this dimension.1

Levels that consist only of label-only members are notstored, and they do not affect the sizing calculations. Forexample, consider the dimension Time with hierarchiesYear-Quarter-Month-Day and YearByWeek-Week-Day.This dimension contains a total of six levels, four in the

tuning of aggregate storage databasesessbase 7.1.2

performance visibility for everyone in the enterprise

The main purpose of this paper is to discuss the sizing and tuning methodsavailable when designing, deploying and maintaining an AggregateStorage database throughout its lifecycle. We will explain how Aggregate

Storage uses resources such as disk space and memory at the various stages of

the database construction and maintenance, and provide directions on how to

optimize the performance and the resource usage. The reader should have

general knowledge of Essbase.

white paper

hyperion.com

2

first hierarchy and two additional levels in the secondhierarchy (Day level is shared between the hierarchies).

Notice that all levels of attribute dimensions count asadditional levels of their primary dimensions.

Determine the total number of members across all dimen-sions.

number of possible viewsWith these metrics, you can determine if the model fits thelimit on the number of possible views. The total count ofviews, or level combinations, cannot exceed 252 in Essbase7.1.2.

To find the total count of views in your model, multiply thenumber of levels for every dimension. For example, a modelwith five dimensions with 3, 5, 2, 1, and 3 levels, respectively,has 3 x 5 x 2 x 1 x 3 for a total of 90 possible views. If the resultis less than 252 or 4,503,599,627,370,496, the model fits. If theresult is more, you must reduce the number of views. Theseare options to reduce the number:

Try to find dimensions that are currently modeled as inde-pendent but could fit the dependent dimension criteria.

Remove some dimensions or attempt to reduce the num-ber of levels in some dimensions.

Change one or more dimensions to be dynamically calcu-lated. Dimensions that contain only dynamically calculatedhierarchies are treated as if they had only one level in thecalculation above. Notice that the dimension marked asAccounts is always dynamically calculated.

Although we tested artificial databases with more than 250

potential views, the most complex real-world model we haveseen so far contained about 1,200,000,000 potential views.

outline memory consumptionThe second thing to consider is whether the number ofmembers in the outline fits the available memory. Notice thaton 32-bit platforms, the amount of memory available to theapplication is limited to 2 to 4 GB (depending on theplatform), even if more RAM is installed on the machine. Also,you may need to set up the system to make as much memoryas possible available to the Essbase server process. Thememory addressability limits are as follows:

On 64-bit platforms, the limit is determined only by thephysical RAM installed.

white paper

2

Operating System

Windows 2000 Advanced Server,2000 Datacenter, 2003

Linux

Solaris

HP-UX

AIX

Default Limit

2 GB

2 GB

3.5 GB

1.75 GB

2 GB

Maximum Limit

3 GB

2 GB

3.5 GB

3.5 GB

3 GB

hyperion.com 3

Of course, if the system does not have that amount ofphysical memory or if there are multiple memory-hungryprocesses running on the system, the actual amount ofmemory that Essbase can use will be lower than the limitindicated in the previous table.

Out of the memory available, roughly 50 MB will be usedby Essbase code and fixed-size data structures. Also, somememory will be required for the ASO cache, which will becovered later in this paper.

The memory requirements of the outline members dependon multiple factors. You can use 150 bytes per member,however, as a rough and somewhat pessimistic estimate.

For a more precise estimate, first you must calculate theaverage memory required for the member name and aliases.This memory requirement for one member can be calculatedas total length of the name and all aliases of this member inbytes, plus (number-of-aliases + 1) x 4 bytes.

The per-member memory amount can then be calculatedas (90+average-memory-for-names) bytes. In addition to that,every member that has a shared copy requires an additional 50bytes. Also, attribute dimensions introduce additionalmemory requirements, which can be calculated as follows:assume that the primary dimension has M members and hasN associated attribute dimensions. Then the memoryrequirement for the attribute mapping will be roughly M x Nx 6 bytes.2

Memory used by the member names can be freed byenabling namespace paging (see the documentation forPRELOADALIASNAMESPACE andPRELOADMEMBERNAMESPACE configurationparameters); however, the performance of the data loads, andto a lesser degree queries, will be impacted. You may alsoconsider using Hybrid Analysis to store the lower level ofdetails of the largest dimension in the relational database.

about building the aso outlineThe process of dimension building in Essbase consists of twostages. The first stage begins with the server opening theexisting outline for editing, using the same outline APIavailable to the client applications. At this point, two copies ofthe existing outline are in memory: one in read-only formatand one in editing format. Then the input data stream isprocessed and new members are added to the new outline.Accordingly, the memory requirement during dimensionbuild should be calculated based on the number of membersequal to (2 x old-number-of-members)+number-of-the-additional-new-members.3

The first stage ends with outline verification, and the newoutline is written down to a separate outline file with an OTNextension.

In the second stage, the new outline is loaded in read-onlymode and the restructuring process takes place. During that

process, both new and old outlines are resident in memory.Also, an additional set of data structures occupyapproximately 25 bytes per member.

It is always best to try and combine as many outlinechanges as possible using incremental dimension build, so thatthere is only one restructuring phase for all changes. If onerestructuring phase is not possible for any reason, it isrecommended that you build the large dimension as the laststep after the small one, so that at no point, two copies of thelarge dimension are in memory.

In most cases, it is much faster to modify an existingoutline with new changes than to build an outline fromscratch.4 When a very large outline already exists in the server,however, it may be impossible to load a small portion of newmembers into it, because there will be no memory for the twocopies of the outline during build and restructuring. It maystill be possible to build the new outline completely, end-to-end from source data.

When modifying a large outline of an existing database, itmay be helpful to restart the application before the build.Because the ASO cache is allocated on demand and not neededduring dimension build, memory used by the ASO cache isfreed for the build and restructuring processing.

To provide a rough idea of dimension build performance,on a test workstation5 a dimension with 3,000,000 memberswas built from a text file in five minutes.6 The largestproduction outline that we are aware of at this point containsabout 10 million members.

When building dimensions with multiple hierarchies andshared members using parent-child data, you will have to keepin mind that ASO applications require that the primarymember appear before its shared copies. This means that theparent-child pairs should be ordered so that the first hierarchyis built completely before the build of the secondary hierarchybegins.

data sizing and the aso cacheThe following information is needed to estimate the amountof disk storage required for the ASO database:7

Number of rows in the input table or text files Number of measure columns in a single row If possible, average number of non-null, nonzero measures

per rowThis number can be approximated by using a sample of

rows. Unless there is a requirement to distinguish betweenzeros and missing values in the database, replacing zeros withmissing values reduces the size of the database.

The size of an ASO database depends on the number ofnon-empty cells, which equals the number of rows multipliedby the average number of non-empty measures per row. Eachcell occupies roughly 10 to 30 bytes of disk space, dependingon the degree of compression.

white paper`

hyperion.com

2

compression dimensionEssbase uses the dimension marked as Accounts to compressthe cells by storing multiple cell values with a single key.Essbase cannot compress data if none of the dimensions aremarked as Accounts, so it is recommended that you alwaysdesignate the compression dimension.

Usually the dimension that corresponds to different valuesin the input rows makes a good choice of compressiondimension. For example, if the values for 12 months come ina single database record and the outline has a Time dimensionwith these 12 months, chances are that Time is a good choicefor the compression.

Often, multiple measures come in a single record instead,and an Accounts tag should be placed on the Measuresdimension. Generally speaking, you should pick the densestdimension. The dimension is a good choice if for everycombination of the members of other dimensions, either noneor all members of the compression dimension have data. Also,Essbase can store as many as 16 cells with a single key, so adense dimension with 10 level zero members will providebetter opportunity for compression than a dimension withonly 2 members.

For a better estimate of the database size, you need to knowthe key length for the database as well as the average numberof cells that can be stored with a single key. The key length inbits is displayed in the ASO database properties. Round thenumber of bits up to the next multiple of 64 and divide by 8to get the number of bytes used by the key. The most commonkey sizes are 8, 16, and 24 bytes.

Assuming that the Accounts dimension in the model iscompletely dense and corresponds to measures in the facttable, the number of cells stored with a single key is simply thenumber of measures. The number of bytes per cell may thenbe calculated as (key-length-in-bytes + 8 x number-of-measures) / number-of-measures.

For example, a database with a typical key length of 16 and6 measures per row of the fact table will use (16 + 8 x 6)/6 orabout 11 bytes per cell. If for half the rows the last 4 measuresare not present, the average number of cells per key will be (2+ 6)/2 = 4, and the number of bytes per cell will be (16 + 8 x4)/4 or 12 bytes.

If none of the dimensions is marked as Accounts, youshould use 1 as the number-of-measures parameter. Thus, inthe example, if the dimension of measures is not marked asAccounts, the space requirement will be (16 + 8 x 1)/1 or 24bytes per cell.

You can sometimes achieve better compression byrearranging the members of the Accounts dimension in theorder of increasing rarity, so that the level zero members thatalways have data occur first in the outline, and the membersthat hardly ever have data occur last.

After the aggregation, the size of the database increases.You have control over the amount of space used by aggregateviews; in most cases, 50 percent or smaller increase in thedatabase size after aggregation provides good queryperformance.

allocation of disk space for aso data filesThe ASO kernel stores data in tablespaces. Two tablespaces areof interest to the server administrator: default and temp.The default tablespace is used to store cube cells, both level-zero and aggregated cells. The temp tablespace is used forintermediate storage of cells during data load, aggregation,and large queries.

Each tablespace contains one or more file locations; thenotion of a file location more or less corresponds to the notionof a volume in the Block Storage kernel. You can change thefile locations of the two tablespaces using applicationproperties in Essbase Administration Services (unlike BlockStorage, ASO tablespaces are logically considered to be a partof the application, not a database).

For every location, you can specify the physical path wherethe data files will be created, the maximum size of eachindividual file, and the total space that all the files at thatlocation are allowed to occupy. Essbase starts creating files inthe second file location of a tablespace only after it exhauststhe disk space or hits the total space limit in the first location.

It is recommended that you allocate space for default andtemp tablespaces on different physical disk devices, ifpossible.

data load performanceData loads from multiple data sources or files should always becombined into a single ASO data load using an ASO loadbuffer. Multiple data loads are accumulated in the buffer andthen moved to the final storage location in a single operation.

For more information on using ASO load buffers fromMAXL scripts, see the documentation of alter database createload_buffer and import database to/from load_bufferMAXL commands. When using the Administration ServicesGUI to load data into the ASO database, multiple data sourcesor files correspond to different lines in the data load dialogbox; Administration Services uses the load bufferautomatically to combine them.

Data load temporarily uses space in the temp tablespace;the amount of space required is about the same as theestimated database size.

ASO cache size has an effect on data load performance. Forsmall databases with 2 million input cells or fewer, the defaultASO cache size of 32 MB is sufficient. For a larger databasewith 20 million cells, 64 or 128 MB cache is more appropriate.For a database with 1 billion cells or more, the cache size maybe set as high as 512 MB or 1 GB if the available memory

white paper

4

hyperion.com 5

permits it. On machines with less than 4 GB of physicalmemory, however, it is usually a good idea to keep ASO cachesize lower than 25 percent of the available physical memory toleave more space for the OS file cache.

Performance of the data load is very much hardware-dependent. The first stage when the data is being loaded intothe buffer is usually CPU-bound, especially when using plaintext or SQL load with load rules. Text substitutions, fieldsplits/joins and white space truncation rules are fairlyexpensive and should be avoided if possible. The buffercommit stage is often IO-bound and may benefit fromallocating the storage for temp and default tablespaces ondifferent physical drives.

To provide an example of data load performance, the dataload of 679 million cells on the test workstation took 63minutes.8 Database size after the load was 9.36 GB. Manyproduction models cross the 1-billion-input-cells mark.

aggregation performanceThe time required to aggregate an ASO database dependsmostly on the database size and the specified amount ofstorage allocated for the aggregate views. The same databasewith 679 million cells was aggregated to the default stoppingsize in 40 minutes,9 to the total database size of 11.0 GB.

By default, ASO uses two threads to perform viewaggregation in parallel whenever possible. You can increase thenumber of parallel threads up to eight by using theCALCPARALLEL setting in the Essbase configuration file. It isrecommended that you increase the parallelism setting to thenumber of CPUs available for the calculation. In some tests,CALCPARALLEL set to eight provided benefits even on fourCPU machines, so setting it higher than the number of CPUscould be worth a try.

Cache size configured for the data load should also providegood aggregation performance; however, when increasing thenumber of calculation threads, you may also need to increasethe cache size, because cache memory will need to be dividedbetween more threads.

Calculation may use temp tablespace for intermediateaggregation results, so allocating temp and defaulttablespaces on different physical devices may help theperformance, although it is of less importance than for thedata load.

query performanceOn a very small database (500 K cells or fewer), queries may bereasonably fast even with no aggregate views. On a larger dataset, it is necessary to create aggregate views to improve queryresponse time. The amount of disk space allocated foraggregate views can be controlled by the administrator. If noamount is specified, Essbase attempts to determine a goodvalue for this parameter; however, notice that this default

setting is only provided as a first estimate for tuning. It isalways a good idea to test query performance for severalaggregate sizes and choose the one that provides the optimalbalance between the aggregation time and the query responsetime for your particular database.

When selecting aggregate views, Essbase assumes that allpossible level combinations in the database are equallyimportant to users and are queried equally often. In mostcases, the selection of aggregate views can be improved byproviding Essbase with information about the actual queriesthat users run by switching to query-based view selection.

An even more important reason to use query-based viewselection is that the standard selection never considers anyviews aggregated along alternative hierarchies. Allaggregations are performed along the first stored hierarchywithin a dimension. In contrast, query-based view selectionwill consider and select views aggregated along alternativehierarchies modeled using shared members, and alternativehierarchies modeled using attribute dimensions,10 as long asthese hierarchies are queried during the test run.

It is recommended that you start from selecting andmaterializing a minimal set of aggregate views using thestandard selection. These views will improve performance oftest queries and will enable the completion of the test queryrun in a reasonable time. Also, these views will help queriesthat were not included in the test run.

After this minimal aggregation is materialized, you canturn on query tracking and run enough queries to representthe common query load for your model. Query tracking has avery low overhead, so it is also possible to deploy the databasein production with query tracking turned on.

When enough queries are recorded for you to consider it arepresentative set, run the selection based on query data andmaterialize the selected views. Query tracking can be turnedon or off by right-clicking the database name inAdministration Services or by using alter databaseenable/disable query_tracking MAXL commands. Theaggregation wizard contains a check box that switches Essbaseto query-based view selection. In MAXL scripts, you can usethe based on query_data clause of the view selection andaggregation commands.

Notice that the query tracking data is lost when you loadadditional cells, materialize any views, or shut down theapplication.

In most cases, the ASO cache configured as describedpreviously in the data load section should work fine forqueries. In most cases, increasing the size of the ASO datacache does not improve query performance more than 10 to15 percent, because the operating system performs its owncaching of files.

One exception is when the ASO kernel needs to storeintermediate results of a query on a disk temporarily, because

white paper`

hyperion.com

2

there is not enough space for the results in the ASO cache. Forthe default 32 MB ASO cache, this happens when the queryretrieves more than about 90,000 non-missing cells.11 Thethreshold number of cells increases proportionally to theconfigured cache size, so for a 64 MB cache, that number willbe approximately 180,000 cells, and so on. If possible, considerincreasing the ASO cache size so that Essbase can store allretrieved cells in memory.

The retrieval buffer size setting has very little effect on theperformance of reports and spreadsheet queries in ASO mode.MDX query performance, however, may still be affected by thesize of the retrieval buffer.

Complexity of formulas on the queried members is amajor factor in query performance. Formulas that need to beexecuted in cell mode are usually expensive to compute;common examples of such formulas are the ones that containIIF expressions or cross-dimensional tuple references. Essbaselists all members with cell-mode formulas in the applicationlog.

Hyperion Visual Explorer uses MDX to retrieve data fromEssbase. To identify queries that take a very long time, you canrefer to the HVE log, usually stored on the client machine, inthe directory %HOMEDRIVE%%HOMEPATH%\MyDocuments\My HVE Repository.12

maintenance of aso applicationsThe following topics describe the operations that usuallyfollow the initial creation in the lifecycle of an ASOapplication.

modification of aso outlinesAny change in the outline causes a restructuring process whenthe outline is saved on the server. In Release 7.1.2, data ispreserved in the following cases:

When a member is renamed or an alias of a member ischanged

When changes are made on a dynamically calculatedmember

Changing attribute associations in any way clears allaggregate views, but the input data is preserved.

When a member is deleted from an ASO outline, the recordfor that member is marked for deletion, but not physicallyremoved from the file. To shrink the outline file after deletinglarge sets of members, you can use the Compact Outlineoperation in the Essbase Administration Services console.

incremental data loadsWhen you load data into an ASO database that already hascells loaded into it, Essbase switches to incremental data loadmode. An incremental data load is executed as a transaction;that is, either all data is loaded to the database, or the databaseremains in the state that it was in before the beginning of thedata load. Essbase merges the incoming data with the existing

data and writes the combined data to a new location withinthe default tablespace. Also, every existing aggregate view isupdated by the incoming data and also written to a newlocation.

At the end of the data load, the space previously occupiedby the old data is marked as free. It is not always possible,however, to return this space to the operating system, so thetotal size of the Essbase data files can be as large as two timesthe old size, plus the size of the incremental data set. Limitingthe file size for the file locations in the default tablespaceincreases the chances that the files can be truncated returningthe space to the operating system.13

Notice that at some point in the incremental data loadprocess, disk space requirements can be as large as two timesthe original size of the ASO database for the defaulttablespace plus three times the size of the incremental datastream (2x in the temp tablespace and 1x in the default). Itis very important to ensure that there is sufficient space on thedisk, because peak disk usage occurs very close to the end ofthe data load process. Performing the data load just to find outif the disk has enough space can be time-consuming.

If you load multiple files without using a load buffer, everystep will be a separate incremental data load, resulting in worsethan possible performance.

reselection of aggregate viewsAggregate view selection can be stored in aggregation scriptsto be used later. As outline changes build up and new data isloaded into the database, however, the selection may graduallybecome obsolete and provide less optimal query performance.At some point, it is a good idea to rerun the selection andreplace the old aggregation scripts with new scripts. You maywant to consider doing this when the increase in the datavolume since the last view selection reaches 20 to 50 percent.

Any outline change that modifies the number of levels inany dimension in the outline invalidates all existing viewselection scripts; an attempt to use an old script on the newoutline results in an error.

BackupBecause text export is not supported for ASO applications, theonly way to back up an ASO database is to make a copy of allapplication and database files when the server is not running.ASO does not support backups of a running application. Iffiles in the application are located outside of the applicationdirectory, the files in those locations must be backed up aswell.

An archive copy of an ASO application can always berestored to a different machine with the same operatingsystem if the application uses only predefined file locationswithin the application directory. You will need to create anASO application and database manually with the same names

white paper

6

hyperion.com

as on the source machine, and then shut down the server andoverride all files in the application directory with the archivecopy.

If the application uses any custom file locations, thedirectory structure on the target machine must be the same ason the source machine. In other words, if on the sourcemachine a custom file location was created on drive E, and onthe target machine there is no drive E, then it is not possible torestore the application to that machine.

Unlike ASO data files, ASO outline files are portable acrossdifferent operating systems and machine architectures.

footnote

1 If the model includes a hierarchy encoded in parent-childtables, determining the number of levels in this hierarchy issomewhat tricky. If the number of levels is not known, theeasiest way to determine it may be to build the hierarchy inEssbase.

2 This formula actually overestimates the memoryrequirements for all cases except when there is an attributedimension with more than 216 members.

3 Strictly speaking, the per-member memory requirements forthe editable and read-only formats differ; however, they areclose enough to be considered equal for the estimationpurposes.

4 Performance of restructuring may be affected when thenumber of children per single parent in the outline is veryhigh. This effect becomes noticeable starting fromapproximately 100,000 members per parent. If the number ofchildren is even higher, it may be faster to rebuild the outlinefrom scratch instead of updating the existing one.

5 IBM Intellistation ZPro, 2x 2.7 GHz Intel CPUs, 1 GB RAM,single 10,000 RPM SCSI hard drive, Windows 2000 AdvancedServer, Essbase 7.1.2, ASO cache set to 256 MB.

6 This result included a 234-second dimension build of asingle large dimension and 26 seconds of restructuring from asmall starting outline.

7 Notice that the size of input text files is almost useless for theestimation of the ASO database size. That is because thenumber of data values stored in a megabyte of text file canvary dramatically depending on the text file format and otherfactors.

8 Tested using export files from a Block Storage database.1810s loading three files into the load buffer, 1965s buffercommit.

9 Not including the time used by the view selection process;aggregation time only. Using the default two-threadaggregation.

10 Even when using query-based view selection, Essbase doesnot materialize views for queries that restrict the values basedon multiple attributes of the same dimension. For example, ifthe dimension Product has attributes Caffeinated andPackage Type, queries such as select total sales for allcaffeinated products may be resolved from a view thatcontains pre-aggregated values for caffeinated-true andcaffeinated-false. In contrast, a query similar to select totalsales for all non-caffeinated products sold in bottles will beresolved dynamically from the lowest level of the Productdimension. If the performance of such queries is stillunacceptable after query-based view selection, the only choiceis to fall back on modeling alternative hierarchies as full-fledged dimensions.

11 This is just an estimate, and the exact threshold may varysignificantly based on the query. To determine whether a givenquery uses temporary on-disk storage, restart the applicationand run the query. If a file is created in the temp tablespace,the query needed to store cells on the disk during execution.

12 HOMEDRIVE and HOMEPATH are environment variablesset for every user by the operating system.

13 On Windows by default, there is no practical limit on the filesize, so you may want to set the limit in file location settings tobe 1 or 2 GB. On UNIX platforms, Essbase always limits thefile size to 2 GB, but you can lower the limit further.

white paper

,

.. . .

Copyright 2005 Hyperion Solutions Corporation. All rights reserved. Hyperion, the Hyperion H logo, and Hyperions product names are trademarks of Hyperion. References toother companies and their products use trademarks owned by the respective companies and are for reference purpose only. 4822_0605