Scalable Real-Time OLAP On Cloud ArchitecturesI

F. Dehnea,∗, Q. Kongb, A. Rau-Chaplinb, H. Zabolia, R. Zhoua

aSchool of Computer Science, Carleton University, Ottawa, Canadab Faculty of Computer Science, Dalhousie University, Halifax, Canada

Abstract

In contrast to queries for on-line transaction processing (OLTP) systems that typically access only a smallportion of a database, OLAP queries may need to aggregate large portions of a database which often leads toperformance issues. In this paper we introduce CR-OLAP, a scalable Cloud based Real-time OLAP systembased on a new distributed index structure for OLAP, the distributed PDCR tree. CR-OLAP utilizes ascalable cloud infrastructure consisting of multiple commodity servers (processors). That is, with increasingdatabase size, CR-OLAP dynamically increases the number of processors to maintain performance. Ourdistributed PDCR tree data structure supports multiple dimension hierarchies and efficient query processingon the elaborate dimension hierarchies which are so central to OLAP systems. It is particularly efficient forcomplex OLAP queries that need to aggregate large portions of the data warehouse, such as “report thetotal sales in all stores located in California and New York during the months February-May of all years”.We evaluated CR-OLAP on the Amazon EC2 cloud, using the TPC-DS benchmark data set. The testsdemonstrate that CR-OLAP scales well with increasing number of processors, even for complex queries. Forexample, for an Amazon EC2 cloud instance with 16 processors, a data warehouse with 160 million tuples,and a TPC-DS OLAP query stream where each query aggregates between 60% and 95% of the database,CR-OLAP achieved a query latency of below 0.3 seconds which can be considered a real time response.

Keywords: Real-time OLAP, Scalability, Cloud Architecture, TPC-DS Benchmark

1. Introduction

On-line analytical processing (OLAP) systemsare at the heart of many business analytics appli-cations. This paper reports on the results of a re-search project (supported by the IBM Centre ForAdvanced Studies Canada) to investigate the useof cloud computing for high performance, scalablereal-time OLAP.

1.1. Background

Decision Support Systems (DSS) are designed toempower the user with the ability to make effec-tive decisions regarding both the current and fu-ture state of an organization. DSS allow users to

IA preliminary (short) version appeared in the conferenceproceeding of IEEE Big Data 2013.

∗Corresponding authorEmail addresses: frank@dehne.net (F. Dehne),

qkong@cs.dal.ca (Q. Kong), arc@cs.dal.ca(A. Rau-Chaplin), zaboli@graduate.org (H. Zaboli),xiaoyunzhou@cmail.carleton.ca (R. Zhou)

study relationships in a chronological context be-tween things such as customers, vendors, products,inventory, geography, and sales. One of the mostpowerful and prominent technologies for knowledgediscovery in DSS environments is on-line analyt-ical processing (OLAP). OLAP is the foundationfor a wide range of essential business applications,including sales and marketing analysis, planning,budgeting, and performance measurement [1, 2].By exploiting multi-dimensional views of the un-derlying data warehouse, the OLAP server allowsusers to “drill down” or “roll up” on dimension hi-erarchies, “slice and dice” particular attributes, orperform various statistical operations such as rank-ing and forecasting. To support this functionality,OLAP relies heavily upon a classical data modelknown as the data cube [3] which allows users toview organizational data from different perspectivesand at a variety of summarization levels. It con-sists of the base cuboid, the finest granularity viewcontaining the full complement of d dimensions (or

Preprint submitted to Elsevier May 6, 2014

attributes), surrounded by a collection of 2d−1 sub-cubes/cuboids that represent the aggregation of thebase cuboid along one or more dimensions.

In contrast to queries for on-line transaction pro-cessing (OLTP) systems which typically access onlya small portion of the database (e.g. update a cus-tomer record), OLAP queries may need to aggre-gate large portions of the database (e.g. calculatethe total sales of a certain type of items duringa certain time period) which may lead to perfor-mance issues. Therefore, most of the traditionalOLAP research, and most of the commercial sys-tems, follow the static data cube approach proposedby Gray et al. [3] and materialize all or a subsetof the cuboids of the data cube in order to ensureadequate query performance. Building the datacube can be a massive computational task, and sig-nificant research has been published on sequentialand parallel data cube construction methods (e.g.[4, 5, 3, 6, 7, 8]). However, the traditional staticdata cube approach has several disadvantages. TheOLAP system can only be updated periodically andin batches, e.g. once every week. Hence, latest in-formation can not be included in the decision sup-port process. The static data cube also requiresmassive amounts of memory space and leads toa duplicate data repository that is separate fromthe on-line transaction processing (OLTP) systemof the organization. Practitioners have thereforecalled for some time for an integrated OLAP/OLTPapproach with a real-time OLAP system that getsupdated instantaneously as new data arrives andalways provides an up-to-date data warehouse forthe decision support process (e.g. [9]). Some recentpublications have begun to address this problemby providing “quasi real-time” incremental main-tenance schemes and loading procedures for staticdata cubes (e.g. [9, 10, 11, 12]). However, these ap-proaches are not fully real-time. A major obstacleare significant performance issues with large scaledata warehouses.

1.2. Contributions

The aim of our research project is to help ad-dress the above mentioned performance issues forreal-time OLAP systems through the use of efficientparallel computing methods. In a recent paper [13]we presented the first parallel real-time OLAP sys-tem designed to be executed on a multi-core pro-cessor. We documented significant performance in-creases with increasing number of processor cores.Our system won the 2012 IBM Canada Research

Impact Of The Year Award and an IBM sponsoredpatent application has been submitted. In this pa-per, we report on the next phase of our project:to scale up our real-time OLAP system to utilizea collection of (m + 1) multi-core processors in ascalable cloud environment.

We introduce CR-OLAP, a scalable Cloud basedReal-time OLAP system that utilizes a new dis-tributed index structure for OLAP, refered to as adistributed PDCR tree. This data structure is notjust another distributed R-tree, but rather a multi-dimensional data structure designed specifically tosupport efficient OLAP query processing on theelaborate dimension hierarchies that are central toOLAP systems. The distributed PDCR tree, basedon the sequential DC tree introduced by Kriegelet al. [14] and our previous PDC tree [13], ex-ploits knowledge about the structure of individualdimension hierarchies both for compact data rep-resentation and accelerated query processing. Thefollowing is a brief overview of the properties of oursystem.

Consider a d-dimensional data warehouse withd dimension hierarchies. CR-OLAP supports aninput stream consisting of insert and query oper-ations. Each OLAP query can be represented asan aggregate range query that specifies for each di-mension either a single value or range of values atany level of the respective dimension hierarchy, or asymbol “*” indicating the entire range for that di-mension. CR-OLAP utilizes a cloud infrastructureconsisting of m+1 multi-core processors where eachprocessor executes up to k parallel threads. As typi-cal for current high performance databases, all datais kept in the processors’ main memories [15]. Withincreasing database size, CR-OLAP will increasem by dynamically allocating additional processorswithin the cloud environment and re-arranging thedistributed PDCR tree. This will ensure that both,the available memory and processing capability willscale with the database size. One of the m+1 multi-core processors is referred to as the master, and theremaining m processors are called workers. Themaster receives from the users the input stream ofOLAP insert and query operations, and reports theresults back to the users (in the form of referencesto memory locations where the workers have de-posited the query results). In order to ensure highthroughput and low latency even for compute inten-sive OLAP queries that may need to aggregate largeportions of the entire database, CR-OLAP utilizesseveral levels of parallelism: distributed processing

2

of multiple query and insert operations among mul-tiple workers, and parallel processing of multipleconcurrent query and insert operations within eachworker. For correct query operation, CR-OLAP en-sures that the result for each OLAP query includesall data inserted prior but no data inserted afterthe query was issued within the input stream.

CR-OLAP is supported by a new distributed in-dex structure for OLAP termed distributed PDCRtree which supports distributed OLAP query pro-cessing, including fast real-time data aggregation,real-time querying of multiple dimension hierar-chies, and real-time data insertion. Note that, sinceOLAP is about the analysis of historical data collec-tions, OLAP systems do usually not support datadeletion. Our system does however support bulkinsert operations of large groups of data items.

The distributed index structure consists of a col-lection of PDCR trees whereby the master storesone PDCR tree (called hat) and each worker storesmultiple PDCR trees (called subtrees). Each indi-vidual PDCR tree supports multi-core parallelismand executes multiple concurrent insert and queryoperations at any point in time. PDCR trees are anon-trivial modification of the authors’ previouslypresented PDC trees [13], adapted to the cloudenvironment and designed to scale. For example,PDCR trees are array based so that they can easilybe compressed and transferred between processorsvia message passing. When the database grows andnew workers are added, sub-trees are split off andsent to the new worker.

We evaluated CR-OLAP on the Amazon EC2cloud for a multitude of scenarios (different ratiosof insert and query transactions, query transac-tions with different sizes of results, different sys-tem loads, etc.), using the TPC-DS “Decision Sup-port” benchmark data set. The tests demonstratethat CR-OLAP scales well with increasing num-ber of workers. For example, for fixed data ware-house size (10,000,000 data items), when increas-ing the number of workers from 1 to 8, the av-erage query throughput and latency improves bya factor 7.5. When increasing the data warehousesize from 10,000,000 data items to 160,000,000 dataitems while, at the same time, letting CR-OLAPincrease the number of workers used from 1 to 16,respectively, we observed that query performanceremained essentially unchanged. That is, the sys-tem performed an 16-fold increase in size, includingan 16-fold increase in the average amount of dataaggregated by each OLAP query, without notice-

able performance impact for the user.A particular strength of CR-OLAP is to effi-

ciently answer queries with large query coverage,i.e. the portion of the database that needs tobe aggregated for an OLAP query. For exam-ple, for an Amazon EC2 cloud instance with 16processors, a data warehouse with 160 million tu-ples, and a TPC-DS OLAP query stream whereeach query aggregates between 60% and 95% ofthe database, CR-OLAP achieved a query latencyof below 0.3 seconds which can be considered areal time response. CR-OLAP also handles wellincreasing dimensionality of the data warehouse.For tree data structures this is a critical issue asit is known e.g. for R-trees that, with increasingnumber of dimensions, even simple range search(no dimension hierarchies, no aggregation) can de-generate to linear search (e.g. [16]). In our ex-periments, we observed that increasing number ofdimensions does not significantly impact the per-formance of CR-OLAP. Another possible disadvan-tage of tree data structures is that they are po-tentially less cache efficient than in-memory linearsearch which can make optimum use of streamingdata between memory and processor caches. Toestablish a comparison baseline for CR-OLAP, weimplemented STREAM-OLAP which partitions thedatabase between multiple cloud processors basedon one chosen dimension and uses parallel memoryto cache streaming on the cloud processors to an-swer OLAP queries. We observed that the perfor-mance of CR-OLAP is similar to STREAM-OLAPfor simple OLAP queries with small query coveragebut that CR-OLAP vastly outperforms STREAM-OLAP for more complex queries that utilize differ-ent dimension hierarchies and have a larger querycoverage (e.g. “report the total sales in all storeslocated in California and New York during themonths February-May of all years”).

The remainder of this paper is organized as fol-lows. In Section 2 we review related work. In Sec-tion 3 we introduce the PDCR tree data structureand in Section 4 we present our CR-OLAP systemfor real-time OLAP on cloud architectures. Section5 shows the results of an experimental evaluation ofCR-OLAP on the Amazon EC2 cloud, and Section6 concludes the paper.

2. Related Work

In addition to the related work discussed inthe introduction, there are many efforts to store

3

and query large data sets in cloud environments.Hadoop[17] and its file system, HDFS, are popularexamples of such systems which are typically builton MapReduce [18]. Related projects most simi-lar to our work are Hive[19] and HadoopDB[20].However, these systems are not designed for real-time (OLTP style) operation. Instead, they usebatch processing similar to [9, 10, 11, 12]. Thesituation is similar for BigTable[21], BigQuery[22],and Dremel[23]. In fact, Dremel[23] uses a colum-nar data representation scheme and is designed toprovide data warehousing and querying supportfor read-only data. To overcome the batch pro-cessing in Hadoop based systems, Storm [24] in-troduced a distributed computing model that pro-cesses in-flight Twitter data. However, Storm as-sumes small, compact Twitter style data packetsthat can quickly migrate between different comput-ing resources. This is not possible for large datawarehouses.

SAP HANA is a real time in-memory databasesystem that also supports OLAP queries. In a cloudcomputing environment, a basic HANA instance isfor a multi-core processor single compute node. Ascale out version of HANA can be executed on mul-tiple compute nodes, using a distributed file system(GPFS) that provides a single shared data view toall compute nodes. Large tables can be partitionedusing various partitioning criteria and complete ta-bles or parts thereof can then be assigned to differ-ent nodes. The execution engine schedules queriesover the different compute nodes and attempts toexecute them on the node that holds the data [25].

For peer-to-peer networks, related work includesdistributed methods for querying concept hierar-chies such as [26, 27, 28, 29]. However, none ofthese methods provides real-time OLAP function-ality. There are various publications on distributedB-trees for cloud platforms such as [30]. However,these method only supports 1-dimensional indiceswhich are insufficient for OLAP queries. There havebeen efforts to build distributed multi-dimensionalindices on Cloud platforms based on R-trees orrelated multi-dimensional tree structures, such as[31, 32, 33]. However, these method do not supportdimension hierarchies which are essential for OLAPqueries.

3. PDCR Trees

Consider a data warehouse with fact table F anda set of d dimensions {D1, D2, ..., Dd} where each

dimension Di, 1 ≤ i ≤ d has a hierarchy Hi in-cluding hierarchical attributes corresponding to thelevels of the hierarchy. The hierarchical attributesin the hierarchy of dimension i are organized asan ordered set Hi of parent-child relationships inthe hierarchy levels Hi = {Hi1, Hi2, ...,Hil} wherea parent logically summarizes and includes its chil-dren. Figure 1 shows the dimensions and hierarchylevels of each dimension for a 4-dimensional datawarehouse.

All Dims

ItemCustomerStore Date

Country

State

City

BYear

BMonth

BDay

Category

Class

Brand

Year

Month

Day

Figure 1: A 4-dimensional data warehouse with 3 hierarchylevels for each dimension. The first box for each dimensiondenotes the name of the dimension.

The sequential DC tree introduced by Kriegel etal. [14] exploits knowledge about the structure ofindividual dimension hierarchies both for compactdata representation and accelerated OLAP queryprocessing. In our previous work [13], we intro-duced the PDC tree, a parallel DC tree for a singlemulti-core processor. In this section, we outline amodified PDC tree, termed PDCR tree, which willbecome the building block for our CR-OLAP sys-tem. Here, we only outline the differences betweenthe PDCR tree and its predecessors, and we refer to[13, 14] for more details. We note that, our PDCRtree data structure is not just another distributedR-tree, but rather a multi-dimensional data struc-ture designed specifically to support efficient OLAPquery processing on the elaborate dimension hierar-chies that are central to OLAP systems. Also notethat, DC tree [14] is particularly well suited to han-dle OLAP queries for both, ordered and unordereddimensions.

For a cloud architecture with multiple proces-sors, each processor will store one or more PDCRtrees. Our CR-OLAP system outlined in the follow-ing Section 4 requires that a sub-tree of a PDCRtree can be split off and transferred to another pro-cessor. This required us to (a) devise an arraybased tree implementation that can be packed into

4

a message to be sent between processors and (b)a careful encoding of data values, using compactIDs related to the different dimension hierarchy lev-els. For our array based PDCR tree implementa-tion, a PDC tree is represented in a single arraywhere all tree links are represented by integer ref-erences to array locations. Allocation of new treenodes was re-implemented as an append operationat the end of the array, and all tree operations werere-implemented to use integer references instead ofmemory pointers. In the following we outline somedetails of the encoding of data values used for ourPDCR tree.

IDs for each dimension represent available enti-ties in the dimension. Each dimension has a hierar-chy of entities with l levels. In the example of Fig-ure 1, an ID may represent an entity at the Countrylevel for the Store dimension, e.g. US or Canada.Similarly, another ID may represent an entity at theCity level, e.g. Chicago or Toronto. It is importantto note that an ID may summarize many IDs atlower hierarchy levels. To build an ID for a dimen-sion with l levels, we assign bj bits to the hierarchylevel j, 0 ≤ j ≤ l − 1. Different entities at each hi-erarchy level are assigned numerical values startingwith “1”. By concatenating the numerical values ofthe levels, a numerical value is created. We reservethe value zero to represent “All” or “*”. The exam-ple in Figure 2 shows an example of an entity atthe lowest hierarchy level of dimension Store. AnID for the state California will have a value of zerofor its descendant levels City and Store S key. Asa result, containment of IDs between different hi-erarchy levels can be tested via fast bit operations.Figure 3 illustrates IDs and their coverages in theStore dimension with respect to different hierarchylevels. As illustrated, each entity in level j (Coun-try) is a country specified by a numerical value andcovers cities that are represented using numericalvalues in level j + 1. Note that IDs used for citieswill have specific values at the city level, while theID of a country will have a value of zero at the citylevel and a specific value only at the country level.

The sequential DC tree introduced by Kriegelet al. [14] and our previous PDC tree [13] storeso called “minimum describing set” (MDS) entriesat each internal tree node to guide the query pro-cess; see [14] for details. The MDS concept wasdeveloped in [14] to better represent unordered di-mensions with dimension hierarchies. Experimentswith our CR-OLAP system showed that in a largercloud computing environment with multiple tree

data structures, the number of MDS entries be-comes very large and unevenly distributed betweenthe different trees, leading to performance bottle-necks. On the other hand, the bit representation ofIDs outlined above gives us the opportunity to con-vert unordered dimensions into ordered dimensions,and then use traditional ranges instead of the MDSentries. An example is shown in Figure 4. Theranges lead to a much more compact tree storageand alleviated the above mentioned bottleneck. Itis important to note that, this internal ordering im-posed on dimensions is invisible to the user. OLAPqueries can still include unordered aggregate val-ues on any dimension such as “Total sales in theUS and Canada” or “Total sales in California andNew York”.

Level 0 Level 1 Level 2 . . . Level l

Store US California Los Angels Store S_key

01 01 1001 101 1011011

b0 bits b1 bits b2 bits bl bits

Figure 2: Illustration of the compact bit representation ofIDs.

1 2 3Country

City

Store S_key

1 2 3 1 2

123 4 5 6 7 8 9 10 11-13 14

HierarchyLevels

All

Figure 3: Example of relationships between different hierar-chy levels of a given dimension.

Root1990-2000US - Canada

2000-2013US

2000-2013UK - Germany

. . .

C1

1995-2000California - Virginia

1993-2000Canada

C2

2000-2010New York -Texas

2000-2013California

Cn

2000-2005UK

2004-2013Germany

. . . . . . . . .

Figure 4: Example of a PDCR tree with 2 dimensions (Storeand Date).

5

4. CR-OLAP: Cloud based Real-time OLAP

CR-OLAP utilizes a cloud infrastructure consist-ing of m + 1 multi-core processors where each pro-cessor executes up to k parallel threads. One ofthe m+1 multi-core processors is referred to as themaster, and the remaining m processors are calledworkers. The master receives from the users theinput stream of OLAP insert and query operations,and reports the results back to the users (in theform of references to memory locations where theworkers have deposited the query results). In or-der to ensure high throughput and low latency evenfor compute intensive OLAP queries that may needto aggregate large portions of the entire database,CR-OLAP utilizes several levels of parallelism: dis-tributed processing of multiple query and insertoperations among multiple workers, and parallelprocessing of multiple concurrent query and insertoperations within each worker. With increasingdatabase size, CR-OLAP will increase m by dynam-ically allocating additional processors within thecloud environment and re-arranging the distributedPDCR tree. This will ensure that both, the avail-able memory and processing capability will scalewith the database size.

We start by outlining the structure of a dis-tributed PDC tree and PDCR tree on m + 1 multi-core processors in a cloud environment. Considera single PDCR tree T storing the entire database.For a tunable depth parameter h, we refer to thetop h levels of T as the hat and we refer to the re-maining trees rooted at the leaves of the hat as thesubtrees s1, . . . , sn. Level h is referred to as the cutlevel. The hat will be stored at the master and thesubtrees s1, . . . , sn will be stored at the m workers.We assume n ≥ m and that each worker stores oneor more subtrees.

CR-OLAP starts with an empty database andone master processor (i.e. m = 0) storing an emptyhat (PDCR tree). Note that, DC trees [14], PDCtrees [13] and PDCR trees are leaf oriented. Alldata is stored in leafs called data nodes. Internalnodes are called directory nodes and contain ar-rays with routing information and aggregate val-ues. Directory nodes have a high capacity and fan-out of typically 10 - 20. As insert operations aresent to CR-OLAP, the size and height of the hat(PDCR tree) grows. When directory nodes of thehat reach height h, their children become roots atsubtrees stored at new worker nodes that are allo-cated through the cloud environment. An illustra-

tion of such a distributed PDCR tree is shown inFigure 5.

Insertion

S1 S2 S3 S4

Query

Hat

Leaf Node

Figure 5: Illustration of a distributed PDCR tree.

For a typical database size, the hat will usuallycontain only directory nodes and all data will bestored in the subtrees s1, . . . , sn. After the initialset of data insertions, all leaf nodes in the hat willusually be directory nodes of height h, and the rootsof subtrees in workers will typically be directorynodes as well. As illustrated in Figure 5, both insertand query operations are executed concurrently.

4.1. Concurrent insert and query operations

Each query operation in the input stream ishanded to the master which traverses the hat. Notethat, at each directory node the query can gen-erate multiple parallel threads, depending on howmany child nodes have a non empty intersectionwith the query. Eventually, each query will accessa subset of the hat’s leaves, and then the query willbe transferred to the workers storing the subtreesrooted at those leaves. Each of those workers willthen in parallel execute the query on the respectivesubtrees, possibly generating more parallel threadswithin each subtree. For more details see Algo-rithm 3 and Algorithm 4.

For each insert operation in the input stream, themaster will search the hat, arriving at one of the leafnodes, and then forward the insert operation to theworker storing the subtree rooted at that leaf. Formore details see Algorithm 1 and Algorithm 2.

Figures 6 and 7 illustrate how new workers andnew subtrees are added as more data items get in-serted. Figures 6 illustrates insertions creating anoverflow at node A, resulting in a horizontal splitat A into A1 and A2 plus a new parent node C.Capacity overflow at C then triggers a vertical split

6

Algorithm 1: Hat Insertion

Input: D (new data item).Output: voidInitialization:Set ptr = root

Repeat:Determine the child node C of ptr that causesminimal MBR/MDS enlargement for thedistributed PDCR/PDC tree if D is insertedunder C. Resolve ties by minimal overlap, thenby minimal number of data nodes.Set ptr = C.Acquire a LOCK for C.Update MBR/MDS and TS of C.Release the LOCK for C.Until: ptr is a leaf node.if ptr is the parent of Data Nodes then

Acquire a LOCK for ptr.Insert D under ptr.Release the LOCK for C.if capacity of ptr is exceeded then

Call Horizontal Split for ptr.if capacity of the parent of ptr isexceeded then

Call Vertical Split for the parent ofptr.if depth of ptr is greater than hthen

Create a new subtree with theparent of ptr as its root, ptr andits sibling node as the childrenof the root.Choose the next availableworker and update the list ofsubtrees in the master.Send the new subtree and itsdata nodes to the chosen worker.

end

end

end

endif ptr is the parent of a subtree then

Find the worker that contains the subtreefrom the list of subtrees.Send the insertion transaction to theworker.

endEnd of Algorithm.

Algorithm 2: Subtree Insertion

Input: D (new data item).Output: voidInitialization:Set ptr = root

Repeat:Determine the child node C of ptr that causesminimal MBR/MDS enlargement for thedistributed PDCR/PDC tree if D is insertedunder C. Resolve ties by minimal overlap, thenby minimal number of data nodes.Set ptr = C.Acquire a LOCK for C.Update MBR/MDS and TS of C.Release the LOCK for C.Until: ptr is a leaf node.Acquire a LOCK for ptr.Insert D under ptr.Release the LOCK for C.if capacity of ptr is exceeded then

Call Horizontal Split for ptr.if capacity of the parent of ptr is exceededthen

Call Vertical Split for the parent of ptr.end

endEnd of Algorithm.

7

Algorithm 3: Hat Query

Input: Q (OLAP query).Output: A result set or an aggregate valueInitialization:Set ptr = rootPush ptr into a local stack S for query Q.

Repeat:Pop the top item ptr′ from stack S.if TS(time stamp) of ptr′ is smaller (earlier)than the TS of ptr then

Using the sibling links, traverse the siblingnodes of ptr until a node with TS equal tothe TS of ptr is met. Push the visitednodes including ptr into the stack (startingfrom the rightmost node) for reprocessing.

endfor each child C of ptr do

if MBR/MDS of C is fully contained inMBR/MDS of Q then

Add C and its measure value to theresult set.

endelse

if MBR/MDS of C overlapsMBR/MDS of Q then

if C is the root of a sub-tree thenSend the query Q to the workerthat contains the subtree.

endelse

Push C into the stack S.end

end

end

endUntil: stack S is empty.if the query Q is dispatched to a subtree then

Wait for the partial results of thedispatched queries from workers.Create the final result of the collectedpartial results.Send the final result back to the client.

endEnd of Algorithm.

Algorithm 4: Subtree Query

Input: Q (OLAP query).Output: A result set or an aggregate valueInitialization:Set ptr = rootPush ptr into a local stack S for query Q.

Repeat:Pop the top item ptr′ from stack S.if TS(time stamp) of ptr′ is smaller (earlier)than the TS of ptr then

Using the sibling links, traverse the siblingnodes of ptr until a node with TS equal tothe TS of ptr is met. Push the visitednodes including ptr into the stack startingfrom the rightmost node for reprocessing.

endfor each child C of ptr do

if MBR/MDS of C is fully contained inMBR/MDS of Q then

Add C and its measure value to theresult set.

endelse

if MBR/MDS of C overlapsMBR/MDS of Q then

Push C into the stack S.end

end

endUntil: stack S is empty.Send the result back to the master or clientdepending on whether Q is an aggregationquery or a data report query.End of Algorithm.

8

illustrated in 7. This creates two subtrees in two dif-ferent workers. As outlined in more details in theCR-OLAP “migration strategies” outlined below,new workers are requested from the cloud environ-ment when either new subtrees are created or whensubtree sizes exceed the memory of their host work-ers. Workers usually store multiple subtrees. How-ever, CR-OLAP randomly shuffles subtrees amongworkers. This ensures that query operations access-ing a contiguous range of leaf nodes in the hat createa distributed workload among workers.

R

A BCut

level

c = 2

R

C B

A1 A2

(a) (b)

Data Node

Directory Node

Figure 6: Insertions triggering creation of new workers andsubtrees. Part 1. (a) Current hat configuration. (b) Inser-tions create overflow at node A and horizontal split.

R

C B

A1 A2

Subtree 1

Subtree 2

R

C B

D A2

Worker 1

A1 A3

Worker 2

(a) (b)

Figure 7: Insertions triggering creation of new workers andsubtrees. Part 2. (a) Same as Figure 6b with critical subtreeshighlighted. (b) Insertions create overflow at node C andvertical split, triggering the creation of two subtrees in twodifferent workers.

For correct real time processing of an inputstream of mixed insert and query operations, CR-OLAP needs to ensure that the result for eachOLAP query includes all data inserted prior butno data inserted after the query was issued withinthe input stream. We will now discuss how this is

achieved in a distributed cloud based system wherewe have a collection of subtrees in different work-ers, each of which is processing multiple concurrentinsert and query threads. In our previous work [13]we presented a method to ensure correct query pro-cessing for a single PDC tree on a mutli-core pro-cessor, where multiple insert and query operationsare processed concurrently. The PDC tree main-tains for each data or directory item a time stampindicating its most recent update, plus it maintainsfor all nodes of the same height a left-to-right linkedlist of all siblings. Furthermore, each query threadmaintains a stack of ancestors of the current nodeunder consideration, together with the time stampsof those items. We refer to [13] for more details.The PDCR tree presented in this paper inheritsthis mechanism for each of its subtrees. In fact, theabove mentioned sibling links are shown as horizon-tal links in Figures 6 and 7. With the PDCR treebeing a collection of subtrees, if we were to main-tain sibling links between subtrees to build linkedlist of siblings across all subtrees then we would en-sure correct query operation in the same way as forthe PDC tree [13]. However, since different subtreesof a PDCR tree typically reside on different work-ers, a PDCR tree only maintains sibling links insidesubtrees but it does not maintain sibling links be-tween different subtrees. The following lemma showthat correct real time processing of mixed insert andquery operations is still maintained.

Theorem 1. Consider the situation depicted inFigure 8 where the split of node B created a newnode D and subtree rooted at D that is stored sepa-rately from the subtrees rooted at A and B. Then,the sibling links labelled “a” and “c” are not re-quired for correct real time query processing (as de-fined above).

Proof. Assume a thread for a query Q that isreturning from searching the subtree below B onlyto discover that B has been modified. Let Bstack

be the old value of B that is stored in the stackstack(Q) associated with Q. If neither B nor anyancestor of B are in stack(Q) then Q does not con-tain any data covered by B. Otherwise, Q will fol-low the sibling link labelled “b” to find B′ and re-maining data from the node split off B.

4.2. Load balancing

CR-OLAP is executed on a cloud platform with(m+1) processors (m workers and one master). Asdiscussed earlier, CR-OLAP uses the cloud’s elas-ticity to increase m as the number of data items in-

9

P

A CD

B B’

a

b

c

Figure 8: Illustration for Theorem 1.

creases. We now discuss in more detail CR-OLAP’smechanisms for worker allocation and load balanc-ing in the cloud. The insert operations discussedabove create independent subtrees for each height hleaf of the hat. Since internal (directory) nodes havea high degree (typically 10 - 20), a relatively smallheight of the hat typically leads to thousands ofheight h leaves and associated subtrees s1, . . . , sn.The master processor keeps track of the subtree lo-cations and allocation of new workers, and it makessure that a relatively high n/m ratio is maintained.

As indicated above, CR-OLAP shuffles thesen >> m subtrees among the m workers. This en-sures that threads of query operations are evenlydistributed over the workers. Furthermore, CR-OLAP performs load balancing among the workersto ensure both, balanced workload and memory uti-lization. The master processor keeps track of thecurrent sizes and number of active threads for allsubtrees. For each worker, its memory utilizationand workload are the total number of threads of itssubtrees and the total size if its subtrees, respec-tively.

If a worker w has a memory utilization above acertain threshold (e.g. 75% of its total memory),then the master processor determines the workerw′ with the lowest memory utilization and checkswhether it is possible to store an additional subtreefrom w while staying well below it’s memory thresh-old (e.g. 50% of its total memory). If that is notpossible, a new worker w′ is allocated within thecloud environment. Then, a subtree from w is com-pressed and sent from w to w′ via message passing.As discussed earlier, PDCR trees are implementedin array format and using only array indices aspointers. This enables fast compression and decom-pression of subtrees and greatly facilitates subtreemigration between workers. Similarly, if a worker w

has a workload utilization that is a certain percent-age above the average workload of the m workersand is close to the maximum workload thresholdfor a single worker, then the master processor de-termines a worker w′ with the lowest workload andwell below its maximum workload threshold. If thatis not possible, a new worker w′ is allocated withinthe cloud environment. Then, the master processorinitiates the migration of one or more subtrees fromw (and possibly other workers) to w′.

Note that, in case workers are under-utilized dueto shrinking workload, the above process can eas-ily be reversed to decrease the number of work-ers. However, since the emphasis of our study ison growing system size for large scale OLAP, thiswas not implemented in our prototype system.

5. Experimental Evaluation On AmazonEC2

5.1. Software

CR-OLAP was implemented in C++, using theg++ compiler, OpenMP for multi-threading, andZeroMQ [34] for message passing between proces-sors. Instead of the usual MPI message passing li-brary we chose ZeroMQ because it better supportscloud elasticity and the addition of new processorsduring runtime. CR-OLAP has various tunable pa-rameters. For our experiments we set the depth hof the hat to h = 3, the directory node capacity cto c = 10 for the hat and c = 15 for the subtrees,and the number k of threads per worker to k = 16.

5.2. Hardware/OS

CR-OLAP was executed on the Amazon EC2cloud. For the master processor we used an AmazonEC2 m2.4xlarge instance: “High-Memory Quadru-ple Extra Large” with 8 virtual cores (64-bit ar-chitecture, 3.25 ECUs per core) rated at 26 com-pute units and with 68.4 GiB memory. For theworker processors we used Amazon EC2 m3.2xlargeinstances: “M3 Double Extra Large” with 8 vir-tual cores (64-bit architecture, 3.25 ECUs per core)rated at 26 compute units and with 30 GiB mem-ory. The OS image used was the standard AmazonCentOS (Linux) AMI.

5.3. Comparison baseline: STREAM-OLAP

As outlined in Section 2, there is no compari-son system for CR-OLAP that provides scalable

10

cloud based OLAP with full real time capabil-ity and support for dimension hierarchies. To es-tablish a comparison baseline for CR-OLAP, wetherefore designed and implemented a STREAM-OLAP method which partitions the database be-tween multiple cloud processors based on one cho-sen dimension and uses parallel memory to cachestreaming on the cloud processors to answer OLAPqueries. More precisely, STREAM-OLAP buildsa 1-dimensional index on one ordered dimensiondstream and partitions the data into approx. 100×m arrays. The arrays are randomly shuffled be-tween the m workers. The master processor main-tains the 1-dimensional index. Each array repre-sents a segment of the dstream dimension and isaccessed via the 1-dimensional index. The arraysthemselves are unsorted, and insert operations sim-ply append the new item to the respective array.For query operations, the master determines via the1-dimensional index which arrays are relevant. Theworkers then search those arrays via linear search,using memory to cache streaming.

The comparison between CR-OLAP (usingPDCR trees) and STREAM-OLAP (using a 1-dimensional index and memory to cache streaming)is designed to examine the tradeoff between a so-phisticated data structure which needs fewer dataaccesses but is less cache efficient and a brute forcemethod which accesses much more data but opti-mizes cache performance.

5.4. Test data

For our experimental evaluation of CR-OLAPand STREAM-OLAP we used the standard TPC-DS “Decision Support” benchmark for OLAP sys-tems [35]. We selected “Store Sales”, the largestfact table available in TPC-DS. For the remain-der, the database size N refers to the number ofdata items from “Store Sales” that were insertedinto the database. Figure 9 shows the fact table’s 8dimensions, and the respective 8 dimension hierar-chies below each dimension. The first box for eachdimension denotes the dimension name while theboxes below denote hierarchy levels from highest tolowest. Dimensions Store, Item, Address, and Pro-motion are unordered dimensions, while dimensionsCustomer, Date, Household and Time are ordered.TPC-DS provides a stream of insert and query oper-ations on “Store Sales” which was used as input forCR-OLAP and STREAM-OLAP. For experimentswhere we were interested in the impact of querycoverage (the portion of the database that needs

to be aggregated for an OLAP query), we selectedsub-sequences of TPC-DS queries with the chosencoverages.

All Dims

ItemCustomerStore TimePromotionHouseholdDate Address

Country

Ordered Ordered

Ordered

Ordered

State

City

BYear

BMonth

BDay

Category

Class

Brand

Year

Month

Day

Country

State

City

Income Band

Name Hour

Minute

Figure 9: The 8 dimensions of the TPC-DS benchmark forthe fact table “Store Sales”. Boxes below each dimensionspecify between 1 and 3 hierarchy levels for the respectivedimension. Some dimensions are “ordered” and the remain-ing are not ordered.

5.5. Test results: impact of the number of workers(m) for fixed database size (N)

We tested how the time of insert and query oper-ations for CR-OLAP and STREAM-OLAP changesfor fixed database size (N) as we increase the num-ber of workers (m). Using a variable number ofworkers 1 ≤ m ≤ 8, we first inserted 40 millionitems (with d=8 dimensions) from the TPC-DSbenchmark into CR-OLAP and STREAM-OLAP,and then we executed 1,000 (insert or query) op-erations on CR-OLAP and STREAM-OLAP. Sinceworkers are virtual processors in the Amazon EC2cloud, there is always some performance fluctuationbecause of the virtualization. We found that the to-tal (or average) of 1,000 insert or query operationsis a sufficiently stable measure. The results of ourexperiments are shown in Figures 10, 11, and 12.

0

0.05

0.1

0.15

0.2

1 2 4 6 8

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Number of workers(m)

PDCR-tree1D-index

Figure 10: Time for 1000 insertions as a function of thenumber of workers. (N = 40Mil, d = 8, 1 ≤ m ≤ 8)

11

0

200

400

600

800

1000

1200

1400

1 2 4 6 8

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Number of workers(m)

PDCR-tree(10% coverage)1D-index(10% coverage)

PDCR-tree(60% coverage)1D-index(60% coverage)

PDCR-tree(95% coverage)1D-index(95% coverage)

Figure 11: Time for 1000 queries as a function of the numberof workers. (N = 40Mil, d = 8, 1 ≤ m ≤ 8)

0

1

2

3

4

5

6

7

8

1 2 4 6 8

Spee

dup

Number of workers(m)

Y=XPDCR-tree(10% coverage)1D-index(10% coverage)

PDCR-tree(60% coverage)1D-index(60% coverage)

PDCR-tree(95% coverage)1D-index(95% coverage)

Figure 12: Speedup for 1000 queries as a function of thenumber of workers. (N = 40Mil, d = 8, 1 ≤ m ≤ 8)

Figure 10 shows the time for 1,000 inser-tions in CR-OLAP (PDCR-tree) and STREAM-OLAP (1D-index) as a function of the numberof workers (m). As expected, insertion times inSTREAM-OLAP are lower than in CR-OLAP be-cause STREAM-OLAP simply appends the newitem in the respective array while CR-OLAP has toperform tree insertions with possible directory nodesplits and other overheads. However, STREAM-OLAP shows no speedup with increasing number ofworkers (because only one worker performs the ar-ray append operation) whereas CR-OLAP shows asignificant speedup (because the distributed PDCRtree makes use of the multiple workers). It is im-portant to note that insertion times are not visibleto the users because they do not create any userresponse. What is important to the user are theresponse times for OLAP queries. Figure 11 showsthe time for 1,000 OLAP queries in CR-OLAP andSTREAM-OLAP as a function of the number ofworkers (m). Figure 12 shows the speedup mea-

sured for the same data. We selected OLAP querieswith 10%, 60% and 95% query coverage, whichrefers to the percentage of the entire range of val-ues for each dimension that is covered by a givenOLAP query. The selected OLAP queries there-fore aggregate a small, medium and large portion ofthe database, resulting in very different workloads.We observe in Figure 11 that CR-OLAP signifi-cantly outperforms STREAM-OLAP with respectto query time (in some cases 2,000%). The dif-ference in performance is particularly pronouncedfor queries with small or large coverages. For theformer, the tree data structure shows close to loga-rithmic performance and for the latter, the tree cancompose the result by adding the aggregate val-ues stored at a few roots of large subtrees. Theworst case scenario for CR-OLAP are queries withmedium coverage around 60% where the tree per-formance is proportional to N1− 1

d . However, evenin this worst case scenario, CR-OLAP outperformsSTREAM-OLAP by between 300% and 500%. Fig-ure 12 indicates that both systems show a closeto linear speedup with increasing number of work-ers, however for CR-OLAP that speedup occurs formuch smaller absolute query times.

In a pay-as-you-go cloud environment, relatingquery response time to cloud computing cost mayalso be of interest. In that context, the close tolinear speedup observed in Figure 12 implies a fixedcost/performance ratio. For example, cutting queryresponse time in half would come at the price ofdoubling the system cost.

5.6. Test results: impact of growing system size (N& m combined)

In an elastic cloud environment, CR-OLAP andSTREAM-OLAP increase the number of workers(m) as the database size (N) increases. In our scaleup experiments, as we increase the number N ofdata items from 10 Mil to 160 Mil, CR-OLAP andSTREAM-OLAP increase the number m of work-ers from 1 to 16. That is, for each 10 Mil inserteditems, CR-OLAP and STREAM-OLAP add oneadditional worker to the system. The impact onthe performance of insert and query operations isshown in Figures 13 and 14, respectively.

With growing system size, the time for insertoperations in CR-OLAP (PDCR-tree) approachesthe time for STREAM-OLAP (1D-index). Moreimportantly however, the time for query opera-tions in CR-OLAP again outperforms the time forSTREAM-OLAP by a significant margin (in some

12

0

0.05

0.1

0.15

0.2

N=10M,m=1

N=20M,m=2

N=40M,m=4

N=60M,m=6

N=80M,m=8

N=100M,m=10

N=120M,m=12

N=140M,m=14

N=160M,m=16

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Increasing system size(N,m)

PDCR-tree1D-index

Figure 13: Time for 1000 insertions as a function of systemsize: N & m combined. (10Mil ≤ N ≤ 160Mil, d = 8,1 ≤ m ≤ 16)

0

1000

2000

3000

4000

5000

N=10M,m=1

N=20M,m=2

N=40M,m=4

N=60M,m=6

N=80M,m=8

N=100M,m=10

N=120M,m=12

N=140M,m=14

N=160M,m=16

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Increasing system size(N,m)

PDCR-tree(10% coverage)1D-index(10% coverage)

PDCR-tree(60% coverage)1D-index(60% coverage)

PDCR-tree(95% coverage)1D-index(95% coverage)

Figure 14: Time for 1000 queries as a function of systemsize: N & m combined. (10Mil ≤ N ≤ 160Mil, d = 8,1 ≤ m ≤ 16)

cases more than 1,000%), as shown in Figure 14.Also, it is very interesting that for both systems, thequery performance remains essentially unchangedwith increasing database size and number of work-ers. This is obvious for STREAM-OLAP where thesize of arrays to be searched simply remains con-stant but it is an important observation for CR-OLAP. Figure 14 indicates that the overhead in-curred by CR-OLAP’s load balancing mechanism(which grows with increasing m) is balanced out bythe performance gained through more parallelism.CR-OLAP appears to scale up without affecting theperformance of individual queries. It performed an16-fold increase in database size and number of pro-cessors, including an 16-fold increase in the averageamount of data aggregated by each OLAP query,without noticeable performance impact for the user.

5.7. Test results: impact of multiple query streams

We evaluated the impact of the number of querystreams on the performance of CR-OLAP. In allother experiments, we use one single stream ofOLAP queries to measure performance. Here, weuse multiple client processes, issuing multiple con-current streams of OLAP queries that are fed intoour CR-OLAP system. As shown in Firgure 15,the number of concurrent query streams (clients)has no impact on query performance.

0

100

200

300

400

500

1, q=1000

2, q=2000

3, q=3000

4, q=4000

5, q=5000

6, q=6000

7, q=7000

8, q=8000

9, q=9000

10, q=10,000

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Increasing number of clients and queries (client#,q)

PDCR-tree(10% coverage)PDCR-tree(60% coverage)PDCR-tree(95% coverage)

Figure 15: Time for 1000 OLAP queries as a function ofof the number of query streams. X-axis first parameter:number of query streams (clients). X-axis second param-eter: total number of queries issued (1,000 queries per querystream). Y-axis: Average time per 1,000 queries in seconds.(N = 160Mil, d = 8, m = 16)

5.8. Test results: impact of the number of dimen-sions

It is well known that tree based search meth-ods can become problematic when the number ofdimensions in the database increases. In Figures16 and 17 we show the impact of increasing don the performance of insert and query operationsin CR-OLAP (PDCR-tree) and STREAM-OLAP(1D-index) for fixed database size N = 40 millionand m = 8 workers.

Figure 16 shows some increase in insert time forCR-OLAP because the PDCR tree insertion inher-its from the PDC tree a directory node split opera-tion with an optimization phase that is highly sen-sitive to the number of dimensions. However, theresult of the tree optimization is improved queryperformance in higher dimensions. As shown inFigure 17, the more important time for OLAP queryoperations grows only slowly as the number of di-mensions increases. This is obvious for the arraysearch in STREAM-OLAP but for the tree searchin CR-OLAP this is an important observation.

13

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

4 5 6 7 8

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Number of Dimensions(d)

PDCR-tree1D-index

Figure 16: Time for 1000 insertions as a function of thenumber of dimensions. (N = 40Mil, 4 ≤ d ≤ 8, m = 8)

0

100

200

300

400

500

600

700

800

4 5 6 7 8

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Number of dimensions(d)

PDCR-tree(10% coverage)1D-index(10% coverage)

PDCR-tree(60% coverage)1D-index(60% coverage)

PDCR-tree(95% coverage)1D-index(95% coverage)

Figure 17: Time for 1000 queries as a function of the numberof dimensions. The values for “1D-index 95% coverage” are828.6, 1166.4, 1238.5, 1419.7 and 1457.8, respectively. (N =40Mil, 4 ≤ d ≤ 8, m = 8)

5.9. Test results: impact of query coverages

Figures 18, 19, 20, and 21 show the impact ofquery coverage on query performance in CR-OLAP(PDCR-tree) and STREAM-OLAP (1D-index).

0

50

100

150

200

250

300

350

10 20 30 40 50 60 70 80 90

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Query coverage per dimension(%)

Store is *Customer is *

Item is *Date is *

Address is *Household is *Promotion is *

Time is *No dimension is *

Figure 18: Time for 1000 queries (PDCR tree) as a functionof query coverages: 10% − 90%. Impact of value “*” fordifferent dimensions. (N = 40Mil, m = 8, d = 8)

0

50

100

150

200

250

300

350

91 92 93 94 95 96 97 98 99

Tim

e pe

r 10

00 o

pera

tions

(sec

)

Query coverage per dimension(%)

Store is *Customer is *

Item is *Date is *

Address is *Household is *Promotion is *

Time is *No dimension is *

Figure 19: Time for 1000 queries (PDCR tree) as a functionof query coverages: 91% − 99%. Impact of value “*” fordifferent dimensions. (N = 40Mil, m = 8, d = 8)

0

5

10

15

20

10 20 30 40 50 60 70 80 90

Tim

e ra

tio o

f 1D

-inde

x ov

er P

DCR

-tre

e

Query coverage per dimension(%)

Store is *Customer is *

Item is *Date is *

Address is *Household is *Promotion is *

Time is *No dimension is *

Figure 20: Time comparison for 1000 queries (Ratio: 1D-index / PDCR tree) for query coverages 10%−90%. Impactof value “*” for different dimensions. (N = 40Mil, m = 8,d = 8)

0

5

10

15

20

91 92 93 94 95 96 97 98 99

Tim

e ra

tio o

f 1D

-inde

x ov

er P

DCR

-tre

e

Query coverage per dimension(%)

Store is *Customer is *

Item is *Date is *

Address is *Household is *Promotion is *

Time is *No dimension is *

Figure 21: Time comparison for 1000 queries (Ratio: 1D-index / PDCR tree) for query coverages 91%−99%. Impactof value “*” for different dimensions. (N = 40Mil, m = 8,d = 8)

For fixed database size N = 40Mil, number ofworkers m = 8, and number of dimensions d = 8,we vary the query coverage and observe the query

14

times. In addition we observe the impact of a “*”in one of the query dimensions. Figures 18 and19 show that the “*” values do not have a signif-icant impact for CR-OLAP. As discussed earlier,CR-OLAP is most efficient for small and very largequery coverage, with maximum query time some-where in the mid range. (In this case, the maximumpoint is shifted away from the typical 60% becauseof the “*” values.) Figures 20, and 21 show the per-formance of STREAM-OLAP as compared to CR-OLAP (ratio of query times). It shows that CR-OLAP consistently outperforms STREAM-OLAPby a factor between 5 and 20.

5.10. Test results: query time comparison for se-lected query patterns at different hierarchylevels

Figure 22 shows a query time comparison be-tween CR-OLAP (PDCR-tree) and STREAM-OLAP (1D-index) for selected query patterns. For

0

200

400

600

800

1000

1200

1400

1600

* * * y * * y m * y m d * m * * m d * * d

Tim

e pe

r 10

00 o

pera

tions

(sec

)

* in different hierarchy levels of dimension Date

PDCR-tree(10% coverage)PDCR-tree(60% coverage)PDCR-tree(95% coverage)1D-index(10% coverage)1D-index(60% coverage)1D-index(95% coverage)

Figure 22: Query time comparison for selected query pat-terns for dimension Date. Impact of value “*” for differenthierarchy levels of dimension Date. (N = 40Mil, m = 8,d = 8).

fixed database size N = 40Mil, number of work-ers m = 8 and d = 8 dimensions, we test for di-mension Date the impact of value “*” for differenthierarchy levels. CR-OLAP is designed for OLAPqueries such as “total sales in the stores located inCalifornia and New York during February-May ofall years’ which act at different levels of multipledimension hierarchies. For this test, we created 7combinations of “*” and set values for hierarchylevels Year, Month, and Day: *-*-*, year-*-*, year-month-*, year-month-day, *-month-*, *-month-day,and *-*-day. We then selected for each combinationqueries with coverages 10%, 60%, and 95%. Thetest results are summarized in Figure 22. The main

observation is that CR-OLAP consistently outper-forms STREAM-OLAP even for complex and verybroad queries that one would expect could be eas-ier solved through data streaming than through treesearch.

6. Conclusion

We introduced CR-OLAP, a Cloud based Real-time OLAP system based on a distributed PDCRtree, a new parallel and distributed index structurefor OLAP, and evaluated CR-OLAP on the Ama-zon EC2 cloud for a multitude of scenarios. Thetests demonstrate that CR-OLAP scales well withincreasing database size and increasing number ofcloud processors. In our experiments, CR-OLAPperformed an 16-fold increase in database size andnumber of processors, including an 16-fold increasein the average amount of data aggregated by eachOLAP query, without noticeable performance im-pact for the user.

Future work: The next phase of our research col-laboration with IBM includes the study of how toadd data replication and fault tolerance to our sys-tem. We also plan to study how the master proces-sor could be replaced by a decentralized system.

Acknowledgment

The authors would like to acknowledge financialsupport from the IBM Centre for Advanced Stud-ies Canada and the Natural Science and Engineer-ing Research Council of Canada. We thank the re-search staff at the IBM Centre for Advanced Stud-ies Canada, and in particular Stephan Jou, for theirsupport and helpful discussions.

References

[1] J. Han, M. Kamber, Data Mining: Concepts and Tech-niques, Morgan Kaufmann Publishers, 2000.

[2] The OLAP Report, http://www.olapreport.com.[3] J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Re-

ichart, M. Venkatrao, F. Pellow, H. Pirahesh, DataCube: A Relational Aggregation Operator Generaliz-ing Group-By, Cross-Tab, and Sub-Totals, Data Min.Know. Disc. 1 (1997) 29–53.

[4] Y. Chen, F. Dehne, T. Eavis, A. Rau-Chaplin, PnP:sequential, external memory, and parallel iceberg cubecomputation, Distributed and Parallel Databases 23 (2)(2008) 99–126.

[5] F. Dehne, T. Eavis, S. Hambrusch, Parallelizing thedata cube, Distributed and Parallel Databases 11 (2002)181–201.

15

[6] Z. Guo-Liang, C. Hong, L. Cui-Ping, W. Shan, Z. Tao,Parallel Data Cube Computation on Graphic Process-ing Units, Chinese Journal of Computers 33 (10) (2010)1788–1798.

[7] R. T. Ng, A. Wagner, Y. Yin, Iceberg-cube computationwith PC clusters, ACM SIGMOD 30 (2) (2001) 25–36.

[8] J. You, J. Xi, P. Zhang, H. Chen, A Parallel Algo-rithm for Closed Cube Computation, IEEE/ACIS In-ternational Conference on Computer and InformationScience (2008) 95–99.

[9] R. Bruckner, B. List, J. Schiefer, Striving towards nearreal-time data integration for data warehouses, DaWaKLNCS 2454 (2002) 173–182.

[10] D. Jin, T. Tsuji, K. Higuchi, An Incremental Mainte-nance Scheme of Data Cubes and Its Evaluation, DAS-FAA LNCS 4947 (2008) 36–48.

[11] R. Santos, J. Bernardino, Real-time data warehouseloading methodology, IDEAS (2008) 49–58.

[12] R. J. Santos, J. Bernardino, Optimizing data warehouseloading procedures for enabling useful-time data ware-housing, IDEAS (2009) 292–299.

[13] F. Dehne, H. Zaboli, Parallel real-time olap onmulti-core processors, in: Proceedings of the 201212th IEEE/ACM International Symposium on Cluster,Cloud and Grid Computing (ccgrid 2012), 2012, pp.588–594.

[14] M. Ester, J. Kohlhammer, H.-P. Kriegel, The dc-tree:A fully dynamic index structure for data warehouses,in: In Proceedings of the 16th International Conferenceon Data Engineering (ICDE, 2000, pp. 379–388.

[15] H. Plattner, A. Zeier, In-Memeory Data Management,Springer Verlag, 2011.

[16] M. Ester, J. Kohlhammer, H.-P. Kriegel, The DC-tree: a fully dynamic index structure for data ware-houses, 16th International Conference on Data Engi-neering (ICDE) (2000) 379–388.

[17] Hadoop.URL http://hadoop.apache.org/

[18] J. Dean, S. Ghemawat, Mapreduce: simplified data pro-cessing on large clusters, Commun. ACM 51 (1) (2008)107–113.

[19] A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka,S. Anthony, H. Liu, P. Wyckoff, R. Murthy, Hive: awarehousing solution over a map-reduce framework,Proc. VLDB Endow. 2 (2) (2009) 1626–1629.

[20] A. Abouzeid, K. Bajda-Pawlikowski, D. Abadi, A. Sil-berschatz, A. Rasin, Hadoopdb: an architectural hy-brid of mapreduce and dbms technologies for analyticalworkloads, Proc. VLDB Endow. 2 (1) (2009) 922–933.

[21] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A.Wallach, M. Burrows, T. Chandra, A. Fikes, R. E. Gru-ber, Bigtable: A distributed storage system for struc-tured data, ACM Trans. Comput. Syst. 26 (2) (2008)4:1–4:26.

[22] Bigquery.URL http://developers.google.com/bigquery/

[23] S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shiv-akumar, M. Tolton, T. Vassilakis, Dremel: interactiveanalysis of web-scale datasets, Proc. VLDB Endow.3 (1-2) (2010) 330–339.

[24] Twitter storm.URL http://storm-project.net/

[25] F. Farber, N. May, W. Lehner, P. Große, I. Muller,H. Rauhe, J. Dees, The sap hana database–an archi-tecture overview., IEEE Data Eng. Bull. 35 (1) (2012)

28–33.[26] K. Doka, D. Tsoumakos, N. Koziris, Online querying of

d-dimensional hierarchies, J. Parallel Distrib. Comput.71 (3) (2011) 424–437.

[27] A. Asiki, D. Tsoumakos, N. Koziris, Distributing andsearching concept hierarchies: an adaptive dht-basedsystem, Cluster Computing 13 (3) (2010) 257–276.

[28] K. Doka, D. Tsoumakos, N. Koziris, Brown dwarf: Afully-distributed, fault-tolerant data warehousing sys-tem, J. Parallel Distrib. Comput. 71 (11) (2011) 1434–1446.

[29] Y. Sismanis, A. Deligiannakis, N. Roussopoulos, Y. Ko-tidis, Dwarf: shrinking the petacube, in: Proceedingsof the 2002 ACM SIGMOD international conference onManagement of data, 2002, pp. 464–475.

[30] S. Wu, D. Jiang, B. C. Ooi, K.-L. Wu, Efficient b-treebased indexing for cloud data processing, Proc. VLDBEndow. 3 (1-2) (2010) 1207–1218.

[31] J. Wang, S. Wu, H. Gao, J. Li, B. C. Ooi, Indexingmulti-dimensional data in a cloud system, in: Proceed-ings of the 2010 ACM SIGMOD International Confer-ence on Management of data, 2010, pp. 591–602.

[32] X. Zhang, J. Ai, Z. Wang, J. Lu, X. Meng, An efficientmulti-dimensional index for cloud data management,in: Proceedings of the first international workshop onCloud data management, 2009, pp. 17–24.

[33] M. C. Kurt, G. Agrawal, A fault-tolerant environmentfor large-scale query processing, in: High PerformanceComputing (HiPC), 2012 19th International Conferenceon, 2012, pp. 1–10.

[34] Zeromq socket library as a concurrency framework.URL http://www.zeromq.org/

[35] Transaction processing performance council, tpc-ds (de-cision support) benchmark.URL http://www.tpc.org

16