Parallel Real-Time OLAP on Multi-Core Processors and Cloud ... · OLAP vs. OLTP Source:...
Transcript of Parallel Real-Time OLAP on Multi-Core Processors and Cloud ... · OLAP vs. OLTP Source:...
Frank Dehne ● www.dehne.net
Frank DehneCarleton University, Ottawa, Canada
IBM Center For Advanced Studies, Toronto, Canada
Joint work withR. Bordawekar (IBM Yorktown), J. Dale (IBM Littletown), R. Grosset (IBM
Toronto), M. Genkin (IBM Toronto), S. Jou (IBM Toronto), P. Jain (IBM Littletown), Q. Kong (Dalhousie), M. Petitclerc (IBM Laval),A. Rau-Chaplin (Dalhousie), D. Robillard (Carleton), F. Thomas (IBM
Ottawa), H. Zaboli (Carleton), R. Zhou (Carleton)
Parallel Real-Time OLAP on Multi-Core Processors and Cloud Architectures
Frank Dehne ● www.dehne.net
About the Speaker
Research Program:
● The design and implementationof efficient parallel algorithms.
● The interrelationship betweenthe theoretical analysis ofparallel algorithms and theperformance observed oncurrent parallel architectures
● The use of efficient parallelalgorithms for large-scale dataanalytics and computationalbiology
Current Projects
● Auto-tuned parallel algorithmsfor multi-core processors,GPUs, clusters & clouds.
● Parallel large-scale dataanalytics: online analyticalprocessing (OLAP).
● Parallel computational biology:protein-protein interactionprediction.
Frank Dehne ● www.dehne.net
Online Analytical Processing (OLAP)
IBM/COGNOS
InsightWorkspaceReport/Studio
Frank Dehne ● www.dehne.net
Online Analytical Processing (OLAP)
A
B
C
AB
AC
BC ABC
Operations:● roll-up
● drill-down
● slice
● dice
Traditional: Data Cube
Pre-compute group-bys toimprove query response time.
Static or Batch Updates
ABCD
ABC ABD ACD BCD
AB AC AD BC BD CD
AA BB CC DD
All
Frank Dehne ● www.dehne.net
OLAP vs. OLTP
Source: AcceleratedAnalytics.com
OLTP System OLAP System
Source of data Operational data Consolidated data
Purpose of data Business operations Planning, decisionsupport
Type of data Snapshot of ongoingbusiness
Multi-dimensionalviews of “historic” data
Updates Small and fast Periodic long-runningbatch jobs
Queries Relatively simple,involving few datarecords
Often complex,involving aggregationsof large data sets
Processing speed Typically very fast Depends on amount ofdata involved; batchupdates and complexqueries may take manyhours
Frank Dehne ● www.dehne.net
The Five V's Of “Big Data”
• Volume• Velocity• Variety• Veracity• Value
ABCD
ABC ABD ACD BCD
AB AC AD BC BD CD
AA BB CC DD
All
Frank Dehne ● www.dehne.net
Real-Time OLAP
• Avoid static data cubestructure and batchupdates.
• Stream of OLAP insert and query operations.
• Inserts are immediate.
Queries operateon latest up-to-date data set.
Real Time OLAPEngine
Results
A
B
C
AB
AC
BC ABC
Insert/Query
Str eam
Frank Dehne ● www.dehne.net
Real-Time OLAP
• Problem: Performance.
• Data cube wasintroduced to improveperformance!
Real Time OLAPEngine
Insert/Query
Str eam
Results
Frank Dehne ● www.dehne.net
Real-Time OLAP
Research Question:
Can parallel computing be used to improveperformance for real-time OLAP?
Real Time OLAPEngine
Insert/Query
Str eam
Results
Frank Dehne ● www.dehne.net
Parallel Computing
Multi-core
Processors
Cloud / Cluster
shared memory distributed memory
Frank Dehne ● www.dehne.net
Parallel real-time OLAPon multi-core processors
Frank Dehne ● www.dehne.net
Parallel Real-Time OLAP
Our solution:
Parallel DC-Tree
Real Time OLAPEngine
Insert/Query
Str eam
Results
Parallel DC-Tree
Frank Dehne ● www.dehne.net
Parallel DC-Tree
• Multidimensional tree datastructure.
• Operations: insert and query.
• Enhanced for dataaggregation and dimensionhierarchies (Kriegel et.al.,ICDE 2000)
• Enhanced for multi-coreparallel computing (Dehneet.al., CCGrid 2012)
Insert/Query
Str eam
Results
Real-Time OLAP
Engine
Parallel DC-Tree
Frank Dehne ● www.dehne.net
Sequential DC-Tree
● Ester, Kohlhammer, Kriegel(ICDE 2000)
● Adaptation of R-tree for OLAP● Replaces total ordering by
concept-hierarchies.
● Replaces minimum boundingrectangles (MBR) byminimum describing sets (MDS)
● Adds internal directorynodes
R-Tree
Frank Dehne ● www.dehne.net
Concept hierarchies
● Data representation:
Frank Dehne ● www.dehne.net
Minimum describing sets (MDS)
MBR MDS
Frank Dehne ● www.dehne.net
Parallel DC-Tree
● Dehne, Zaboli(CCGrid 2012)
● Parallelization:● Insert and query
operations areexecuted concurrently.
● Query operations thatneed to searchmultiple subtrees of anode are split intomultiple concurrentprocesses.
OLAP queries:
insert
query (aggregate range query)
parallel DC-tree
multi-core processor
memory
inserts/queries results
Frank Dehne ● www.dehne.net
Parallel DC-Tree
Main Problem OLAP queries:
insert
query (aggregate range query)
parallel DC-tree
multi-core processor
memory
inserts/queries results
Interference betweenconcurrent insert and query operations:
Query results have toinclude transient insertsthat have been issued prior.
Frank Dehne ● www.dehne.net
Parallel DC-Tree
Race Conditions OLAP queries:
insert
query (aggregate range query)
parallel DC-tree
multi-core processor
memory
inserts/queries results● Inserts and queries run at
different speeds.
● Insert traverse root to leafand back to root
● Queries need to traversesubtrees depending on datavolume to be aggregated.
● Insert and query operationscan overtake each other.
Frank Dehne ● www.dehne.net
Parallel Insert & Query Operations
D1 1 20
R
L1 3 10 L2 6 10
D2 2 20
L3 4 10 L4 5 10
ID TimeStamp
Measure
Solution:
Add to data structure Right sibling links Timestamps
● Lengthy case analysis
● Most challenging case:● Insert creates node split. Transient
query needs to detect and re-calculate.
Frank Dehne ● www.dehne.net
Parallel Insert & Query Operations
1D1
4D1
1
2 3 2 3
D2 D3 D2
D3
D4
D3,3D2,2D1,1 D1,1
D3,3D4,1D1,1
Alreadycounted?
Stack
Insert
Query
CASE:● Insert creates a directory node split● Concurrent query returns back up the tree and finds tree structure changed.
New node getsold time stamp !
Frank Dehne ● www.dehne.net
Parallel DC-Tree Performance
● Transaction Processing Performance CouncilTPC-DS (Decision Support) Benchmark.
● Two processor architectures:
1.Intel Sandy Bridge, 4 Cores, 8 Hardware Threads(Hyperthreading), 16 GB Memory.
2.Intel Xeon Westmere EX (2 Sockets), 20 Cores, 40Hardware Threads (Hyperthreading), 256 GBMemory.
Frank Dehne ● www.dehne.net
TPC-DS Benchmark
TPC.org
Frank Dehne ● www.dehne.net
TPC-DS Benchmark
8 Dimensions
HierarchyLevels
HierarchyLevels
Frank Dehne ● www.dehne.net
Intel Sandy Bridge, 4 CoresDB initialized with 400,000 records. I = # inserts and Q = # queries in the input stream.
5% querycoverage
25% querycoverage
Frank Dehne ● www.dehne.net
Intel Sandy Bridge, 4 Cores
Comparison with multi-threaded MySQL
DB initialized with 400,000 records. Stream of 1000 queries.
Parallel DC-Tree
25% querycoverage
Frank Dehne ● www.dehne.net
Intel Xeon Westmere EX, 20 Cores
100 GB data set (10 Mil. Records) 10,000 queries1,000 insertions
Frank Dehne ● www.dehne.net
Intel Xeon Westmere EX, 20 Cores
Parallel DC-tree
number of roots
response time
Hotspot at root requires multiple root copies...
Frank Dehne ● www.dehne.net
Intel Xeon Westmere EX, 20 Cores
Response time5 sec. -> .25 sec.
Response time2.7 sec. -> .13 sec.
100 GB data set (10 Mil. Records) 10,000 queries1,000 insertions
Total Total
Frank Dehne ● www.dehne.net
Parallel real-time OLAP on multi-core processors
● Published in ACM/IEEECCGrid 2012
Frank Dehne ● www.dehne.net
Parallel real-time OLAP on multi-core processors
● Published in ACM/IEEECCGrid 2012
● IBM Research Impact OfThe Year Award
Frank Dehne ● www.dehne.net
Parallel real-time OLAP on multi-core processors
● Published in ACM/IEEECCGrid 2012
● IBM Research Impact OfThe Year Award
● IBM patent submission
● IBM implemenationgroup for TM1
Frank Dehne ● www.dehne.net
Parallel real-time OLAP
● Published in ACM/IEEECCGrid 2012
● IBM Research Impact OfThe Year Award
● IBM patent submission
● IBM implemenationgroup for TM1
● New 3 year fundedproject (2013-2016):scale up to cloud
● $1M hardware (privatecloud at Carleton)
● Carleton/IBM DataScience Institute
Frank Dehne ● www.dehne.net
Parallel real-time OLAPon cloud architectures
Version 1
Presented at IEEE BigData 2013
Frank Dehne ● www.dehne.net
Cloud Computing Architecture
● Large scale computecluster
● Virtual machines ondemand
● Elastic: dynamic additionof compute resources
● Dedicated storagedevices (e.g. S3buckets)
Frank Dehne ● www.dehne.net
Real-Time OLAP “In The Cloud”
OLAP operations:
insert
query (aggregate range query)
Pointers to results(on S3)
Cloud
Master (multi-core)
Worker (multi-core)
Storage device (S3)
Zookeeper
Frank Dehne ● www.dehne.net
Real-Time OLAP “In The Cloud”
Frank Dehne ● www.dehne.net
“Elastic” System Growth
Hat
Hat
Frank Dehne ● www.dehne.net
“Elastic” System Growth
Frank Dehne ● www.dehne.net
Correctness
Worker 1 Worker 2
Worker 3
Theorem:
Horizontal links a and c betweenworkers arenot needed.
Frank Dehne ● www.dehne.net
Load Balancing
● Insertion/query loadand memory usage
● More subtrees thanworkers
● Global statistics inZookeeper
● Migrate subtrees
● Split subtrees
Frank Dehne ● www.dehne.net
TPC-DS Benchmark
8 Dimensions
HierarchyLevels
HierarchyLevels
Frank Dehne ● www.dehne.net
Performance
N: database size, d: # dimensions, m: # workers
Frank Dehne ● www.dehne.net
Performance
Frank Dehne ● www.dehne.net
Performance
0.25 sec
Parallel real-time OLAPon cloud architectures
Version 2
Fully Distributed (no “Master” processor)
Frank Dehne ● www.dehne.net
System Overview
● Multiple servers● Multiple workers● Each worker stores multiple
PDC trees● Each server stores a local
“system image” (PDC treehat)
● Zookeper stores a global“system image” (PDC treehat) and worker load statistics
Frank Dehne ● www.dehne.net
System Overview
● Global system image stored atZookeeper
● Servers store local system image● Local system images get pushed and
aggregated into Zookeeper● Zookeper returns new global systems
image to servers● Strong serialization among user
sessions connected to the same server(workgroup).
● “Best effort” serialization between usersessions on different servers(typical freshness bound <= 8 seconds;worst case freshness bound <= 15seconds)
Frank Dehne ● www.dehne.net
Load Balancing
● Manager queries Zookeeperand examines worker loadstatistics
● Manager initiates loadbalancing operationsbetween workers
● Concurrent withInsert/Query operations
● Manager is NOT involved inInsert/Query operations
Frank Dehne ● www.dehne.net
Data Ingestion Performance
16 workers
Frank Dehne ● www.dehne.net
Load Balancing
16 workers
Frank Dehne ● www.dehne.net
Real Time OLAP: Insert/Query Stream
Database size: 400 M
16 workers4 servers
Frank Dehne ● www.dehne.net
Impact Of Number Of Servers
Database size: 400 M
16 workers
Frank Dehne ● www.dehne.net
System Scale-UP
Data size and #workers are Both increasing
25 M data itemsPer worker
4 servers
average over allquery coverages(5% - 95%)
Frank Dehne ● www.dehne.net
Conclusion
Parallel data structurescan enable real-time OLAP on multi-core andcloud architectures.