IMDB_Scalability
-
Upload
israel-gold -
Category
Documents
-
view
14 -
download
0
Transcript of IMDB_Scalability
The Goal: In-Memory Database Scalability
core core
ore core
core core
core corecore core
ore core
core core
core core
core core
ore core
core core
core core
core core
ore core
core core
core corecore
Multicores (CMP) Multisocket
Multicores
How can DBMS engine scale across hardware islands ?
in-memory database on multicore
Two Approaches to In-Memory Database Scalability on Multicore
Employing a Symmetric Database Engine Example Systems: SILO and Hekaton
Employing a Partitioned Database Engine Example Systems: DORA, PLP, ATraPos and
H-Store
in-memory database on multicore
A Symmetric Database Engine Architecture
Symmetric Database Engine using Shared Memory
Database Process consists ofmultiple transaction threadseach running on a separate core
Transaction threads access theentire shared memory with cache coherency enabled
Transaction Engine takes careabout Serializability
Processor takes care about Cache Coherency
in-memory database on multicore
A Partitioned Database Engine Architecture Partitioned Database as Primary Scalability Mechanism
Database is divided into multiple partitions each managed by a dedicated core
A partition is owner of its data and indexes A centralized routing table is required
to route transactions to partitions Multi-partition transactions is executed
by migrating transactions betweencores
Partitioning may be static or dynamic dynamic re-partitioning mechanism
enables to load balance partitionaccess
All partitions must reside in sharedmemory with Cache Coherency enabled
in-memory database on multicore
Problem: Cache Coherency Limits Scalability
Cache Coherency Mechanism (CCM) limits processor scalability
Cache Coherency of thousand cores is not feasible Problem: How can a DBMS engine scale in the
presence of non-coherent hardware islands ?
Hardware Island with CCM Hardware Island with CCM
CCM
in-memory database on multicore
In-memory Database on Thousand cores
The paper: Staring Into The Abyss: An Evaluation Of Concurrency Control With One Thousand Cores concludes with the following observation
Software-hardware co-design is the only solution to address the performance issues resulting from high core count. For certain functionalities, specialized hardware can significantly improve performance while reducing power consumption
The next slides provides some design guidelines
in-memory database on multicore
System Design Principles
Avoid many of the centralized critical sections present in classic database engines (see next slide)
Be sensitive to the cost of Cache Coherency (CC) Do you really CC enabled on all cores?
Leveraging hardware message passing for efficient thread synchronization (inter-core communication); example the TILE-Gx processor family
in-memory database on multicore
Avoid Centralized Critical Sections
Avoid centralized contention points and make all synchronization scale with the data, allowing larger databases to support more concurrency.
Use lightweight locking mechanism and eliminate the bottleneck of a centralized lock manager, by co-locating locks with each DB record
Use optimistic based concurrency control possibly combined with multi-version mechanism
Use transaction’s local memory - private workspace - to cache its write-set and avoid installing writes until commit time
in-memory database on multicore