KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for...

7
KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le * , The De Vu * , Duc Hieu Nguyen * , Tien Hai Ho * , Duc Hai Nguyen * , and Tran Vu Pham Faculty Computer Science and Engineering Ho Chi Minh City University of Technology - VNU-HCM Ho Chi Minh City, Vietnam contact email: {haind, ptvu}@cse.hcmut.edu.vn ABSTRACT The emergence of advanced network technologies and de- creasing DRAM costs have provided great opportunities for data centers to deploy high performance key-value in-memory stores for real-time applications. Although key-value model helps in-memory stores to simplify their query processing and data management mechanisms, its simplicity have appli- cations to generate multiple item lookups to perform simple requests such as constructing HTML files. Such lookups are often executed sequentially and seriously affect the overall performance. In this paper, we introduce KELI, a novel in- memory key-value store that employs Key-value-with-Links, a simple but quite efficient data model, to solve this issue. On the top of the key-value model, KELI establishes links between items so instead of invoking expensive item lookups, it could simply follow those links to retrieve needed data. KELI also uses RDMA Read to implement link chasing to minimize searching overhead. Some experiments have been conducted to examine the performance of KELI. Their re- sults indicate that KELI runs three times faster than Re- dis for data lookups. Furthermore, KELI runs seven times faster than Redis when handling real workloads generated by a practical application. CCS Concepts Information systems Database design and mod- els; Data structures; Keywords in-Memory Stores; Key-Value; RDMA; InfiniBand 1. INTRODUCTION The emergence of Big Data era and real-time Web-based applications makes reducing the latency of processing data * Those authors contribute equally to this work Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. SoICT ’16, December 08-09, 2016, Ho Chi Minh City, Viet Nam c 2016 ACM. ISBN 978-1-4503-4815-7/16/12. . . $15.00 DOI: http://dx.doi.org/10.1145/3011077.3011098 requests become crucial. In order to provide services that could manipulate a huge amount of data within tight dead- lines, using traditional (i.e. on-disk) databases is inappropri- ate due to the limitation in the speed of secondary storages. Moving data to main memory (e.g. RAM) is a common ap- proach. In-memory stores have become important parts of many applications including social networking [11], big data analytics, and high performance computing [7]. Because of strict requirements in latency and throughput, most of in-memory stores organize their data by using key- value model to avoid expensive operations such as join or sort, which are popular in traditional (e.g. relational) mod- els. However, the simplicity of key-value model has applica- tions to decompose the original request into a lot of items lookups. For example, in order to construct an HTML file, Facebook has to create about 130 data lookups in average [13]. Similarly, Amazon requires about 100 to 200 internal requests for each of its pages [4]. As those requests are often executed sequentially, their cumulative latency is the major factor that limits the overall performance. Since most of in-memory stores are deployed over dis- tributed environment to handle big datasets, communica- tion is another problem affecting their performance. Dif- ferent from High Performance Computing (HPC) centers, most of data centers use traditional interconnection tech- nologies (i.e. Ethernet) to connect data nodes due to their low prices. This prevents in-memory stores from reaching their full potential because even though DRAM is extremely fast, the slow speed of data transmission makes applications to spend a lot of time waiting for the results. To tackle this issue, some data centers have begun to employ high perfor- mance network technologies such as RDMA and InfiniBand. The emergence of these technologies is a great opportunity for us to build ultra-low latency services. In this article, we introduce KELI (KEy-value-with-Links In-memory store), an in-memory cache that relies on state- of-the-art network technologies to offer low latency caching services for real-time applications. KELI employs a new data model, named Key-Value-with-Links, to manage data. Using Key-Value-with-Links model, KELI organizes items as key-value pairs and inside the value, there could be links to other items. The main advantage of this model over the traditional key-value is that it could reduce significantly the amount of data requests when processing items that having relationships with each other. In such cases, instead of de- composing the original request into multiple lookups, appli- cations could have KELI to chase the links between related

Transcript of KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for...

Page 1: KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le, The De Vu, Duc Hieu Nguyen, Tien Hai Ho, Duc

KELI: A Key-value-with-Links In-memory Store forRealtime Applications

Minh Duc Le∗, The De Vu

∗, Duc Hieu Nguyen

∗, Tien Hai Ho

∗, Duc Hai Nguyen

∗,

and Tran Vu PhamFaculty Computer Science and Engineering

Ho Chi Minh City University of Technology - VNU-HCMHo Chi Minh City, Vietnam

contact email: {haind, ptvu}@cse.hcmut.edu.vn

ABSTRACTThe emergence of advanced network technologies and de-creasing DRAM costs have provided great opportunities fordata centers to deploy high performance key-value in-memorystores for real-time applications. Although key-value modelhelps in-memory stores to simplify their query processingand data management mechanisms, its simplicity have appli-cations to generate multiple item lookups to perform simplerequests such as constructing HTML files. Such lookups areoften executed sequentially and seriously affect the overallperformance. In this paper, we introduce KELI, a novel in-memory key-value store that employs Key-value-with-Links,a simple but quite efficient data model, to solve this issue.On the top of the key-value model, KELI establishes linksbetween items so instead of invoking expensive item lookups,it could simply follow those links to retrieve needed data.KELI also uses RDMA Read to implement link chasing tominimize searching overhead. Some experiments have beenconducted to examine the performance of KELI. Their re-sults indicate that KELI runs three times faster than Re-dis for data lookups. Furthermore, KELI runs seven timesfaster than Redis when handling real workloads generatedby a practical application.

CCS Concepts•Information systems → Database design and mod-els; Data structures;

Keywordsin-Memory Stores; Key-Value; RDMA; InfiniBand

1. INTRODUCTIONThe emergence of Big Data era and real-time Web-based

applications makes reducing the latency of processing data

∗Those authors contribute equally to this work

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected].

SoICT ’16, December 08-09, 2016, Ho Chi Minh City, Viet Namc© 2016 ACM. ISBN 978-1-4503-4815-7/16/12. . . $15.00

DOI: http://dx.doi.org/10.1145/3011077.3011098

requests become crucial. In order to provide services thatcould manipulate a huge amount of data within tight dead-lines, using traditional (i.e. on-disk) databases is inappropri-ate due to the limitation in the speed of secondary storages.Moving data to main memory (e.g. RAM) is a common ap-proach. In-memory stores have become important parts ofmany applications including social networking [11], big dataanalytics, and high performance computing [7].

Because of strict requirements in latency and throughput,most of in-memory stores organize their data by using key-value model to avoid expensive operations such as join orsort, which are popular in traditional (e.g. relational) mod-els. However, the simplicity of key-value model has applica-tions to decompose the original request into a lot of itemslookups. For example, in order to construct an HTML file,Facebook has to create about 130 data lookups in average[13]. Similarly, Amazon requires about 100 to 200 internalrequests for each of its pages [4]. As those requests are oftenexecuted sequentially, their cumulative latency is the majorfactor that limits the overall performance.

Since most of in-memory stores are deployed over dis-tributed environment to handle big datasets, communica-tion is another problem affecting their performance. Dif-ferent from High Performance Computing (HPC) centers,most of data centers use traditional interconnection tech-nologies (i.e. Ethernet) to connect data nodes due to theirlow prices. This prevents in-memory stores from reachingtheir full potential because even though DRAM is extremelyfast, the slow speed of data transmission makes applicationsto spend a lot of time waiting for the results. To tackle thisissue, some data centers have begun to employ high perfor-mance network technologies such as RDMA and InfiniBand.The emergence of these technologies is a great opportunityfor us to build ultra-low latency services.

In this article, we introduce KELI (KEy-value-with-LinksIn-memory store), an in-memory cache that relies on state-of-the-art network technologies to offer low latency cachingservices for real-time applications. KELI employs a newdata model, named Key-Value-with-Links, to manage data.Using Key-Value-with-Links model, KELI organizes itemsas key-value pairs and inside the value, there could be linksto other items. The main advantage of this model over thetraditional key-value is that it could reduce significantly theamount of data requests when processing items that havingrelationships with each other. In such cases, instead of de-composing the original request into multiple lookups, appli-cations could have KELI to chase the links between related

Page 2: KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le, The De Vu, Duc Hieu Nguyen, Tien Hai Ho, Duc

DISK

UP

DA

TE

LOA

D D

ATA

RAM

STORAGE SERVER

APPLICATIONS

Figure 1: Overall architecture of KELI.

items until reaching needed data. With the support fromRDMA technology, link chasing is much faster than itemlookup. Therefore, KELI could help real-time applicationsto achieve very low overall latency.

The rest of this paper is divided into five parts. In Section2, we introduce the application that motives us to implementKELI. Section 3 provides useful information about advancedtechnologies backing KELI design. We describe KELI de-sign in detailed in Section 4. Section 5 reports the resultsof experiments we have carried out to evaluate KELI’s per-formance. After comparing KELI with related studies inSection 6, we conclude the paper in Section 7.

2. MOTIVATION APPLICATIONLet us start with a problem that motivates us to design

KELI. In Ho Chi Minh City, there is a need to monitorthe traffic condition in real-time to detect and resolve trafficcongestion in rush hours. To do so, we collect GPS datain real-time from multiple sources such as buses and mobilephones. Those signals are plotted on a digital map of thecity to produce the information about the current trafficcondition.

In order to properly identify congestion points, we con-sider streets as a group of contiguous straight lines calledsegments. Each segment is defined by a pair of points (i.e.its endpoints). Segments are grouped into disjoint areas callcells according to their geographical locations. We save theinformation about streets, cells, segments, and their end-points in the database.

For each GPS signal, we determine the cell that it belongsto according to its position (latitude and longitude). Afterthat, we retrieve all segments that are grouped into this cellto find out the segment to which this point belongs. If wecould find such segment, the information of the GPS signal(e.g. velocity, density, etc.) is then used to predict thecurrent traffic information on this segment.

In order to provide traffic information in real-time, dataare pushed into the system continuously and the deadlinesof calculations over GPS signals are very strict. Further-more, the number of signals to be processed is very large.

Assume that each device generates one signal for every 30seconds, we have to process more than 20 thousand signalsper second if only 10% of seven million vehicles in Ho ChiMinh City send data to the system. Such kind of workloadcannot be handled efficiently by traditional databases andin-memory approach must be considered. However, with ourcurrent data layout, using traditional key-value in-memorystores such as Memcached or Redis [3] results about 30 datalookups in average for each new GPS signal. Which meanswe have to generate 600 million requests per second withmentioned workload. Such requirement is almost impossi-ble for the average computer systems that we currently have.Therefore, a new solution is necessary.

3. BACKGROUNDTo solve problems mentioned in the previous section, we

have developed KELI. The design of KELI is inspired by thetwo key technologies: InfiniBand and RDMA. Thus, beforediscussing KELI’s design and implementation, let us firstbriefly introduce those technologies.

InfiniBand is the standard interconnection technologiesin High Performance Computing community thanks to itshigh throughput and low latency. With the emergence ofreal-time and online analytic applications, the need of highperformance network infrastructure is visible. Especially,decreasing in InfiniBand NIC prices in recent years makesdata centers start to consider this technology as a replace-ment for the traditional Ethernet technology. One of themost important features that help InfiniBand achieve veryhigh performance is its communication interface. In Infini-Band networks, nodes exchange data through messages us-ing verbs API instead of treating data as a stream of byteand sending it through socket interface like traditional Eth-ernet. This interface allows applications to interact directlywith the devices without interfering the Operating System.Bypassing the kernel yields very high communication effi-ciency as it avoids many sources of overhead such as waitingfor system calls, buffer copying, etc.

The Verb API allows applications to access InfiniBandNICs through four types of verbs: RDMA Read, RDMAWrite, Send, and Receive. Those verbs are posted by ap-plications to send queue and receive queue stored inside theInfiniBand NIC. The NIC uses the information posted inits queues to perform communication tasks. There are twotypes of verb semantics: channel semantics and memory se-mantics. Send and Receive verbs have channel semantics: inorder to get a message, the receiver must first explicitly posta Receive request to the queue. After that, the sender postsa send request for sending the message to the queue beforeactually transferring the data to the receiver. This is simi-lar to the ”read()” operation used in socket implementation.RDMA Read and RDMA Write verbs, on the other hand,have memory semantics. Instead of explicitly invoking theoperation, these verbs allow local machine to perform mem-ory accesses (i.e. read and write) over the remote machinewithout letting the CPU on the remote machine to be awareof these operations.

Read and write data without interfering the CPU of theremote machine makes RDMA Read and RDMA Write havemany performance advantages over Send/Receive verbs. Theabsence of remote CPU during read and write process makessystems use RDMA Read and RDMA Write scale out moreeasily since their performance does not bound by remote

Page 3: KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le, The De Vu, Duc Hieu Nguyen, Tien Hai Ho, Duc

student_id: 001

name: Duc

date_of_birth: 08/11/1995

university_id: QSB

hometown: 700000

university_id: QSB

name: HCMUT

location: 700000

city_id: 700000

name: Ho Chi Minh City

neighbor:

city_id: 850000

name: Long An

neighbor:city_id: 810000

name: Dong Nai

neighbor:810000850000

700000

700000

Figure 2: An example for the data model used byKELI.

CPU. This also reduces communication time and let the re-mote machine to save its CPU time for other tasks. In fact,RDMA Read and RDMA Write have the highest through-put and lowest latency among all verbs [10]. RDMA Read,in this article, is the key technology that inspires the designof KELI.

4. KELI DESIGN

4.1 System ArchitectureOur objective when designing KELI is to provide a light-

weight in-memory store for hardly-changed datasets storedin complicated databases. That is, KELI copies items storedby the database to memory and let applications to accessthem through its interface instead of sending requests di-rectly to the database. The design of KELI also assumesthat update occurs very rarely and a few changes do notcause serious impact on application performance and cor-rectness.

The overall architecture of KELI is illustrated in Figure 1.Data is originally stored permanently on disk to ensure dura-bility and availability. KELI is deployed entirely in memory.After starting up, KELI accesses data on disk and loadsthem into memory. It then reconstructs items’ structuresbased on its data model (i.e. key-value-with-links) and or-ganizes them on dedicated memory regions. After KELIfinishes loading data from disk, applications could createconnections to it to get needed data instead of accessing thedatabase. KELI does not support update operations (e.g.modify, write, and delete) so if applications want to changethe content of data, it still has to send those requests to thedatabase. Updates occurring at disk do not take effect im-mediately to the in-memory store. KELI, however, reloadsthe content of data on disk after predefined and fixed inter-vals.

4.2 Key-Value-with-Links Data ModelSimilar to other caches, KELI treats records as pairs of key

and value. Key of each record is a block of bytes with vari-able length and is used as its identifier. Record’s value is itspayload. Unlike some key-value stores such as RAMCloud[13] or memcached [2], KELI considers record’s value as a tu-ple consisting of multiple attributes. Those attributes couldbe numbers, strings, arrays, and even links other records.

Figure 2 shows an example of how KELI represents datafrom real life by its data model. Suppose we have a databasestoring the information about universities and students inHo Chi Minh City. Each university (or student) is a single

key-value pair whose key is its (his/her) ID. The value con-tains multiple attributes representing information about thestudent or university. It may contain links or a list of multi-ple links which can be used as pointers to other items. Forexamples, the “university id” field of the student whose IDis 001 is a link to another item (e.g. a university) whose IDis QSB. Therefore, from this student, we can directly accessthat university by just following “university id” link.

Storing the link to other items inside item’s value makesit easier to reason useful information that involve retrievingmultiple records. For examples, in Figure 2, given an ID ofa student, suppose that user wants to know if his hometownis the neighbor of the city where he is studying. KELI mustfirst access the record whose key equals to this ID. From this,it then travels across the link to his university and then goesto the city where this university located. At the same time,from the student’s record, KELI goes through the link tohis hometown and checks if the list of its neighbor containsthe link to his university’s location to make the final answer.This process only requires some link chasing while in rela-tional models, we must perform multiple cumbersome joinsto produce the answer. In the case of traditional key-valuedata model, the application must decompose the request intomultiple item lookups, each lookup is equivalent to one linkchasing. However, with the emergence of RDMA technology,item lookups are much more expensive than just followinglinks so using traditional key-value model takes more timethan applying key-value-with-links model.

4.3 Data LayoutSince RAM capacity is much smaller than that of sec-

ondary storage, utilizing memory space effectively is a cru-cial requirement. To do so, avoiding or reducing fragmen-tation is necessary since this is the primary source of lowmemory utilization. [15] showed that standard memory al-locators such as malloc in C do not handle this problem well.Therefore, to avoid fragmentation, we do not allocate spacefor new items at the time they are created but we reservemany contiguous memory slots in advance and then fill themup sequentially by the content of new items.

KELI updates its content in batch-style. When the updateprocess is triggered, it first allocates new memory regionsfor new items then fills them up with the content of datastored in the disk-based database. After that, KELI deallo-cates the memory regions of old data and use new memoryregions for answering upcoming requests from applications.To ensure applications do not access old items after updatetakes place, KELI halts all active connections from clientsand have the clients to reestablish those connections to theserver to obtain the new content.

Similar to other in-memory key-value, KELI employs ahash table to index items by their key. The implementa-tion of the hash table is based on Cuckoo hashing [14] sincethis technique ensures constant complexity in the worst case.Hash table does not hold the content of hashed items butit instead stores the pointer to the actual data. This meansgiven a key, access to the hash table would return null (i.e.item not found) or a pointer to the value of the item owningthis key. For each new item, KELI fist finds a room in al-located memory regions for it and write its content on thisregion. After that, the key of this item and the pointer toits content are added to the hash table.

Some store such as HBase keeps items in memory in form

Page 4: KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le, The De Vu, Duc Hieu Nguyen, Tien Hai Ho, Duc

Client Listener

Worker pool Data Regions

Request (1)

RDMA Read (5)SERVER

Hash table

Worker 2

Worker 1

Worker n

Figure 3: Request processing.

of memory objects to simplify data management tasks. How-ever, this approach often requires the server to serialize thoseobjects to an array of bytes before transferring them to theclient. Serialization adds significant overhead to request pro-cessing especially in small ones like item lookups. KELIavoids serialization by converting the content of item intoan array of bytes right after it being put into memory.

For each lookup request, KELI first uses the hash table forsearching the item using the key included in the request. Ifit finds out the pointer to needed item, KELI transfers thecontent of memory region pointed by this pointer back tothe client. The client then uses the data schema to interpretthe meaning of the result.

4.4 Request ProcessingKELI takes the advantages of advanced network technolo-

gies such as RDMA and InfiniBand to deliver very low la-tency data service. Figure 3 shows how KELI process datarequests from client. KELI’s communication modules arebuilt upon IB verb programming model. Given a key, theclient asks for its value by sending a request using ”ib send”operation. The request is received at server side by a listenerwhich has responsibility for receiving any incoming requestsfrom the client. In order to maximize KELI performance,we let the listener continuously ask input queue for new re-quests instead of passively waiting for the queue to informit about the new message like traditional techniques. Al-though this approach wastes a lot of CPU cycle for pollinginput queue, it makes KELI response to the new requestvery quickly.

When listener discovers that there is a new request in theinput queue, it then pops this request out and forwards itto one of worker threads in worker pool. Worker thread israndomly chosen to ensure the workload scatters evenly be-tween threads. After receiving a request from the listener,worker thread then searches the needed item by accessingthe hash table. If the required item is not found, a responsewith empty payload is sent back to the client using ”ib recv”operation. If worker thread can locate the needed item viathe pointer returned by the hash table then it puts the con-tent of the item to the payload of the response and sends itback to the client. ”ib recv” is also used to transmit data.

The client has responsibility for interpreting the meaning

Name: Segment

Key:

Value

segment_id: uint64

id_node_start: Linkid_node_end: Linklist_cell_id: List<Link>street_id: Link

Name: Node

Key:

Value

node_id: uint64

lon: Doublelat: Double

Name: Street

Key:

Value

street_id: uint64

list_segment_id: List<Link>

Name: Cell

Key:

Value

cell_id: uint64

list_segment: List<Link>

Figure 4: Schema of datasets used in experiments.

of the payload of the response received from the server. Ifthe needed item contains links to other items and applicationalso wants to retrieve them, the client does not make anotherlookup request but using RDMA Read to directly read thecontent of linked items from the server. The reason of usingRDMA Read instead of creating a new lookup request isthat item lookups involve in executing APIs associated withInfiniBand verbs interface. Those functions require bothclient and server to participate in processing the request.In contrast, RDMA Read allows the client to directly accessserver’s memory without informing server’s CPU. Therefore,executing RDMA Reads is much cheaper than invoking a fullitem lookup request.

5. PERFORMANCE EVALUATIONThis section illustrates experiments we have conducted to

evaluate the performance of KELI. In order to examine thebenefits of using KELI, we compare its performance withRedis. All experiments are carried out on a small cluster.Data nodes are equipped with CPU Intel Xeon E5-2670 and32GB memory. Data nodes are connected via InfiniBand 40Gbps network. We use Redis version 4.0.3 in all experiments.

Figure 4 shows the data schema we use in experiments.This schema is constructed according to the motivation ap-plication we mentioned in Section 2. There are four datasetswe have to maintain in memory: street, node, segment, andcell. All of them are generated based on OpenStreetMap’s

Page 5: KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le, The De Vu, Duc Hieu Nguyen, Tien Hai Ho, Duc

Table 1: Size of datasets used in experiments.Dataset Size Explainationstreet 288669 Represent streets on digital map

segment 4204072 Lines that can be put togetherto form streets

node 4067026 Segments’ endpointscell 145857 Disjoint areas used to group

segments

Redis KELI0

20

40

60

80

100

Latency(microsec)

Figure 5: Item lookup latency of KELI in compari-son with Redis.

database. Table 1 shows the number of items in each datasets.Redis is used as a cache (logging mode is disabled) which

means data will be lost when it stops. Therefore, we savedatasets permanently on disk in form of JSON files and pushthem to Redis each time it starts. On the other hand, KELIlets MongoDB keep its datasets permanently and each timeit starts, KELI reloads data from MongoDB to main mem-ory. Since MongoDB manages data in form of documents,KELI must transform them to key-value format and linkrelated items together before using them.

5.1 LatencyLet us first examine the latency of item lookups of KELI

and Redis. To do so, we create a client and make it generatemultiple item lookups. The client then sends the lookuprequests to each store to have them search for the valueof the keys contained in the requests. If there exist itemswhose key matches that contained in the request then storeswill send its value back to the client. The client chooses keyrandomly from the set of keys available in datasets to ensurethat there is request causes the server to create a responsewith a null payload.

Figure 5 shows the 99th percentile latency of KELI andRedis. Clearly, KELI is three times faster than Redis. Themain reason for this is that while KELI implements Infini-Band verbs to transmit both requests and responses, Redisstill uses traditional communication mechanism (e.g. Socket).This decision makes Redis suffer from many disadvantages.Firstly, exploiting InfiniBand verbs allows KELI to commu-nicate directly with Network drivers without invoking sys-tem calls to the kernel (kernel bypass). Secondly, kernelbypassing helps KELI to move data directly from its lo-cal memory regions to NIC’s buffer without using interme-diate buffers provided by Operating System as Redis does

Thou

sand

reco

rds/

s

Number of clients

100

80

60

40

20

03025201510

RedisKELI

5

Figure 6: Throughput of KELI and Redis whenchanging the number of clients generating lookuprequests.

when using Socket interface (zero copying). Therefore, KELIcould reduce significantly communication overhead causedby system calls and copy operations.

5.2 System ThroughputUsing the same configuration with experiments in the pre-

vious section, we evaluate the throughput of KELI and Re-dis by changing the number of clients sending the request toeach store. Figure 6 shows the experiment results. Clearly,KELI has better throughput than that of Redis when thereare just a few clients. However, KELI does not performlookup requests well when the number of clients increases.Particularly, its throughput almost does not change whenthe number of clients is greater than four. On the otherhands, Redis scales very well. Its throughput keeps increas-ing as the number of clients increases.

Although using InfiniBand verbs helps KELI to utilize thenetwork must better than Redis. Its communication mod-ules’ implementation is still relatively naive. Handling somany tasks at server side makes KELI’s performance beeasily bounded by server’s CPU. Also, in order to bypassthe kernel, memory regions holding KELI’s items must bepinned to physical memory to avoid page swapping. Sincethe amount of data to be pinned is very large compared tothe page size, the page table that maintains the mapping in-formation between virtual pages and physical addresses can-not be held entirely on the NIC driver. Actually, it can justcache some parts of the table in its local memory. Naive im-plementation and choosing keys randomly make KELI sufferfrom heavy cache misses leading to poor performance.

5.3 Experiments on Real DatasetsIn this section, we will show that despite scaling poorly,

KELI still works well with real workloads by letting bothKELI and Redis handle workloads generated by practicalapplications. In our experiment, we deploy the traffic appli-cation that we described in Section 2 at the client side andlet another machine push GPS signals continuously to thisapplication. To improve overall throughput, we do not senddata to the application one-by-one but by a group of 10000

Page 6: KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le, The De Vu, Duc Hieu Nguyen, Tien Hai Ho, Duc

Redis KELI0

50

100

150

200

250

Exec

utio

n tim

e (s

ec)

Figure 7: Throughput of KELI and Redis whenchanging the number of clients generating lookuprequests.

signals at a time. Also, application splits the input data into32 sub-groups and letting 32 worker threads to handle themindividually. Those threads process data by retrieving mapdatasets from KELI or Redis and combining them with GPSsignals to generate traffic information.

Figure 7 shows the amount of time application spendswhen cooperating with KELI or Redis. Apparently, usingKELI makes the application run seven times faster than us-ing Redis. This is because KELI uses link chasing to re-trieve items needed by the application while Redis gets datathrough multiple lookup operations. For a single GPS, ap-plication must generate more than 2000 internal requests inaverage. As we have explained in Section 4, link chasingconsumes much less time than item lookups do. Therefore,the cumulative latency of using Redis is much higher thanthat of using KELI.

Also note that the lookup operation of KELI is not well-implemented. As we have shown in the previous section,when the number of clients is high, KELI is outperformedby Redis. However, if the application requires items thathave links to each other, KELI could avoid those expensiveoperations by chasing those links to reduce overall latency.As Figure 7 shows, even though handling requests from 32clients − the case that Redis outperforms KELI in previousexperiment − chasing links helps KELI to boost its perfor-mance significantly and finish its job seven times sooner thanRedis.

6. RELATED WORKExploiting fast network technologies such as RDMA and

InfiniBand to improve the performance of key-value in-memorystores is an attractive research topic. Similar to KELI, Pilaf[12] also uses Cuckoo hashing for item lookups. However,it lets the client use RDMA Reads to perform lookup op-erations instead of processing the request inside the serveras KELI does. This decision makes lookups do not sufferfrom server’s CPU bound but since a single lookup requiresabout two to three RDMA Reads in average, their cumu-lative latency is much higher than that of KELI as pointerchasing requires only one RDMA Read. FaRM [5] also uti-lizes RDMA Read for item lookups and suffers from the sameissues as Pilaf.

HERD [10], on the other hand, is implemented in an un-

conventional way: its communication layer is constructedupon unreliable connection types. Since unreliable trans-ports reduce a lot of communication overhead, HERD couldperform lookup operations at very high speed. This de-sign, however, does not ensure reliable transmission whichis highly required by modern data centers.

RAMCloud [13] and MICA [11] also bypass the kernel toachieve ultra-low latency request processing. Those stores,however, do not utilize RDMA like other systems listedabove. They instead interact directly with the NIC to en-sure their service still work well on commodity hardware.Although those stores could process item lookups at veryhigh speed, using traditional key-value model still makesthem sensitive to cumulative latency.

Together with new designs, other noticeable efforts havebeen made to improve the performance of existing systems.Jithin Jose et. al. [9] redesigned Memcached communicationlayer for RDMA capable networks. Bin Fan et. al. [6] en-hanced Memcached’s performance by combining optimisticcuckoo hashing, a compact LRU-approximating eviction al-gorithm, and comprehensive implementation of optimisticlocking. More recently, J. Huang et. al. [8] modified com-munication modules of HBase [1] to improve its performanceover high performance networks. Most of those works onlychange the internal organization of well-known stores, theytry to avoid or minimize modification on stores’ APIs anddata model. Therefore, those solutions are still subjectedfrom the cumulative latency of performing multiple lookups.

7. CONCLUSION AND FUTURE WORKThis paper describes KELI, a novel in-memory key-value

store that relies on state-of-the-art network technologies tooffer low latency caching services for real-time applications.The principle underlying KELI design is to allow items to belinked to each other. Doing so helps reduce the cumulativelatency significantly when breaking the original request intomultiple item lookups. KELI also exploits high performancenetwork technologies such as RDMA and InfiniBand to de-liver very low latency data service. Experiment results showthat KELI works much better than Redis on real datasets.

However, there are still works to be done. Firstly, KELIdoes not scale well due to naive communication implemen-tation. Improving communication modules is necessary toensure KELI handle requests from multiple clients efficiently.Secondly, KELI is currently running on a standalone dataserver. However, since the amount of data is rapidly in-creasing, distributed approach should be taken into consid-eration. Finally, KELI is constructed upon the assumptionthat changes on the datasets rarely occurs and those changesdo not severely affect the lookup results. In the future, inorder to make KELI work well with a wide range of applica-tions, a flexible update mechanism should be implemented.

8. REFERENCES[1] Hbase. https://hbase.apache.org/. Accessed:

2016-08-31.

[2] Memcached. https://memcached.org/. Accessed:2016-08-31.

[3] Redis. http://redis.io/. Accessed: 2016-08-31.

[4] G. DeCandia, D. Hastorun, M. Jampani,G. Kakulapati, A. Lakshman, A. Pilchin,S. Sivasubramanian, P. Vosshall, and W. Vogels.

Page 7: KELI: A Key-value-with-Links In-memory Store for …KELI: A Key-value-with-Links In-memory Store for Realtime Applications Minh Duc Le, The De Vu, Duc Hieu Nguyen, Tien Hai Ho, Duc

Dynamo: Amazon’s highly available key-value store.ACM SIGOPS Operating Systems Review,41(6):205–220, 2007.

[5] A. Dragojevic, D. Narayanan, M. Castro, andO. Hodson. FaRM: Fast Remote Memory. In 11thUSENIX Symposium on Networked Systems Designand Implementation (NSDI 14), pages 401–414,Seattle, WA, Apr. 2014. USENIX Association.

[6] B. Fan, D. G. Andersen, and M. Kaminsky. MemC3:Compact and Concurrent Memcache with DumberCaching and Smarter Hashing. In Presented as part ofthe 10th USENIX Symposium on Networked SystemsDesign and Implementation (NSDI 13), pages371–384, 2013.

[7] H. Greenberg, J. Bent, and G. Grider. Mdhim: AParallel Key/Value Framework for HPC. In 7thUSENIX Workshop on Hot Topics in Storage and FileSystems (HotStorage 15), 2015.

[8] J. Huang, X. Ouyang, J. Jose, M. Wasi-ur Rahman,H. Wang, M. Luo, H. Subramoni, C. Murthy, andD. K. Panda. High-Performance Design of Hbase withRDMA over Infiniband. In Parallel & DistributedProcessing Symposium (IPDPS), 2012 IEEE 26thInternational, pages 774–785. IEEE, 2012.

[9] J. Jose, H. Subramoni, M. Luo, M. Zhang, J. Huang,M. Wasi-ur Rahman, N. S. Islam, X. Ouyang,H. Wang, S. Sur, et al. Memcached Design on HighPerformance RDMA Capable Interconnects. In 2011International Conference on Parallel Processing, pages

743–752. IEEE, 2011.

[10] A. Kalia, M. Kaminsky, and D. G. Andersen. UsingRDMA Efficiently for Key-Value Services. In ACMSIGCOMM Computer Communication Review,volume 44, pages 295–306. ACM, 2014.

[11] K. Lim, D. Meisner, A. G. Saidi, P. Ranganathan, andT. F. Wenisch. Thin Servers with Smart Pipes:Designing SoC Accelerators for Memcached. In ACMSIGARCH Computer Architecture News, volume 41,pages 36–47. ACM, 2013.

[12] C. Mitchell, Y. Geng, and J. Li. Using One-SidedRDMA Reads to Build a Fast, CPU-EfficientKey-Value Store. In Presented as part of the 2013USENIX Annual Technical Conference (USENIX ATC13), pages 103–114, 2013.

[13] J. Ousterhout, P. Agrawal, D. Erickson, C. Kozyrakis,J. Leverich, D. Mazieres, S. Mitra, A. Narayanan,G. Parulkar, M. Rosenblum, et al. The case forRAMClouds: Scalable High-Performance StorageEntirely in DRAM. ACM SIGOPS Operating SystemsReview, 43(4):92–105, 2010.

[14] R. Pagh and F. F. Rodler. Cuckoo Hashing. InEuropean Symposium on Algorithms, pages 121–133.Springer, 2001.

[15] S. M. Rumble, A. Kejriwal, and J. Ousterhout.Log-Structured Memory for DRAM-based Storage. InProceedings of the 12th USENIX Conference on Fileand Storage Technologies (FAST 14), pages 1–16,2014.