Highly scalable caching service on cloud - Redis

HUAWEI TECHNOLOGIES CO., LTD.

www.huawei.com

Slide title :40-47pt

Slide subtitle :26-30pt

Color::white

Corporate Font :

FrutigerNext LT Medium

Font to be used by customers and

partners :

Arial

Highly Scalable Caching Service on Cloud Krishna Kumar & Irfan Rehman Huawei India Cloud Platforms Team



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

2

We will discuss

Introduction

In-memory caching

Industry use cases

Types of cloud service provisioning

The problem to scale..

Resourse scheduler - Solution 1

Resourse scheduler - Solution 2

Demo

Q&A



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

3

Introduction to Cloud Caching Service

Cache Component that stores data so future requests for that data can be served faster. Distributed Cache Extension of the traditional concept of cache used in a single locale. A distributed cache may span multiple servers so that it can grow in size and in transactional capacity. It is mainly used to store application data residing in database and web session data. The distributed cache architecture is built on scale-out, with inherent partitioning capabilities to spread the workload. Cloud Service Any resource that is provided over the Internet. The most common cloud service resources are Software as a Service (SaaS), Platform as a Service (PaaS) and Infrastructure as a Service (IaaS).

users cache

http://searchcloudcomputing.techtarget.com/definition/Software-as-a-Service

http://searchcloudcomputing.techtarget.com/definition/Platform-as-a-Service-PaaS

http://searchcloudcomputing.techtarget.com/definition/Infrastructure-as-a-Service-IaaS



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

4

• We focus on in-memory caches and exclude

others (ssd, multi-tier, etc) from our discussion



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

5

Major in-Memory Cache Uses Cases

Database offloading in case of slow database or disk access Web content caching in case of high latency and narrow bandwidth to actual servers Low latency data access for time critical operations Pub/Sub mechanisms for multiple services and multiple consumers Message queues distributed network message passing mechanisms CDN caches in-memory caches especially for content index and ranking, not necessarily for content itself Caching and computation sorted sets, lists, queues, etc



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

6

Leading Industry Cache Offerings

Redis Memcached Apache Hbase CounchBase Basho Riak Oracle Coherence DataStax Cloudant AeroSpike

See other players in the domain of caching: https://www.g2crowd.com/products/redis/competitors/alternatives

https://www.g2crowd.com/products/redis/competitors/alternatives



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

7

• We chose redis for cloud integration evaluation..



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

8

Most popular in-memory cache in the world

Redis is an open source (BSD licensed), in-memory data structure store, used as database,

cache and message broker. It supports data structures such as strings, hashes, lists, sets,

sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius

queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and

different levels of on-disk persistence, and provides high availability via Redis Sentinel and

automatic partitioning with Redis Cluster.

Web

App

DB

Web

App Web

App Web

App Volatile/temp data

Why Redis NoSQL?

No lock execution

Effectively uses individual core per

instance

Lot of data structure use cases

Blazingly fast for a single core

performer

http://redis.io/topics/data-types-intro







http://redis.io/commands/geoadd

http://redis.io/topics/replication

http://redis.io/commands/eval



http://redis.io/topics/lru-cache

http://redis.io/topics/transactions

http://redis.io/topics/persistence



http://redis.io/topics/sentinel



http://redis.io/topics/cluster-tutorial





Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

9

• The problem to scale…



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

10

Redis Service in cloud – major Vendors 1) Amazon - Elastic Cache: it easy to deploy, operate, and scale an in-memory cache in the cloud. It

support Redis Master/Slavce replication and Multi AZ - https://aws.amazon.com/elasticache/

2) Microsoft - gives you Redis reliable hosting and monitoring - https://azure.microsoft.com/en-us/documentation/services/redis-cache/

3) Google – Deploy Redis on compute Engine. With Redis-as-a-Service, Redislabs will manage and monitor your instance on Google Cloud Platform. https://cloud.google.com/solutions/redis/

4) IBM - Support Redis service on SoftLayer - https://marketplace.ibmcloud.com/apps/87#!overview

5) RedHat OpenShift – Redis support for community-contributed cartridges - https://blog.openshift.com/enabling-redis-for-your-app/

6) Digital Ocean – Supports using Redis in the cloud - https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis

7) Alibaba Aliyun – Aliyun Key-Value Store (KVStore for short) is an online Key-Value storage service compatible with the open-source Redis protocol - http://www.aliyun.com/product/kvstore/?lang=en

8) Redis Labs – Enterprise cluster; offered as a fully managed service, also used as third party offering by other vendors.

https://aws.amazon.com/elasticache/

https://azure.microsoft.com/en-us/documentation/services/redis-cache/





https://cloud.google.com/solutions/redis/

https://marketplace.ibmcloud.com/apps/87

https://blog.openshift.com/enabling-redis-for-your-app/









https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis











http://www.aliyun.com/product/kvstore/?lang=en



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

11

One view of Redis service provisioning in cloud

Instance on Bare metal - Toughest to maintain, but provides best performance - A single instance wastes rest of the cpu resources (complicated

multi-tenancy)

Instance on Virtual Machine - Lesser performance - Scaling means provisioning another VM

Instance in a container - Lesser performance, given the network translations - Scaling is easy, but seamless is difficult if the redis cluster is non

uniform (server is different type, proxy is another, etc) • Eventually all the above is resources. • Service Instances could be placed anywhere in the pool. • In fact different types could coexist, and communicate, if the

network allows so.



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

12

• The problem to scale given resources..

needs resource scheduling…



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

13

Redis service instance topologies

When a customer requests for a redis instance from cloud it could be one for the following configuration.

Single Redis-Server •CPU = 1 •mem = variable •Disk = optional

Master/Slave Redis • Primarily for replication • CPU= 1 + N Slaves • mem = variable • Disk = optional

master

Slave 1 Slave 2

Proxy

Redis Cluster with Proxy • Only Using twem proxy or Codis for data shard • Popular with older version of redis eg:2.8 • CPU= Number of nodes

Redis 3.0 Cluster •Latest offering from Redis ; No proxy required. •Possible to horizontally scale ; CPU= Number of nodes



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

14

Proxy based sharding Twitter handled redis cluster by implementing twemproxy

Shard aware clients

External Cluster manager Wayfair an online furniture store managed redis cluster using zookeeper

Enterprise Cluster Redis Labs Enterprise cluster - https://redislabs.com/redis-cloud

Redis 3.0 http://redis.io/download

How traditionally Industry handled Redis so far…

https://redislabs.com/redis-cloud



http://redis.io/download



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

15

• We tried couple of solutions using

mesos and kubernetes…



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

16

Mesos Kubernetes

Apache Mesos is a cluster manager that provides efficient

resource isolation and sharing across distributed

applications, or frameworks.

Apache Mesos abstracts CPU, memory, storage, and other

compute resources away from machines (physical or

virtual), enabling fault-tolerant and elastic distributed

systems.

Google Kubernetes brings 'Google style' cluster

management capabilities to the world of Docker

containers on the data center scenarios.

For production Google uses Omega/Borg, not Kubernetes

which is open sourced in June 2014.

• Mesos using same principle as Linux Kernel but higher

layer of abstraction.

• It can run Hadoop, MPI, Hypertable, Spark, and other

frameworks on a dynamically shared pool of nodes.

• Manage a cluster of Linux containers as a single system.

• There is a Kubernetes Framework for Mesos by

Mesosphere.

Was developed at the University of California, Berkeley

year 2009 developed by Benjamin Hindman, Andy

Konwinski, Matel Zaharia as well as professor Ion Stoica

Become top level Apache project from July 2013

Project Started at Google

Recently launched 1.0 production code (2nd half 2015)

Getting more popular day by day along with Docker.

Works with various Frameworks like Marathon, Cronos. You can

write your own. There is a core algorithm(DRF) to share

resources and you can define your own scheduling policy.

Frameworks does Fault Tolerance, Load balancing, Quotas,

Discovery, etc.

Kubernetes establishes robust declarative primitives for

maintaining the desired state requested by the user. Self-

healing mechanisms, such as auto-restarting, re-scheduling,

and replicating containers require active controllers, not just

imperative orchestration.

Introduction



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

17

Mesos Kubernetes

The Architecture



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

18

Solution 1 : Kubernetes (Cluster manager)

We could provide redis instance in docker containers

Kubernetes has a Mesos framework which can co-exist with

other frameworks/workloads in the same Mesos cluster

The concept of ‘POD’ can be used to club multiple containers

together. Kubernetes ensures that these containers are started

on the same slave

Kubernetes automatically provides fault- tolerance

Why kubernetes?



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

19

Mesos

Master

Kubernetes

Framework for

Mesos

Cloud service

broker Kube- Executor kube- Executor Kube- Executor Kube- Executor

Redis

Request

from cloud Each Service instance will be a POD, provisioned

for given type

Solution 1 : Kubernetes as fw of mesos



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

20

Cannot add additional containers in a already running POD. Very

necessary for Redis 3.0 horizontal scalability. Below discussion in

google groups explains the implementation difficulties in

Kubernetes https://groups.google.com/forum/#!topic/google-

containers/_Gz9dH2hqg4

However it’s a popular demand from the kubernetes community

Limited Flexibility by using Kubernetes Framework

Solution 1 : Kubernetes (Cluster manager) Contd..

Limitations..

https://groups.google.com/forum/





Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

21

Implement a simplistic Mesos framework for Long Running

Process

Use “mesos-go” an opensource framework library written in

golang for development

Check feasibility for vertical scaling of redis-instances

Collect statistics against each redis instance

Enable the fw to ensure instances can communicate to create a

cluster (user 3.0)

Solution 2: Mesos Framework dealing directly with Redis

Instances

Scope



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

22

New Redis Mesos Framework

Mesos

Master

Redis

Framework for

Mesos

Cloud service

broker Redis Executor Redis Executor Redis Executor Redis Executor

Redis

Request

from cloud



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

23

Redis Executor Redis Executor

POC : Design of Simple Redis Framework

Redis Scheduler

http server Monitor

Master

Select offer

&

Submit tasks

Collect Stats

Redis Executor

Monitor

pid=123

http server

UI

PID = 123

PID = 124

PID = 125

Monitor

pid=124

Monitor

pid=125

Launch Task

Mesos

Master

Mesos Slaves



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

24

To build a Mesos framework we have to develop two components.

Scheduler – Talk to the mesos master and submit tasks

Executor – Execute your tasks at the slave

By Default Mesos considers each work load is a task not LRP (Long Running Process)

What is a Mesos framework (a developers perspective):

Usual Logic of Scheduler Usual Logic of Executor

Loop (Wait for offer from Mesos Master) if (offer.Resource > Task.Resource) then Select Offer else Reject Offer Repeat

Loop (wait for tasks to Launch ) Update task.State = STARTED execute task Update task.State = Finished / Error Repeat


Instances



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

25

Its easier to write a Mesos framework

We have better controller & flexibility over the running redis

instances

We can collect better statistics on the running tasks

We can plan migration of running tasks among different slaves

We can group bunch of running processes and allocate them in a

single slave (like kubernetes POD)

We would be able to offer Redis 3.0 and scale Horizontally

We could use any container technology of our choice like Docker

or RKT or RunC in bare-metal servers for additional instance

security

WHY and what more can we do?


Instances



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

26

DEMO



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

27

Q&A



Color: R153 G0 B0

Corporate Font :



partners :

Arial

Slide text :20-22pt

Bullets level 2-5:

18pt

Color:Black

Corporate Font :



partners :

Arial

Top right corner

for field-mark,

customer or

partner logotypes.

----------------

The following

nine groups of

colors are an

example of how

our design colors

can be used,

please take note

that you should

only use one

design color

group per slide.

For specific

usage details,

refer to the

“Typesetting

Standard”.

28

Astro Project – Spark SQL on Hbase Astro is fully distributed SQL on Hbase solution based on Spark Ecosystem

100% Open source Any contributions are welcome.

HBase

Spark

Astro

Spark Application

SQL ML Graph Stream Community

Package

•Astro query processing is fully distributed. •CRUD(Create, Read, Update and Delete) operations support •Fully utilizing Spark SQL query optimizer. •Advanced optimizations to push down computations close to storage. •Co-Location along with Hbase.

As Astro plugs into spark, Astro query results can be seamlessly used for further in-depth analysis using Spark SQL DataFrame, MLLib, GraphX etc…

Address： https://github.com/HuaweiBigData/astro

Compared to existing SQL on Hbase solution the major Advantages are:

Highly scalable caching service on cloud - Redis

Software

Transcript of Highly scalable caching service on cloud - Redis