Home of RedisWhat’s New with Enterprise Redis
Leena Joshi
2
Agenda
• What is Enterprise Redis?
• Redis on Flash
• Redis and Spark
• Redis Module Hub
3
Enterprise Conjures Up..
OR
4
But What It Really Means Is:
5
Which In Turn Means:
Infinite Seamless Scaling
True High Availability
Top notch expert support
6
Redis Labs Enhances OSS RedisRedis Labs Node
Open Source
Zero latency proxy Cluster Manager
REST API
Odd number of nodes needed to handle network splits- not three copies of data
Redis Labs Cluster• Shared nothing cluster
architecture• Fully compatible with open
source commands & data structures
Proprietary
7
The Same Technology Runs Redis Cloud
Cluster Management Path
ProxiesNode WatchdogCluster Watchdog
Node 1 Node 2 Node N (uneven number)…
Redis Shards
Unique multi-tenant “Docker” like architecture enables running hundreds of databases over a single, average cloud instance without performance degradation and with maximum security provisions
Data Path
Distributed ProxiesSingle or Multiple Endpoints
50,000+ Customers
8
Tremendous Customer TractionRedis Cloud
Available since mid-2013
6000+ enterprise customers
Redis Labs Enterprise Cluster (RLEC)Available since early-2015
100+ enterprise customers
9
Always On - Highly Available & Persistent
Simple, SeamlessClustering. Linear
Scalability.
Enterprise-Class Management and
Support
Enterprise-Class Redis – The Benefits
Stable & Predictable Top Performance
Operational Cost Savings
10
Simple, SeamlessScaling and Clustering
Auto- scaling/re-sharding/re-balancing
No downtime while scaling
Supports cross-shard operations
Simple, Seamless Clustering. Linear Scalability.
Linear Scalability
11
Always On - Highly Available & Persistent
Seamless cross datacenter/region/cloud
replication
Instant auto-failover
Persistence, backups and DR
Always On - Highly-Available & Persistent
12
Stable & Predictable Top Performance
Consistent high performance achieved under any load or
cluster size
Database processed by multiple cores
Built-in performance enhancement techniques
Stable & Predictable Top Performance
13
Operational Cost Savings
OSS Redis Redis Labs
More efficient hardware utilization: fewer servers, lower power & cooling and operational costs
Reduced manual labor through automation - reduced time writing scripts, scaling,
configuration, monitoring, re-balancing and more
Run Redis on flash memory as RAM extender – up to 10 times cheaper
Reduced downtime incidents
Shorten time to deploy Redis by over 50%
14
Enterprise Management and Support
UI, CLI, REST API -based management & alerting
Proven technology supporting thousands of customers
24x7 enterprise support, top notch Redis expertise
Enterprise-Class Management & Support
15
Redis Labs: Fastest Recovery, No Data Loss
% o
f tim
es d
ata
was
lost
Aver
age
time
to re
cove
r in
seco
nds
Redis Labs recovers in 5 seconds and does not lose data.All other vendors lose data and take many minutes to recover
Vendors evaluated include(not in order)• Heroku Redis• AWS ElastiCache• Microsoft AzureCache• Compose.io
16
Redis Labs: The Only True HA Redis16
Failure Event In-memory Replication
Multi-DC/Zone replication
Auto-failover AOF Data Persistence
Backup (using snapshots)
Multi-region/Cloud replication
Process failure Instant recovery* Slow recoveryNode failure Instant recovery* Slow recoveryMulti-node failure Instant recovery*Network split Instant recovery*Zone/Rack failure Instant recovery* Slow recovery Fast recoveryRegion/Cloud failure Slow recovery Fast recovery
Type
of o
utag
e
Essential features for high availability
*Auto-failover should run on same nodes as Redis deployment
Redis Labs provides all the essential HA features that protect against every type of outage
Redis on Flash
18
Why Analyze Data In-Memory?“Information is the oil of 21st century, and analytics is the combustion engine”- Peter Sondergaard, Gartner Analyst
19
Decision Speeds Are Accelerating..
“Big Data” gains popularity as tools become available to harness it
Batch insights start to drive business
Real time insights automate decision-
making
2005 2012 - 2015 2016…
THE DATA REVOLUTION IS MATURING..
20
The Race Is On..INSIGHTS FROM YOUR DATA NEED TO BE
INSTANTANEOUS
COST EFFECTIVE
21
Price/Performance of Memory Technology
22
Redis on Flash Flash used as a RAM extender and NOT as persistent storage
23
Redis on Flash Concepts
• Flash used as a RAM extender (NOT as a persistent storage)• Global key list in RAM; ‘hot’ values in RAM; ‘cold’ values on Flash. • Multi-threaded & async Redis when accessing objects on Flash.
Utilizes multi-core and Flash concurrency architecture• 100% compatibility with Redis
24
How to Achieve Optimal Price/PerformanceBy dynamically setting RAM/Flash ratio
25
Single Server Results with Dell & Samsung NVMe
readwrite
readwrite
Avg: 2.04M ops/sec
Max: 2.14M ops/sec
Avg: 0.91msec
Max: 0.98 msec
% below 1msec: 100%
Avg: 313RMB / 9.4WMB
Max: 1.71RGB / 96WMB
Avg: 1.45Gbps (Tx) / 0.97Gbps (Rx)
Max: 1.6Gbps (Tx) / 1.2Gbps (Rx)
Test setup:• Redis Labs Enterprise
Cluster v3.2• Dell Xeon CPU E5-
2670 v3 @ 2.50GHz• 4x Samsung NVMe
PM1725• Memtier benchmark-
open source tool• 100B object size• 80% read• 20% write
Throughput – ops/sec
Latency – msec
Disk Bandwidth – MB/sec
NW Bandwidth – Gb/sec
>2M Ops/sec, <1 ms latency, > 1GB disk bandwidth
26
A Real Life Example With Redis On Flash
Customer Scenario:• Genome dataset • Key sizes: 32B, value sizes : 5-12B• No of keys: 250 x109… 250 x1012
Key1: AAAAAAAAAAAAAAAAAAAAAAAACCCCAAA Value1 = Freq=4 IE=A OE=A
Key2: AAAAAAAAAAAAAAAAAAAAAAAAACAACCC Value2 = Freq=7 IE=A,C,T,G OE=A,C,T,G
*IE – Inside End sequences
*OE – Outside End sequences
27
Optimizing Redis UsageRAW
• # of keys 250x10^9
• Key size = 32B
• Value size = 8B (average)
• Overhead per object (key+value) = ±61B (key) + 9 (value) = 70
• Internal fragmentation per object = 14B
• RAM size = ±31TB
28
Further OptimizationsEncoding keys and values to compress sizes
• # of keys 1.25x10^9
• Key size = 4B
• Value size = 3612B
• RAM overhead per object (key+value) = ±40B
• RAM size = ±55GB // for optimal performance we used 500GB to keep 10/90 RAM/Flash ratio
• Flash size = ±4.5TB
Using Redis Hashes to store compressed keys/values (200 keys and values per hash)
1 2
29
Memory Usage and Cost Comparison
Redis on RAMStrings
Redis on FlashHashes
RAM size 31TB 0.5 TB
Flash size - 4.5TB
EC2 instances 155 x r3.8xlarge 2 x i2.8xlarge
1yr costs (reserved instances)
$2,017,325 $49,862
Savings with Flash & Hashes %
97.6%
Home of Redis
Redis in Analytics
31
Spark & Redis - Connector & Service Layer
Data Source Serving Layer
Spark SQL &Data Frame
RDD, Data Source,Data Set
RDD, Data Source,Data Set
Analytics & BI
32
Spark & Redis – Internal Accelerator
Data Source
RDD, Data Source,Data Set
RDD, Data Source,Data Set
Spark SQL &Data Frame
Analytics & BI
RDD, Data Source,
Data Set,Redis API
Data Sink
33
Accelerate Spark Time-Series with Redis
Redis sorted sets accelerate time series data processing by 100 times compared to other in-
memory K/V stores
Example time series data: Stock prices for 1024 stocks over 32 years
34
Spark-Redis Package : The Results
Redis faster by upto 100 times compared to HDFS and over 45 times compared to Tachyon or Spark
Home of Redis
Redis Modules
36
36
Modules Extend Redis’ Use Case CoverageMongoDB
Cassandra/ Datastax
Couchbase Redis (original) Redis + Modules
Single View Coming Soon
Personalization
Catalog Coming Soon
IoT
Real-Time Analytics
Content Management Coming Soon
Messaging
Fraud Detection Coming Soon
Graph Coming Soon
Time Series
Caching
Text Search Coming Soon
Image Processing Coming Soon
Machine Learning Coming Soon
Linear Algebra Coming Soon
Probabilistic data structures for processing continuous,
Coming Soon
More
37
Modules Turn Redis into a Multi-Model Database37
MongoDBCassandra/
DatastaxCouchbase Redis (original) Redis + Modules
Document-Based Coming Soon
Column-Based Coming Soon
Key-Value Data Structures Data Structures
Graph Coming Soon
38
Redis Module Hub
• A Redis Module Marketplace – for everyone
• Every problem a developer solves with Redis – now extended to the Enterprise
• Will help developers monetize their work and reach enterprise Redis users
• Will give enterprise Redis users the confidence and peace of mind to easily deploy modules
www.redismodules.com
39
4040
Full text search Prefix search0.00
5.00
10.00
15.00
20.00
25.00
30.00
3.15 2.40
21.00
8.70
24.57
10.61
Average Latency (msec)
RLEC Elasticsearch Solr
Full text search Prefix search0
5,000
10,000
15,000
20,000
25,000
20,045
6,831
690
3,686
621
3,133
Ops/sec
RLEC Elasticsearch Solr
85% higher
32x higher
7.8x faster 4.1x faster
redisearch
The world fastest text search engine
41
What Can Modules Do41
• All modules are certified by Redis Labs for full compliance with OSS Redis, Redis Cloud and Redis Labs Enterprise Cluster (RLEC)
Full Text Search Enhanced JSON Graph Operations Secondary Indexes
Linear Algebra SQL Support Image Processing N-Dimension Queries …
Thank You!
Top Related