Redis: Need for speed
-
Upload
elena-kolevska -
Category
Software
-
view
273 -
download
1
Transcript of Redis: Need for speed
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 1
About Our Company
REDIS: NEED FOR SPEEDElena Kolevska
2
Hello World!
A free spirit, traveller, capoeirista, mum, music lover, wannabe entrepreneur,
proud geek.
Elena Kolevska Software engineer
We're going to talk about Redis:
Things you didn't know
How much faster does it get?
2
HOW MUCH FASTER DOES IT GET?
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 4
How much faster does it get?
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751TOTAL TWEETS
751Main stream 120sub-stream foo 107sub-stream bar
...Foo
Bar
Baz
0 30 60 90 120
86
107
120
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 4
How much faster does it get?
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751TOTAL TWEETS
751Main stream 120sub-stream foo 107sub-stream bar
...Foo
Bar
Baz
0 30 60 90 120
86
107
120
Total tweets
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 4
How much faster does it get?
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751TOTAL TWEETS
751Main stream 120sub-stream foo 107sub-stream bar
...Foo
Bar
Baz
0 30 60 90 120
86
107
120
Total tweets Main Stream
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 4
How much faster does it get?
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751TOTAL TWEETS
751Main stream 120sub-stream foo 107sub-stream bar
...
Substream:keyword 1
Foo
Bar
Baz
0 30 60 90 120
86
107
120
Total tweets Main Stream
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 4
How much faster does it get?
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751TOTAL TWEETS
751Main stream 120sub-stream foo 107sub-stream bar
...
Substream:keyword 1 Substream:keyword 2
Foo
Bar
Baz
0 30 60 90 120
86
107
120
Total tweets Main Stream
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 4
How much faster does it get?
OUR SCENARIO: TWITTER ANALYZER DASHBOARD
751TOTAL TWEETS
751Main stream 120sub-stream foo 107sub-stream bar
...
Substream:keyword 1 Substream:keyword 2
Foo
Bar
Baz
0 30 60 90 120
86
107
120
Total tweets
Scores
Main Stream
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 5
How much faster does it get?
THE PROCESS:
This is a tweet about the
words foo and bar
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 5
How much faster does it get?
THE PROCESS:
This is a tweet about the
words foo and bar
Send tweet to main stream
Main stream
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 5
How much faster does it get?
THE PROCESS:
This is a tweet about the
words foo and bar
Send tweet to main stream
Main stream
Increase counter
of total tweets
Total tweets counter
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 5
How much faster does it get?
THE PROCESS:
This is a tweet about the
words foo and bar
foo
Send tweet to appropriate sub-stream
Stream foo
Stream barbar
Send tweet to main stream
Main stream
Increase counter
of total tweets
Total tweets counter
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 5
How much faster does it get?
THE PROCESS:
This is a tweet about the
words foo and bar
foo
Send tweet to appropriate sub-stream
Stream foo
Stream barbar
Increase counters for
streams foo/bar
Counter foo
Counter bar
Send tweet to main stream
Main stream
Increase counter
of total tweets
Total tweets counter
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 6
How much faster does it get?
THE STEPS:
•Save the complete tweet
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for total tweets
•Add tweet to main feed
•Increase counter for found keywords
•Add tweet to sub-feed
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 7
How much faster does it get?
Redis is TCP server using client/server model. A request to the server is accomplished in two steps:
•The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response
•The server processes the command and sends the response back to the client
Client Server
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 7
How much faster does it get?
Redis is TCP server using client/server model. A request to the server is accomplished in two steps:
•The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response
•The server processes the command and sends the response back to the client
Client Server
Networking link
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 7
How much faster does it get?
Redis is TCP server using client/server model. A request to the server is accomplished in two steps:
•The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response
•The server processes the command and sends the response back to the client
Client Server
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 7
How much faster does it get?
Redis is TCP server using client/server model. A request to the server is accomplished in two steps:
•The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response
•The server processes the command and sends the response back to the client
Client Server
Request from client to server
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 7
How much faster does it get?
Redis is TCP server using client/server model. A request to the server is accomplished in two steps:
•The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response
•The server processes the command and sends the response back to the client
Client Server
Request from client to server
Response from server to client
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 8
How much faster does it get?
Redis is TCP server using client/server model. A request to the server is accomplished in two steps:
•The client sends a query to the server, and reads from the socket, usually in a blocking way, for the server response
•The server processes the command and sends the response back to the client
Client Server
Request from client to server
Response from server to client
RTT: Round Trip Time
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 9
How much faster does it get?
THE STEPS:
SET request to the server
•Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 9
How much faster does it get?
THE STEPS:
SET request to the server
1Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 10
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
1
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 10
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
1
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 10
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
2
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 10
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
+ LTRIM
2
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 10
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
+ LTRIM
3
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 11
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
3
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 11
How much faster does it get?
THE STEPS:
INCR
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
3
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 11
How much faster does it get?
THE STEPS:
INCR
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
4
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 12
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
4
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 12
How much faster does it get?
THE STEPS:
SMEMBERS
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
4
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 12
How much faster does it get?
THE STEPS:
SMEMBERS
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
5
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 13
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
5
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 13
How much faster does it get?
THE STEPS:
Logic on client side
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
5
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 14
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords•Add tweet to sub-feed
•Update score
5
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 14
How much faster does it get?
THE STEPS:
INCR
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords•Add tweet to sub-feed
•Update score
5
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 14
How much faster does it get?
THE STEPS:
INCR
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords•Add tweet to sub-feed
•Update score
6
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 15
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed•Update score
6
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 15
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed•Update score
6
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 15
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed•Update score
7
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 15
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed•Update score
7
+ LTRIM
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 15
How much faster does it get?
THE STEPS:
LPUSH
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed•Update score
+ LTRIM
8
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 16
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
8
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 16
How much faster does it get?
THE STEPS:
ZINCRBY
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
8
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 16
How much faster does it get?
THE STEPS:
ZINCRBY
Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
9
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 17
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
9
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 17
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
9
} + 4 requests for every found keyword
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 17
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score
13?
} + 4 requests for every found keyword
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 17
How much faster does it get?
THE STEPS:Number of requests to
server •Save the tweet
•Add tweet to main feed
•Increase counter for total tweets
•Get all monitored keywords
•Check tweet for any keywords
•Increase counter for found keywords
•Add tweet to sub-feed
•Update score } + 4 requests for every found keyword
17?
LEVEL1: PIPELINING
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 19
How much faster does it get?
Using the PRedis library for PHP</php
$keywords = $client->smembers("keywords");
$responses = $client->pipeline(function ($pipe) use ($keywords) {
$pipe->set(...); // Save the tweet $pipe->lpush(...); // Save the tweet to main feed $pipe->ltrim(...); // Trim the main feed to X tweets so it doesn't grow forever
foreach ($keywords as $keyword) { $pipe->incr(...); // Increase the counter for the keyword $pipe->lpush(...); // Add to the subbed $pipe->ltrim(...); // Trim the stream to X tweets $pipe->zincrby(...); // Update score }
});
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 19
How much faster does it get?
Using the PRedis library for PHP</php
$keywords = $client->smembers("keywords");
$responses = $client->pipeline(function ($pipe) use ($keywords) {
$pipe->set(...); // Save the tweet $pipe->lpush(...); // Save the tweet to main feed $pipe->ltrim(...); // Trim the main feed to X tweets so it doesn't grow forever
foreach ($keywords as $keyword) { $pipe->incr(...); // Increase the counter for the keyword $pipe->lpush(...); // Add to the subbed $pipe->ltrim(...); // Trim the stream to X tweets $pipe->zincrby(...); // Update score }
});
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 19
How much faster does it get?
Using the PRedis library for PHP 1Number of requests to
server </php
$keywords = $client->smembers("keywords");
$responses = $client->pipeline(function ($pipe) use ($keywords) {
$pipe->set(...); // Save the tweet $pipe->lpush(...); // Save the tweet to main feed $pipe->ltrim(...); // Trim the main feed to X tweets so it doesn't grow forever
foreach ($keywords as $keyword) { $pipe->incr(...); // Increase the counter for the keyword $pipe->lpush(...); // Add to the subbed $pipe->ltrim(...); // Trim the stream to X tweets $pipe->zincrby(...); // Update score }
});
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 20
How much faster does it get?
Using the PRedis library for PHP 2Number of requests to
server </php
$keywords = $client->smembers("keywords");
$responses = $client->pipeline(function ($pipe) use ($keywords) {
$pipe->set(...); // Save the tweet $pipe->lpush(...); // Save the tweet to main feed $pipe->ltrim(...); // Trim the main feed to X tweets so it doesn't grow forever
foreach ($keywords as $keyword) { $pipe->incr(...); // Increase the counter for the keyword $pipe->lpush(...); // Add to the subbed $pipe->ltrim(...); // Trim the stream to X tweets $pipe->zincrby(...); // Update score }
});
LEVEL2: LUA SCRIPTING
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 22
How much faster does it get?
Create a hello.lua file:
return "Hello"
> redis-cli --eval /path/to/script/hello.lua"Hello"
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 22
How much faster does it get?
Create a hello.lua file:
return "Hello"
> redis-cli --eval /path/to/script/hello.lua"Hello"
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 23
How much faster does it get?
hello.lua:local tweet = ARGV[1] local key_tweet = KEYS[1] local key_keywords = KEYS[2] local key_total_tweets_count = KEYS[3] local key_scores = KEYS[4] local key_main_feed = KEYS[5]
redis.call("SET", key_tweet, tweet) -- Save the tweet redis.call("INCR", key_total_tweets_count) -- Increase the total tweet count redis.call("LPUSH", key_main_feed, tweet) -- Push the tweet to the main feed redis.call("LTRIM", key_main_feed, 0, 100) -- Trim the main feed
local keywords = redis.call("SMEMBERS", key_keywords) -- Get the keywords
for i, name in ipairs(keywords) do if string.find(tweet, name) then local substream_name = "sub_feed:" .. name redis.call("LPUSH", substream_name, tweet) -- Push the tweet to the sub feed redis.call("LTRIM", substream_name, 0, 100) -- Trim the sub feed redis.call("ZINCRBY", key_scores, 1, name) -- Increment the score for the keyword in the leaderboard end end
return "OK"
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 24
How much faster does it get?
> redis-cli --eval hello.lua tweet_123 keywords ... , "A tweet about the words foo and bar""OK"
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 24
How much faster does it get?
> redis-cli --eval hello.lua tweet_123 keywords ... , "A tweet about the words foo and bar""OK"
1!Number of requests to
server
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 25
How much faster does it get?
SCRIPT
•Evaluates a script cached on the server side by its SHA1 digest. The command is otherwise identical to EVAL.
> redis-cli SCRIPT LOAD "$(cat hello.lua)""6d52847f03028ab1d4620b60dd6ef4a14c8727d7"
> redis-cli evalsha 6d52847f03028ab1d4620b60dd6ef4a14c8727d7 5 \ > tweet_123 keywords ... "A tweet about the words foo and bar""OK"
EVALSHA
•Load a script into the scripts cache, without executing it. Returns a SHA-1 digest of the script.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 26
How much faster does it get?
CONS:
•While your script is running, everything else is on hold
•Trickier to maintain
PROS:•Speed
•Atomicity
•Logic moves out of your app layer
•Logic moves out of your app layer
WARP DRIVE: REDIS MODULES
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 28
How much faster does it get?
What are Redis modules?
•Dynamically loaded libraries
•Written in C
•Almost as fast as the Redis core
•Let you extend Redis commands, create new data structures, access data almost as fast as native Redis commands
•Add-ons to Redis
•Coming in version 4.0, currently in Beta (RC)
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 29
How much faster does it get?
Layers of the Modules API:
•Low-level: Close to native access to core data structures
•High-level: Client-like access to core and modules' commands
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 30
How much faster does it get?
0
0.35
0.7
1.05
1.4
Python Lua M:HighLevel M:LowLevel
1.2 1.25
1.05
0.1
Time needed for summing 1 000 000
Sorted Set Scores
Statistics from Itamar Haber @ redislabs
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 31
How much faster does it get?
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 32
How much faster does it get?
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 33
How much faster does it get?
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 34
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 34
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
The context object
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 34
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
Command
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 34
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
Arguments
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 34
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
Format specifier
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 35
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 35
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
Format specifier
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 36
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 36
How much faster does it get?
High level API
RedisModule_Call(ctx,"INCR","sc",argv[1],"10");
Format specifier
c -- Null terminated C string pointer.
b -- C buffer, two arguments needed: C string pointer and size_t length.
s -- RedisModuleString as received in argv or by other Redis module APIs returning a RedisModuleString object.
l -- Long long integer.
v -- Array of RedisModuleString objects.
! -- This modifier just tells the function to replicate the command to slaves and AOF. It is ignored from the point of view of arguments parsing.
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
Definitions of the API
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
Definitions of the API
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
Definitions of the APICalled whenever the module is loaded Must be present in each Redis module
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
Definitions of the API
Register the module or error out
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
Definitions of the API
Register the command or error out
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
Definitions of the API
Command name
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 37
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "tweet_processor", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "tweet.process", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
Definitions of the API
Function name
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 38
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 38
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
The logic of the module
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 39
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
// We need EXACTLY 5 arguments if (argc != 5) return RedisModule_WrongArity(ctx);
RedisModule_AutoMemory(ctx);
RedisModuleCallReply *reply; reply = RedisModule_Call(ctx, "INCR", "s", argv[3]); RedisModule_ReplyWithCallReply(ctx, reply);
return REDISMODULE_OK; }
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 39
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
// We need EXACTLY 5 arguments if (argc != 5) return RedisModule_WrongArity(ctx);
RedisModule_AutoMemory(ctx);
RedisModuleCallReply *reply; reply = RedisModule_Call(ctx, "INCR", "s", argv[3]); RedisModule_ReplyWithCallReply(ctx, reply);
return REDISMODULE_OK; }
Enable automatic memory management
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 39
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
// We need EXACTLY 5 arguments if (argc != 5) return RedisModule_WrongArity(ctx);
RedisModule_AutoMemory(ctx);
RedisModuleCallReply *reply; reply = RedisModule_Call(ctx, "INCR", "s", argv[3]); RedisModule_ReplyWithCallReply(ctx, reply);
return REDISMODULE_OK; }
Call Redis commands
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 39
How much faster does it get?
Anatomy of a module
#include "redismodule.h" #include <stdlib.h>
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // ... }
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) { // Register the module or error out if (RedisModule_Init(ctx, "process_tweet", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR) return REDISMODULE_ERR;
// Register the command or error out if (RedisModule_CreateCommand(ctx, "process_tweet.rand", ProcessTweet_RedisCommand,
"readonly", 1,1,1) == REDISMODULE_ERR) return REDISMODULE_ERR;
return REDISMODULE_OK; }
int ProcessTweet_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
// We need EXACTLY 5 arguments if (argc != 5) return RedisModule_WrongArity(ctx);
RedisModule_AutoMemory(ctx);
RedisModuleCallReply *reply; reply = RedisModule_Call(ctx, "INCR", "s", argv[3]); RedisModule_ReplyWithCallReply(ctx, reply);
return REDISMODULE_OK; }
Reply with a call object
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 40
How much faster does it get?
Low level API
RedisModule_AutoMemory(ctx);
RedisModuleKey *key;
key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ|REDISMODULE_WRITE);
if (RedisModule_KeyType(key) == REDISMODULE_KEYTYPE_EMPTY) { RedisModule_StringSet(key,argv[2]); }
RedisModule_CloseKey(key);
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 40
How much faster does it get?
Low level API
RedisModule_AutoMemory(ctx);
RedisModuleKey *key;
key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ|REDISMODULE_WRITE);
if (RedisModule_KeyType(key) == REDISMODULE_KEYTYPE_EMPTY) { RedisModule_StringSet(key,argv[2]); }
RedisModule_CloseKey(key);
Open a key. Return a key pointer
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 40
How much faster does it get?
Low level API
RedisModule_AutoMemory(ctx);
RedisModuleKey *key;
key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ|REDISMODULE_WRITE);
if (RedisModule_KeyType(key) == REDISMODULE_KEYTYPE_EMPTY) { RedisModule_StringSet(key,argv[2]); }
RedisModule_CloseKey(key);
Checks the type of the key
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 40
How much faster does it get?
Low level API
RedisModule_AutoMemory(ctx);
RedisModuleKey *key;
key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ|REDISMODULE_WRITE);
if (RedisModule_KeyType(key) == REDISMODULE_KEYTYPE_EMPTY) { RedisModule_StringSet(key,argv[2]); }
RedisModule_CloseKey(key);
Set a string value for a key
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 40
How much faster does it get?
Low level API
RedisModule_AutoMemory(ctx);
RedisModuleKey *key;
key = RedisModule_OpenKey(ctx,argv[1],REDISMODULE_READ|REDISMODULE_WRITE);
if (RedisModule_KeyType(key) == REDISMODULE_KEYTYPE_EMPTY) { RedisModule_StringSet(key,argv[2]); }
RedisModule_CloseKey(key);
Close the key
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 41
How much faster does it get?
Compiling
$ gcc -fPIC -std=gnu99 -c -o process_tweet process_tweet.c $ ld -o process_tweet.so process_tweet.o -shared -Bsymbolic -lc
On Linux:
$ gcc -dynamic -fno-common -std=gnu99 -c -o process_tweet.o process_tweet.c $ ld -o process_tweet.so process_tweet.o -bundle -undefined dynamic_lookup -lc
On OSX:
Loading
./redis-unstable/src/redis-server --loadmodule ./process_tweet.so
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 42
How much faster does it get?
Calling the command
127.0.01:6379> tweet.process tweet_id:42 "This is my tweet about the words foo and bar" ...
Our command
THINGS YOU DIDN'T KNOW
THINGS YOU probably DIDN'T KNOW
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 45
Big O Notation: O(1) commands
•ZSCORE
•LPOP, BLPOP, LPUSHX, BRPOPLPUSH
•EXISTS
•GET, GETSET, HGET, GETBIT
•GETRANGE, SETRANGE
•HLEN, HSTRLEN
•LLEN
•PFADD, PFCOUNT
•SISMEMBER
•APPEND
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 45
Big O Notation: O(1) commands
•ZSCORE
•LPOP, BLPOP, LPUSHX, BRPOPLPUSH
•EXISTS
•GET, GETSET, HGET, GETBIT
•GETRANGE, SETRANGE
•HLEN, HSTRLEN
•LLEN
•PFADD, PFCOUNT
•SISMEMBER
•APPEND
Things you didn't know
O(N), but can be considered O(1) for short substrings
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 45
Big O Notation: O(1) commands
•ZSCORE
•LPOP, BLPOP, LPUSHX, BRPOPLPUSH
•EXISTS
•GET, GETSET, HGET, GETBIT
•GETRANGE, SETRANGE
•HLEN, HSTRLEN
•LLEN
•PFADD, PFCOUNT
•SISMEMBER
•APPEND
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 46
Things you didn't know
Expiration and eviction
Keys are expired in two ways:
•Active
•Passive 1
3 2
Test 20 random keys from the set of keys with an associated expire
10 times per second
Delete all the keys found expiredIf more than 25% of keys were expired, start again from step 1
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
PublisherSubscriber A Subscriber B
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
PublisherSubscriber A Subscriber B
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1
PublisherSubscriber A Subscriber B
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1
127.0.0.1:6379> subscribe bgwebsummit sofia
PublisherSubscriber A Subscriber B
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
PublisherSubscriber A Subscriber B
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
PublisherSubscriber A Subscriber B
127.0.0.1:6379> pubsub channels
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
PublisherSubscriber A Subscriber B
127.0.0.1:6379> pubsub channels
1) "sofia" 2) "bgwebsummit"
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 127.0.0.1:6379> publish bgwebsummit Hello
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
PublisherSubscriber A Subscriber B
127.0.0.1:6379> pubsub channels
1) "sofia" 2) "bgwebsummit"
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 127.0.0.1:6379> publish bgwebsummit Hello
(integer) 1
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
PublisherSubscriber A Subscriber B
127.0.0.1:6379> pubsub channels
1) "sofia" 2) "bgwebsummit"
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 127.0.0.1:6379> publish bgwebsummit Hello
(integer) 11) "message" 2) "bgwebsummit" 3) "Hello"
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
PublisherSubscriber A Subscriber B
127.0.0.1:6379> pubsub channels
1) "sofia" 2) "bgwebsummit"
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 127.0.0.1:6379> publish bgwebsummit Hello
(integer) 11) "message" 2) "bgwebsummit" 3) "Hello"
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
1) "message" 2) "bgwebsummit" 3) "Hello"
PublisherSubscriber A Subscriber B
127.0.0.1:6379> pubsub channels
1) "sofia" 2) "bgwebsummit"
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 127.0.0.1:6379> publish bgwebsummit Hello
(integer) 11) "message" 2) "bgwebsummit" 3) "Hello"
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
1) "message" 2) "bgwebsummit" 3) "Hello"
PublisherSubscriber A Subscriber B
127.0.0.1:6379> publish sofia Zdraveite
127.0.0.1:6379> pubsub channels
1) "sofia" 2) "bgwebsummit"
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 47
Pub Sub
127.0.0.1:6379> subscribe bgwebsummit
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 127.0.0.1:6379> publish bgwebsummit Hello
(integer) 11) "message" 2) "bgwebsummit" 3) "Hello"
127.0.0.1:6379> subscribe bgwebsummit sofia
Reading messages... (press Ctrl-C to quit) 1) "subscribe" 2) "bgwebsummit" 3) (integer) 1 1) "subscribe" 2) "sofia" 3) (integer) 2
1) "message" 2) "bgwebsummit" 3) "Hello"
PublisherSubscriber A Subscriber B
127.0.0.1:6379> publish sofia Zdraveite
127.0.0.1:6379> pubsub channels
1) "sofia" 2) "bgwebsummit"
1) "message" 2) "sofia" 3) "Zdraveite"
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 48
Things you didn't know
Pub Sub
•PSUBSCRIBE can be used to subscribe to all channels matching a pattern
•Once a Redis client enters the subscriber mode, it can no longer perform any operation other than subscribe to more channels or unsubscribe from the subscribed ones. An additional connection will be needed in order to publish messages.
•Publishing on db 10, will be heard by a subscriber on db 1
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 49
Persistence
Persistence options:
•RDB (Point-in-time snapshots)
•AOF (Append only file)
•Combination of AOF and RDB
•No persistence
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 50
Geo API
•Search a GeoSet for members within ‘radius’ distance from given coordinates
•Return the distance in unit (meters by default) between elem1 and elem2 in a GeoSet
Things you didn't know
•Storing and querying named pairs of longitude/latitude coordinates into Redis keys
•Sorted sets with GeoHashing
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 51
redis-cli
•redis-cli --bigkeys•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 51
redis-cli
•redis-cli --bigkeys•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
> redis-cli --bigkeys
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 51
redis-cli
•redis-cli --bigkeys•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
> redis-cli --bigkeys# Scanning the entire keyspace to find biggest keys as well as # average sizes per key type. You can use -i 0.1 to sleep 0.1 sec # per 100 SCAN commands (not usually needed).
[00.00%] Biggest string found so far 'foo1' with 10285 bytes [00.00%] Biggest zset found so far 'foo2' with 1 members [00.00%] Biggest string found so far 'foo4' with 100095 bytes [00.00%] Biggest set found so far 'foo5' with 1 members [12.66%] Biggest string found so far 'foo5' with 13505361 bytes [25.32%] Biggest list found so far 'foo6' with 10001 items -------- summary ------- Sampled 79 keys in the keyspace! Total key length in bytes is 3243 (avg len 41.05)
Biggest string found 'foo5' has 13505361 bytes Biggest list found 'foo6' has 10001 items Biggest set found 'foo5' has 12 members Biggest zset found 'foo7' has 8 members
50 strings with 13946784 bytes (63.29% of keys, avg size 278935.68) 3 lists with 10007 items (03.80% of keys, avg size 3335.67) 3 sets with 3 members (03.80% of keys, avg size 1.00) 0 hashs with 0 fields (00.00% of keys, avg size 0.00) 23 zsets with 80 members (29.11% of keys, avg size 3.48)
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 52
redis-cli
•redis-cli --bigkeys
•redis-cli monitor•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 52
redis-cli
•redis-cli --bigkeys
•redis-cli monitor•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
> redis-cli monitor
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 52
redis-cli
•redis-cli --bigkeys
•redis-cli monitor•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
OK 1491054587.455533 [0 127.0.0.1:35532] "zadd" "foo" "1" "one" 1491054591.768172 [0 127.0.0.1:35532] "zadd" "foo" "2" "two" 1491054605.088297 [0 127.0.0.1:35532] "get" "bar" 1491054609.336546 [0 127.0.0.1:35532] "del" "bar" 1491054618.424296 [0 127.0.0.1:35532] "set" "foobar" "here I am" 1491054669.240038 [0 127.0.0.1:35532] "zcount" "foo" "-inf" "+inf"
> redis-cli monitor
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 53
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 53
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
> redis-cli --scan --pattern '*foo*'
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 53
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'•redis-cli --stat
•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
key-foo12 key-foo23 foo24 foobarbaz foobarbaz43 foo:bar:12
> redis-cli --scan --pattern '*foo*'
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 54
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 54
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
> redis-cli --stat
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 54
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat•redis-cli -x set foo < /etc/services
•redis-cli get bar > ~/bar.txt
------- data ------ -------------------------------------- load ----------------------------------- - child - keys mem clients blocked requests connections 506 1015.00K 1 0 24 (+0) 7 506 1015.00K 1 0 25 (+1) 7 506 3.40M 51 0 60461 (+60436) 57 506 3.40M 51 0 146425 (+85964) 107 507 3.40M 51 0 233844 (+87419) 157 507 3.40M 51 0 321715 (+87871) 207 508 3.40M 51 0 408642 (+86927) 257 508 3.40M 51 0 497038 (+88396) 257
> redis-cli --stat
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
> cat /etc/foo.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
> redis-cli 127.0.0.1:6379>
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
> redis-cli 127.0.0.1:6379> get foo
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 55
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt•redis-cli get bar > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
> redis-cli 127.0.0.1:6379>"A long long text\nIn multiple lines\n"
get foo
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 56
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
> redis-cli 127.0.0.1:6379>"A long long text\nIn multiple lines\n"
get foo
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 56
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
> redis-cli 127.0.0.1:6379>"A long long text\nIn multiple lines\n"
get foo
127.0.0.1:6379> set foo "A long long text"
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 56
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
> redis-cli 127.0.0.1:6379>"A long long text\nIn multiple lines\n"
get foo
127.0.0.1:6379> set foo "A long long text"
> cat /etc/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 56
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
A long long text In multiple lines
> cat /etc/foo.txt
> redis-cli -x set foo < /etc/foo.txtOK
> redis-cli 127.0.0.1:6379>"A long long text\nIn multiple lines\n"
get foo
127.0.0.1:6379> set foo "A long long text"
A long long text
> cat /etc/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 57
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 57
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
> cat /commands.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 57
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
set foo 100 incr foo append foo xxx get foo
> cat /commands.txt
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 57
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
set foo 100 incr foo append foo xxx get foo
> cat /commands.txt
> cat /commands.txt| redis-cli
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 57
redis-cli
•redis-cli --bigkeys
•redis-cli monitor
•redis-cli --scan --pattern '*foo*'
•redis-cli --stat
•redis-cli -x set foo < /etc/foo.txt
•redis-cli get foo > ~/bar.txt
set foo 100 incr foo append foo xxx get foo
> cat /commands.txt
OK (integer) 101 (integer) 6 "101xxx"
> cat /commands.txt| redis-cli
•cat /commands.txt| redis-cli
Things you didn't know
REDIS: Need for speed
@elena_kolevska
Elena Kolevska 58
Tips and best practices
Thank you!@elena_kolevska