Redis
-
Upload
ivayr-farah-netto -
Category
Software
-
view
475 -
download
2
description
Transcript of Redis
![Page 1: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/1.jpg)
redis love at 8tracks
@nettofarah
![Page 3: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/3.jpg)
8tracks
![Page 4: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/4.jpg)
8tracks
![Page 5: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/5.jpg)
![Page 6: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/6.jpg)
![Page 7: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/7.jpg)
![Page 8: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/8.jpg)
![Page 9: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/9.jpg)
stats
• users: 11,351,961!
• public playlists: 1,545,586 (3,5 mi total)!
• uploaded tracks: 28,185,644
![Page 10: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/10.jpg)
not a sysadmin talk
![Page 11: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/11.jpg)
getting sudo
http://devopsreactions.tumblr.com/post/83704083404/getting-sudo
![Page 12: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/12.jpg)
the right tool for the job
http://peopletakingpictureswithipads.tumblr.com/
![Page 13: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/13.jpg)
RIGHT TOOL
http://peopletakingpictureswithipads.tumblr.com/
![Page 14: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/14.jpg)
FOR THE JOB!
http://peopletakingpictureswithipads.tumblr.com/
![Page 15: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/15.jpg)
engineering effort machine effort
![Page 16: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/16.jpg)
engineering effort machine effort
![Page 17: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/17.jpg)
engineering effort machine effort
![Page 18: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/18.jpg)
engineering effort machine effort
![Page 19: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/19.jpg)
When development tries to blame operations for the outage
http://devopsreactions.tumblr.com/post/84309007419/when-development-tries-to-blame-operations-for-the
![Page 20: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/20.jpg)
R.T.F.J. = engineer <3 + machine <3
![Page 21: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/21.jpg)
![Page 22: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/22.jpg)
key-value store
• open source
• in memory (most of the times)
• data-structure server
![Page 23: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/23.jpg)
data types• strings
• lists
• sets
• sorted sets
• hashes
![Page 24: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/24.jpg)
just like that data structures class
![Page 25: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/25.jpg)
except it is COOL!
![Page 26: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/26.jpg)
you can turn this
![Page 27: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/27.jpg)
into this
next next next next
head
tail
![Page 28: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/28.jpg)
or this
next next next
![Page 29: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/29.jpg)
things to keep in mind
• all in memory (but not restricted to)
• no joins or iterating over data (but not restricted to)
• you should be able to rebuild data
![Page 30: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/30.jpg)
why?
• sometimes basic data structures is all you need
• some problems are hard to fit in a relational way
![Page 31: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/31.jpg)
redis @ 8tracks = <3sidekiq
feature toggles
caching
player (moved to couchdb, then to mysql)
autocomplete (moved to SOLR, then to ElasticSearch)
tag browsing!
news feed
listening sessions
rate limiter
![Page 32: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/32.jpg)
awesome documentation
![Page 33: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/33.jpg)
![Page 34: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/34.jpg)
sidekiq
![Page 35: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/35.jpg)
![Page 36: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/36.jpg)
what’s indie?tag
artistplaylist
![Page 37: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/37.jpg)
1 SELECT name !! ! FROM tags !! ! WHERE name !! ! LIKE 'ind%' LIMIT 10;!!2 SELECT name !! ! FROM mixes !! ! WHERE name !! ! LIKE 'ind%' LIMIT 10;!!3 SELECT login !! ! FROM users !! ! WHERE login !! ! LIKE 'ind%' LIMIT 10;!!4 SELECT name !! FROM artists !! WHERE name !! LIKE 'ind%' LIMIT 10;!
![Page 38: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/38.jpg)
you could do
1 (SELECT name FROM tags WHERE name LIKE 'ind%' LIMIT 10)!2 UNION ALL!3 (SELECT name FROM mixes WHERE name LIKE 'ind%' LIMIT 10);!
![Page 39: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/39.jpg)
the problems
• cross-type search
• I don’t want 10 of each, I want the 10 most relevant ones
• can’t select different column counts
![Page 40: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/40.jpg)
not a fair comparison
people don’t use sql databases for search. they use SOLR (or elasticsearch)
![Page 41: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/41.jpg)
sometimes basic data structures!
is all you need
![Page 42: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/42.jpg)
sorted sets
• score
• add, remove, or update log(n)
• elements in order, fast existence test, fast access to elements in the middle
![Page 43: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/43.jpg)
breaking it down
indie =>
probably irrelevant
relevant
i !! ! ! ! ! ! ! in !!
! ! ! ! ! ! ! ind!! ! ! ! ! ! ! indi!! ! ! ! ! ! ! indie
![Page 44: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/44.jpg)
our collection
The indie summer Indiana Industrial
![Page 45: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/45.jpg)
one zset per n-gram
ind [ ][ ][ ]
indi
indie
![Page 46: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/46.jpg)
which is the same asZADD ind, <score>,
ZADD indi, <score>,
ZADD indie, <score>,
![Page 47: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/47.jpg)
searching
1 ind = $r.zevrange('ind', 0, 10)!!
2 indie = $r.zevrange('indie', 0, 10)!
![Page 48: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/48.jpg)
tag browsing
![Page 49: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/49.jpg)
how?
![Page 50: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/50.jpg)
tags mixes
taggings
![Page 51: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/51.jpg)
a single tag1 SELECT m.* FROM mixes m!2 INNER JOIN taggings tg!3 ON tg.mix_id = m.id!4 WHERE tg.tag_id = 28!5 LIMIT 10;!
chill
![Page 52: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/52.jpg)
two tags1 SELECT m.* FROM mixes m!2 INNER JOIN taggings tg!3 ON tg.mix_id = m.id!4 INNER JOIN taggings tg2!5 ON tg2.mix_id = m.id!6 WHERE tg.tag_id = 28!7 AND tg2.tag_id = 12!8 LIMIT 10;! chill
acoustic
![Page 53: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/53.jpg)
adding sort1 SELECT m.* FROM mixes m!2 INNER JOIN taggings tg!3 ON tg.mix_id = m.id!4 INNER JOIN taggings tg2!5 ON tg2.mix_id = m.id!6 WHERE tg.tag_id = 28!7 AND tg2.tag_id = 12!8 ORDER BY m.first_published_at!9 LIMIT 10;!
![Page 54: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/54.jpg)
![Page 55: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/55.jpg)
ZINTERSTORE
http://redis.io/commands/zinterstore
![Page 56: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/56.jpg)
zinterstore
Time complexity: O(N*K)+O(M*log(M))
intersects two sets, then store them in a new different set
we’ll get to this
![Page 57: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/57.jpg)
chill
![Page 58: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/58.jpg)
acoustic
![Page 59: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/59.jpg)
chill acoustic
+
![Page 60: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/60.jpg)
chill acoustic
1 sets = ['chill', 'acoustic']!2 cache_key = 'chill:acoustic:cache'!3 $r.zinterstore(cache_key, sets, :weights => [1, 1])!
![Page 61: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/61.jpg)
chill
chill acoustic
popular
1 sets = ['popular', 'chill', 'acoustic']!2 cache_key = 'popular:chill:acoustic:cache'!3 $r.zinterstore(cache_key, sets, :weights => [1, 0, 0])!
![Page 62: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/62.jpg)
back to the BIG O thing
O(N*K)+O(M*log(M))
- N: smallest input - K: number of sorted sets - M: number of elements of the resulting set
![Page 63: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/63.jpg)
![Page 64: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/64.jpg)
cool stuff worth checking out
• pipelining
• lua scripting
• replication + persistence
![Page 65: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/65.jpg)
lessons
• it consumes a LOT of memory (duuuh)
• sentinel for failure recovery
• DO NOT USE “keys” in production
• duplication is tricky
![Page 66: Redis](https://reader035.fdocuments.us/reader035/viewer/2022070303/54b6bb414a79592f7a8b45b0/html5/thumbnails/66.jpg)
gracias!
thank you!
obrigado!