© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Redisfor Java Software Engineers
Copyrights © Moshe [email protected]
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Redisfor Java Software Engineers
Moshe KaplanScale Hacker
http://top-performance.blogspot.comhttp://blogs.microsoft.co.il/vprnd
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
About Me: It’s all About
3
Scale
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
NOSQL MARKETIntroduction
4
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
When Should I Choose NoSQL?
• Eventually Consistent
• Document Store
• Key Value
5
http://guyharrison.squarespace.com/blog/tag/nosql
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Key Value Store
• insert
• get
• multiget
• remove
• truncate
6
<Key, Value>http://wiki.apache.org/cassandra/API
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Redis
• Very simple protocol (SMTP like)
• Amazing Performance (60Kqps ops on 1 CPU machine)
• Persistency to disk
• Very little security
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
HELLO. MY NAME IS REDISIntroduction
8
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
9
#10 Most Popular DB Engine
http://db-engines.com/en/ranking
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Who is Using Redis?
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Who is Behind Redis
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Installation: Give Yourself 5min• sudo apt-get install -y build-essential tcl8.5
• wget http://download.redis.io/releases/redis-stable.tar.gz
• tar xzf redis-stable.tar.gz
• cd redis-stable
• make
• sudo make install
• cd utils
• sudo ./install_server.sh
• sudo service redis_6379 start
• redis-cli
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Redis and Data Models
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Redis Data ModelsData Types What?
Binary-safe strings Strings
Lists Strings List
Sets Unique unsorted strings
Sorted Sets Score field
Hashes O(1)
Bit Arrays/Bitmaps Bit operations on set
HyperLogLogs Estimate set cardinality
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Key Design
• Keep keys short
• But not too short (readable, existing overhead)• Use Convention:
• comment:1234:reply.to
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Setssimple storage
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Basic Sets> SET counter 100
OK
> GET counter
(integer) 100
> inc counter
(integer) 101
> incby counter 50
(integer) 151
DECR DECRBY
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Multi Set
> MSET counter 100 cnt2 102 cnt3 103
OK
> MGET counter cnt2 cnt 3
(integer) 100
(integer) 102
(integer) 103
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Existence
> exists counter
(integer) 1
> type counter
integer
> del counter
(integer) 1
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
TTL Keys> expire counter 5
(integer) 1
> set cnt2 ex 5
OK
> ttl cnt
5
> persist cnt2
OK
Will Expire in 5 secondsWill Expire in 5 seconds
Will Expire in 5 secondsWill Expire in 5 seconds
How much time was left?How much time was left?
Remove TTLRemove TTL
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Linked Listsa basis for queues
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Existence
> Adding/Removing for end/start: O(1)
> Getting other items: O(N)
LPUSH/RPUSH => Add items (Head/Tail)
LPOP/RPOP => Take items (Head/Tail)
LRANGE/RRANGE => Get items (Head/Tail)
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Blocking Operations
Best for Producer/Consumer (Q)
If list is empty,calls are blocked
> BRPOP <LIST> <# of Seconds>
> BLPOP <LIST> <# of Seconds>
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Capped Lists
Define how many items we want to have
> LPUSH mylist <some element>
> LTRIM mylist 0 999
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Hashesa column like data store
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Existence> hmset user:1000 username antirez birthyear 1977 OK > hget user:1000 username "antirez" > hget user:1000 birthyear"1977" > hgetall user:1000 1) "username" 2) "antirez" 3) "birthyear" 4) "1977"
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Setsa basis for social and permissions
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Set Ops> sismember myset 3
(integer) 1
> sadd news:1000:tags 1 2 5 77
(integer) 4
> sadd tag:1:news 1000
(integer) 1
> sinter news:1000:tags news:2000:tags
,,,,
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Ordered Sets (Z)A leader board solution
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Set Ops
> zadd hackers 1940 "Alan Kay" (integer) 1
> zadd hackers 1957 "Sophie Wilson" (integer 1)
> zrange hackers 0 -1
> zremrangebyscore hackers 1940 1960
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
BitMapsEfficient Configuration and
Permissions
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Set Ops
> setbit key 10 1
(integer) 1
> getbit key 10
(integer) 1
> bitcount key
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Advanced Operations
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Scan: You can iterate all keys!
Scan
SScan
HScan
ZScan
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Lua Scripting: You have script engine
> eval "return redis.call('set','foo','bar')" 0
OK
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Pub/Sub
Very nice tool, but not as good as Kafka and RabbitMQ
SUBSCRIBE first second
PUBLISH second Hello
UNSUBSCRIBE
PSUBSCRIBE news.*
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
SortingSORT mylist
SORT mylist DESC
SORT mylist ALPHA
SORT mylist BY weight_*
SORT mylist BY nosort
SORT mylist BY weight_* GET object_*
SORT mylist BY weight_* GET object_* GET #
SORT mylist BY weight_* STORE resultkey
SORT mylist BY weight_*->fieldname GET object_*->fieldname
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Transactions1. Atomic
2. Blocking all other operations
3. All or Nothing
MULTI: Start transaction
EXEC: Commit
DISCARD: “Roll back” (only before EXEC, nothing done)
WATCH: Don’t use in multi client (race conditions)
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Transaction by Example> MULTI
OK
> INCR foo
QUEUED
> INCR bar
QUEUED
> EXEC
1) (integer) 1
2) (integer) 1
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Java Programming
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Many Options
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Securing Redis
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Authentication
• By default: None
• Enable: requirepass in the config file
• Authenticate: AUTH <PASSWORD>
• Simple, Clear text
• No measures to prevent brute force
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Communication Encryption
• SSL Communication encryption using stunnel/spiped
• Webdis web front layer
• Encrypt the data in the app level
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Persistency
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
The Good, bad and Evil• RDB
• Point in time snapshots
• Backup before critical operation
• AOF
• On every write to log (or once a seocnd)
• Log rerun at startup
• None
• In memory only solution
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
CACHING
47
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Don’t Use Caching
48
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
If You Have To
49
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Invalidation Can Be Nightmare
50
http://luauf.com/2009/06/08/%C2%BFque-es-memcached/
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Geo Load Balancing Can be Worse
51
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Finally, Recovery May Not Be Better
52
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
KISS
53
http://marriagelifeministries.org/?p=962
© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers© All rights reserved: Moshe Kaplan© All rights reserved: Moshe Kaplan Redis for Java Software Engineers
Thank You !Moshe Kaplan
Top Related