Riak Intro
-
Upload
jon-meredith -
Category
Technology
-
view
1.330 -
download
5
description
Transcript of Riak Intro
![Page 1: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/1.jpg)
Jon Meredith Basho Technologies
Tuesday, December 7, 2010
![Page 2: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/2.jpg)
WHAT IS IT?
Tuesday, December 7, 2010
![Page 3: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/3.jpg)
basho
Riak is a scalable, highly-available, networked
key/value store.
Tuesday, December 7, 2010
![Page 4: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/4.jpg)
basho
Inspired by Amazon’s Dynamo...with some enhancements.
Open Source
Tuesday, December 7, 2010
![Page 5: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/5.jpg)
Riak is a scalable, highly-available, networked
key/value store.
Tuesday, December 7, 2010
![Page 6: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/6.jpg)
basho
•Stores and retrieves data against a key.
•Encode your data how you like it.
•Keys are organized into buckets.
Tuesday, December 7, 2010
![Page 7: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/7.jpg)
basho
BASIC OPERATIONS
getput
delete
Tuesday, December 7, 2010
![Page 8: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/8.jpg)
basho
•Riak also stores metadata against keys
•Content Type, Charset, Encoding, ...
•Custom application metadata
Tuesday, December 7, 2010
![Page 9: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/9.jpg)
basho
Riak is a scalable, highly-available, networked
key/value store.
Tuesday, December 7, 2010
![Page 10: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/10.jpg)
basho
bucket: pivotalkey: greetingvalue: “<h1>Hi Pivotal<h1>” metadata: content-type=text/html charset=utf8
Tuesday, December 7, 2010
![Page 11: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/11.jpg)
basho
> PUT /riak/pivotal/greeting HTTP/1.1> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3> Host: 127.0.0.1:8098> Accept: */*> content-type: text/html> charset: utf8> Content-Length: 19> > <h1>Hi Pivotal<h1>
< HTTP/1.1 200 OK< X-Riak-Vclock: a85hYGBgzWDKBVIsTNPiOTOYEhnzWBm+5uUf5YMIszUnMdcLv4dKsOTDJVhYPx1JwiLMeH/bU6iwWj6yMUwtt6yQJbIA< Vary: Accept-Encoding< Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)< Link: </riak/pivotal>; rel="up"< Date: Mon, 24 May 2010 21:44:06 GMT< Content-Type: text/html< Content-Length: 19<
Tuesday, December 7, 2010
![Page 12: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/12.jpg)
basho
> GET /riak/pivotal/greeting HTTP/1.1> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3> Host: localhost:8098> Accept: */*> < HTTP/1.1 200 OK< X-Riak-Vclock: a85hYGBgymDKBVIsrDk3LTOYEhnzWBmUmtOP8kGE2ZqTmNf8Xg6VCAJJZAEA< Vary: Accept-Encoding< Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)< Link: </riak/pivotal>; rel="up"< Last-Modified: Tue, 18 May 2010 21:32:02 GMT< ETag: 1VW7QmCuyofVCuQbqBNVYh< Date: Tue, 18 May 2010 21:32:06 GMT< Content-Type: text/html; charset=utf8< Content-Length: 19< * Connection #0 to host localhost left intact* Closing connection #0<h1>Hi Pivotal</h1>
Tuesday, December 7, 2010
![Page 13: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/13.jpg)
basho
• Protocol Buffers Client
•Messages encoded using Protocol Buffers
• 3-10x speedups depending on data
•Native Erlang client
OTHER INTERFACES TOO!
Tuesday, December 7, 2010
![Page 14: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/14.jpg)
Riak is a scalable, highly-available, networked
key/value store.
Tuesday, December 7, 2010
![Page 15: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/15.jpg)
basho
Riak was designed to scale horizontally.
Tuesday, December 7, 2010
![Page 16: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/16.jpg)
Tuesday, December 7, 2010
![Page 17: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/17.jpg)
Tuesday, December 7, 2010
![Page 18: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/18.jpg)
Tuesday, December 7, 2010
![Page 19: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/19.jpg)
Tuesday, December 7, 2010
![Page 20: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/20.jpg)
Tuesday, December 7, 2010
![Page 21: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/21.jpg)
Riak is a scalable, highly-available, networked
key/value store.
Tuesday, December 7, 2010
![Page 22: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/22.jpg)
Tuesday, December 7, 2010
![Page 23: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/23.jpg)
Tuesday, December 7, 2010
![Page 24: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/24.jpg)
Tuesday, December 7, 2010
![Page 25: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/25.jpg)
#Replicas - N#Get response - R#Put responses - W
R+W > N for quorum
Tuesday, December 7, 2010
![Page 26: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/26.jpg)
Tuesday, December 7, 2010
![Page 27: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/27.jpg)
Tuesday, December 7, 2010
![Page 28: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/28.jpg)
Tuesday, December 7, 2010
![Page 29: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/29.jpg)
Tuesday, December 7, 2010
![Page 30: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/30.jpg)
Tuesday, December 7, 2010
![Page 31: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/31.jpg)
basho
WILL THE REAL OBJECT PLEASE STEP FORWARD?
• Riak always accepts writes - even during network partitions and other badness
•Objects are versioned to automatically detect ancestors/siblings
• Conflicts are detected on reads
• Eventually consistent
Tuesday, December 7, 2010
![Page 32: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/32.jpg)
That sounds great... but can I do anything without a key?
Tuesday, December 7, 2010
![Page 33: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/33.jpg)
basho
Query datasets withMap / Reduce
Tuesday, December 7, 2010
![Page 34: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/34.jpg)
bashoTuesday, December 7, 2010
![Page 35: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/35.jpg)
basho
function(v) { var m = v.values[0].data. match('\\w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r;}
Tuesday, December 7, 2010
![Page 36: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/36.jpg)
basho
map()map()map()map()map()map()map()map()function(v) { var m = v.values[0].data. match('\\w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r;}
Tuesday, December 7, 2010
![Page 37: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/37.jpg)
basho
map()
map()
map()
map()
map()
map()
map()
map()
Tuesday, December 7, 2010
![Page 38: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/38.jpg)
basho
R8
R1
R2
R3
R4
R5
R6
R7
Tuesday, December 7, 2010
![Page 39: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/39.jpg)
basho
R8R1R2R3R4R5R6R7reduce()
function(v) { var r = {}; for (var i in v) { for(var w in v[i]) { if (w in r) r[w] += v[i][w]; else r[w] = v[i][w]; } } return [r]; }
Tuesday, December 7, 2010
![Page 40: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/40.jpg)
basho
Store relationshipswith links
Tuesday, December 7, 2010
![Page 41: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/41.jpg)
basho
child
child
child
people/alicepeople/bob
people/dean
people/claire
Tuesday, December 7, 2010
![Page 42: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/42.jpg)
basho
child
father
child
mother
childmother
people/alicepeople/bob
people/dean
people/claire
Tuesday, December 7, 2010
![Page 43: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/43.jpg)
basho
child
father
daughter child
motherdaughter
childmother
son
grandchild
grandchild
people/alicepeople/bob
people/dean
people/claire
Tuesday, December 7, 2010
![Page 44: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/44.jpg)
basho
people/alice
child
father
daughter child
motherdaughter
childmother
son
people/bob
GET /riak/person/bob/people,child,_key link
bucketlinktag
people/dean
people/claire
Tuesday, December 7, 2010
![Page 45: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/45.jpg)
basho
people/claire
child
father
daughter child
motherdaughter
childmother
son
GET /riak/person/bob/people,child,_/_,child,_key link1
bucketlink1tag
link2bucket
link2tag
people/alicepeople/bob
people/dean
Tuesday, December 7, 2010
![Page 46: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/46.jpg)
basho
people/claire
people/alicepeople/bob
people/dean
child
father
daughter child
motherdaughter
childmother
son
GET /riak/person/bob/people,child,1/people,child,_
Tuesday, December 7, 2010
![Page 47: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/47.jpg)
COMPARED TO RDBMS
•No schema
•No transactions
•No joins (but we do have map/reduce)
•No secondary indices (but we do have links!)
•No schema migrations - version your objects
•No replication topologies - just add nodes to scale
Tuesday, December 7, 2010
![Page 48: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/48.jpg)
basho
CLIENT LIBRARIES
JavascriptJava
.NET linq Twisted Python
Ruby
Tuesday, December 7, 2010
![Page 49: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/49.jpg)
basho
RIAK IN PRODUCTION
Tuesday, December 7, 2010
![Page 50: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/50.jpg)
basho
MOCHI MEDIA
•Multiple clusters in production
• Sessions (web and Flash) - 5 nodes, ~20MM keys - highest traffic
•User data storage - 3 nodes, bucket-per-game
•URL Shortener - 2 nodes, bucket-per-game
• Social graph - 3 nodes, OAuth credentials, relationships
Text and link here but not here
Title on white
Developer Center Advertiser CenterPublisher Center
www.mochimedia.com
Tuesday, December 7, 2010
![Page 51: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/51.jpg)
basho
MOZILLA TEST PILOT
•Data collector for anonymized Firefox usage statistics
•High peak throughput - pure writes
• Large MapReduce queries for analytics
http://bit.ly/aC0i4f
“Riak meets or exceeds all the write performance requirements...it was nice to
see predictable performance and stability under load.”
Tuesday, December 7, 2010
![Page 52: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/52.jpg)
basho
WEGEO
•Mobile Productivity / Collaboration App- Task lists, Shopping lists, Calendar
• Event and Check-in logs
•Monthly reports archive
• Third-party coupon and deal aggregation
“Scalability I could understand, on a platform I could understand.”
Tuesday, December 7, 2010
![Page 53: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/53.jpg)
GETTING RIAKPrebuild Packages
http://downloads.basho.com
https://github.com/basho
The Bleeding Edge
http://bitbucket.org/basho
Tuesday, December 7, 2010
![Page 54: Riak Intro](https://reader034.fdocuments.us/reader034/viewer/2022052304/559784161a28ab227b8b45fe/html5/thumbnails/54.jpg)
basho
http://wiki.basho.comfollow twitter.com/basho/team
[email protected]#riak on Freenode
Tuesday, December 7, 2010