Cassandra Summit 2012 - Building a Cassandra Based App From Scratch
-
Upload
patrick-mcfadin -
Category
Technology
-
view
6.278 -
download
0
description
Transcript of Cassandra Summit 2012 - Building a Cassandra Based App From Scratch
![Page 1: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/1.jpg)
Building a Cassandra based application from scratch
Patrick McFadinCassandra Summit 2012
#cassandra12
![Page 2: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/2.jpg)
This is me
• Chief Architect at Hobsons– Hobsons is an education services company. More
here: www.hobsons.com• Cassandra user since .7• Follow me here: @PatrickMcFadin
![Page 3: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/3.jpg)
Goals
• Take a new concept• What’s the data model?!?!• Some sample code• You get homework! (If you want)
![Page 4: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/4.jpg)
Here’s the plan
• Conceptualize a new application• Identify the entity tables• Identify query tables• Code. Rinse. Repeat.• Deploy• …• Profit!
* I’ll be using the term Tables which is equivalent to Column Families
![Page 5: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/5.jpg)
Start with a concept
*Cat drawing by goodrob13 on Flickr
Video Sharing Website
![Page 6: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/6.jpg)
Break down the features
• Post a video*• View a video• Add a comment• Rate a video• Tag a video
* Not talking about transcoding! Check out zencoder.com, it’s pretty sweet.
![Page 7: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/7.jpg)
Create Entity Tables
Basic storage unit
![Page 8: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/8.jpg)
Users
• Similar to a RDBMS table. Fairly fixed columns • Username is unique• Use secondary indexes on firstname and lastname for lookup• Adding columns with Cassandra is super easy
CREATE TABLE users ( username varchar PRIMARY KEY, firstname varchar, lastname varchar, password varchar);
![Page 9: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/9.jpg)
Users: The set codestatic void setUser(User user, Keyspace keyspace) { // Create a mutator that allows you to talk to casssandra Mutator<String> mutator = HFactory.createMutator(keyspace, stringSerializer);
try {
// Use the mutator to insert data into our table mutator.addInsertion(user.getUsername(), "users", HFactory.createStringColumn("firstname", user.getFirstname())); mutator.addInsertion(user.getUsername(), "users”, HFactory.createStringColumn("lastname", user.getLastname())); mutator.addInsertion(user.getUsername(), "users", HFactory.createStringColumn("password", user.getPassword()));
// Once the mutator is ready, execute on cassandra mutator.execute();
} catch (HectorException he) { he.printStackTrace(); }}
You can implement the get…
![Page 10: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/10.jpg)
Videos
• Use a UUID as a row key for uniqueness• Allows for same video names• Tags should be stored in some sort of delimited format• Index on username may not be the best plan
CREATE TABLE videos ( videoid uuid PRIMARY KEY, videoname varchar, username varchar, description varchar, tags varchar);
![Page 11: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/11.jpg)
Videos: The get codestatic Video getVideoByUUID(UUID videoId, Keyspace keyspace){ Video video = new Video(); //Create a slice query. We'll be getting specific column names SliceQuery<UUID, String, String> sliceQuery = HFactory.createSliceQuery(keyspace, uuidSerializer, stringSerializer, stringSerializer); sliceQuery.setColumnFamily("videos"); sliceQuery.setKey(videoId); sliceQuery.setColumnNames("videoname","username","description","tags");
// Execute the query and get the list of columns ColumnSlice<String,String> result = sliceQuery.execute().get(); // Get each column by name and add them to our video object video.setVideoName(result.getColumnByName("videoname").getValue()); video.setUsername(result.getColumnByName("username").getValue()); video.setDescription(result.getColumnByName("description").getValue()); video.setTags(result.getColumnByName("tags").getValue().split(",")); return video;}
You can implement the set…
![Page 12: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/12.jpg)
Comments
Time Order
• Videos have many comments• Use Composite Columns to store user and time• Value of each column is the text of the comment• Order is as inserted• Use getSlice() to pull some or all of the comments
CREATE TABLE comments ( videoid uuid PRIMARY KEY comment varchar);
![Page 13: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/13.jpg)
Rating a video
CREATE TABLE video_rating ( videoid uuid PRIMARY KEY, rating_counter counter, rating_total counter);*
* Only valid in CQL 3+
• Use counter for single call update• rating_count is how many ratings were given• rating_total is the sum of rating• Ex: rating_count = 5, rating_total = 23, avg rating = 23/5 = 4.6
![Page 14: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/14.jpg)
Video Event
CREATE TABLE video_event ( videoid_username varchar PRIMARY KEY, event varchar);
• Track viewing events• Combine Video ID and Username for a unique row• Stop time can be used to pick up where they left off• Great for usage analytics later
Time Order
![Page 15: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/15.jpg)
Create Query Tables
Indexes to support fast lookups
![Page 16: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/16.jpg)
Lookup Video By Username
CREATE TABLE username_video_index ( username varchar PRIMARY KEY, videoid_timestamp varchar);
• Username is unique• One column for each new video uploaded• Column slice for time span. From x to y• VideoId is added the same time a Video record is added
![Page 17: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/17.jpg)
Videos by Tag
CREATE TABLE tag_index ( tag varchar PRIMARY KEY, videoid varchar);
• Tag is unique regardless of video• Great for “List videos with X tag”• Tags have to be updated in Video and Tag at the same time• Index integrity is maintained in app logic
![Page 18: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/18.jpg)
Deployment strategies
• Measure your risk– Replication factor?– Multi-datacenter?– Cost?
• Performance– Today != tomorrow. Scale when needed– Have a expansion plan ready
![Page 19: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/19.jpg)
Wrap up
• Similar data model process to RDBMS… to start
• Query -> Index table• Don’t be afraid to write in multiple tables at
once• Bonus points: Hadoop and Solr!
![Page 20: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/20.jpg)
Go play!
• Go to: http://github.com/pmcfadin• Look for projects with cassandra12• Clone or fork my examples• Implement stubbed methods• Send me your solutions: [email protected]• Follow me for updates: @PatrickMcFadin
![Page 21: Cassandra Summit 2012 - Building a Cassandra Based App From Scratch](https://reader036.fdocuments.us/reader036/viewer/2022070303/54b6ca8e4a7959e5268b47f8/html5/thumbnails/21.jpg)
Thank You!
Connect with me at @PatrickMcFadinOr linkedIn
Conference tag #cassandra12