Mongo db at_customink

33
@ @nathenharvey Nathen Harvey

description

Presented at MongoDC in June, 2011. This talk describes how we use MongoDB at CustomInk, some challenges we faced introducing the technology, and some lessons learned.

Transcript of Mongo db at_customink

Page 1: Mongo db at_customink

@

@nathenharveyNathen Harvey

Page 2: Mongo db at_customink

Hello!

Nathen HarveyWeb Operations at [email protected]@nathenharvey

Organize MongoDC MeetupsSee you there next month!

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 3: Mongo db at_customink

Thanks for attending

In the other trackEliot Horowitz, CTO & Co-Founder, 10gen

Schema Design at Scale

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 4: Mongo db at_customink

CustomInk

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 5: Mongo db at_customink

CustomInk

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 6: Mongo db at_customink

CustomInk

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 7: Mongo db at_customink

CustomInk Customers

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 8: Mongo db at_customink

CustomInk Customers

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 9: Mongo db at_customink

CustomInk Customers

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 10: Mongo db at_customink

CustomInk Customers

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 11: Mongo db at_customink

Customer Reviews

Nathen HarveyNathen Harvey

Page 12: Mongo db at_customink

CustomInk Technology

Divide our technology into two classes:ECommerce – pre-checkout and shopping cartOperations Technology – post-check-out, fulfillment, etc.

Ruby on RailsWith some legacy Java

Approximately 30 different applications running in production

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 13: Mongo db at_customink

CustomInk Databases

Primarily OracleWas the “right” choice 11 years ago when the company was startedNo longer a good solution for us

MySQLPercona Server with XtraDB

MongoDB

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 14: Mongo db at_customink

MongoDB

ChampionsEasy to set-up and configure“Greenfield” projects with loose schema definitionsMakes development easier and developers happyLimited amount of time between releases

ChallengesNo prior experience with running productionLimited amount of time between releasesUnproven solution

Nathen HarveyNathen Harvey

Page 15: Mongo db at_customink

Problem

Complex Production EnvironmentsOrchestratedServices

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 16: Mongo db at_customink

Why Mongo?

Introduces company to MongoDBEasily to modify document structureLightweight developmentEasy to install and deployCapped Collections

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 17: Mongo db at_customink

Central Logger

Subclasses Buffered LoggerIt’s a gem loaded into Rails applications as a RailtieCaptures useful metadataUse the MongoDB Ruby Driverhttp://github.com/customink/central_logger

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 18: Mongo db at_customink

Central Logger

Page 19: Mongo db at_customink

Record Structure

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 20: Mongo db at_customink

Log Viewer

Page 21: Mongo db at_customink

Product Catalog

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 22: Mongo db at_customink

Why Mongo?

Frequently changing requirementsGridFSWide variety of products with varied characteristics

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 23: Mongo db at_customink

Case for Document-based Schema

@nathenharveyNathen Harvey

Page 24: Mongo db at_customink

Application Architecture

Rails application for managing the product catalogWrites to MongoDB as it’s databaseUtilizes GridFS for storing filesUses Mongoid as the ODM

@nathenharveyNathen Harvey

Page 25: Mongo db at_customink

Implementation Flow

Oracle MongoDB

ECommerceApplication

CatalogApplication

@nathenharveyNathen Harvey

Page 26: Mongo db at_customink

Next Steps

Continue to adopt MongoDB in the Operations Technology applicationsBegin using MongoDB with ECommerce applications

Reporting artwork usageCapturing JavaScript logs

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 27: Mongo db at_customink

Automated Deployment

Chef is an open source systems integration framework built to bring the benefits of configuration management to your entire infrastructure.We use Chef to manage our infrastructure and deploy MongoDB

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 28: Mongo db at_customink

MongoDB Cookbook

Use Vagrant and VirtualBox as the development environment for our CookbooksCookbook shared on Github and, soon, OpsCode Community Site.

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 29: Mongo db at_customink

Monitoring Production

http://github.com/mzupan/nagios-plugin-mongodb

Page 30: Mongo db at_customink

Monitoring Production

Monit

Page 31: Mongo db at_customink

Lessons Learned

Capped CollectionsTesting replica set failoverSchema free != design freeJSON in the DB, JQuery in the browser == happy developersmongo_import could not handle carriage returns in CSV filesUnserializable data typesData transformations on a large dataset still painfulUnless you’re sharding, map reduce probably slower and more complex than what you’re used to

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 32: Mongo db at_customink

MongoDC Meetup

Join fellow MongoDB enthusiasts for a monthly meetupMeetings are held at CustomInk in McLean, VAhttp://www.meetup.com/Washington-DC-MongoDB-Users-Group/

@nathenharveyNathen Harvey @nathenharveyNathen Harvey

Page 33: Mongo db at_customink

Thank You!

Nathen HarveyWeb Operations at [email protected]@nathenharvey

Organize MongoDC MeetupsSee you there next month!

@nathenharveyNathen Harvey @nathenharveyNathen Harvey