Post on 09-May-2015
description
Application Development SeriesBack to BasicsInteracting with the database
Daniel Roberts@dmroberts
#MongoDBBasics
2
• Recap from last session
• MongoDB Inserts & Queries– ObjectId– Returning documents – cursors– Projections
• MongoDB Update operators– Fixed Buckets– Pre Aggregated Reports
• Write Concern– Durability vs Performance trade off
Agenda
3
• Virtual Genius Bar
– Use the chat to post questions
– EMEA Solution Architecture / Support team are on hand
– Make use of them during the sessions!!!
Q & A
Recap from last time….
5
• Looked at the application architecture– JSON / RESTful– Python based
Architecture
Client-sideJSON
(eg AngularJS) (BSON)
Pymongo driver
Python web app
HTTP(S) REST
6
• Schema design– Modeled
• Articles• Comments• Interactions• Users
Schema and Architecture
7
Modeling Articles
• Posting articles• insert
• Get List of articles• Return Cursor
• Get individual article
{ '_id' : ObjectId(...),
'text': 'Article content…',
'date' : ISODate(...),
'title' : ’Intro to MongoDB',
'author' : 'Dan Roberts',
'tags' : ['mongodb',
'database',
'nosql’]
}
Articles collection
METHODSdef get_article(article_id)def get_articles():def create_article():
8
Modeling Comments
• Storing comments
• Quickly retrieve most recent comments
• Add new comments to document
• ‘Bucketing’
{‘_id’ : ObjectId(..),
‘article_id’ : ObjectId(..), ‘page’ : 1, ‘count’ : 42 ‘comments’ : [
{ ‘text’ : ‘A great article, helped me understand schema design’, ‘date’ : ISODate(..), ‘author’ : ‘johnsmith’ }, …}
Comments collection
METHODSdef add_comment(article_id):def get_comments(article_id):
9
Modeling Interactions
• Used for reporting on articles
• Create “pre-aggregated” reports
{‘_id’ : ObjectId(..),
‘article_id’ : ObjectId(..), ‘section’ : ‘schema’,
‘date’ : ISODate(..),‘daily’: { ‘views’ : 45,
‘comments’ : 150 } ‘hours’ : { 0 : { ‘views’ : 10 }, 1 : { ‘views’ : 2 }, … 23 : { ‘views’ : 14,
‘comments’ : 10 } }}
Interactions collection
METHODS def add_interaction(article_id, type):
Inserting / Querying
11
>db.articles.insert({'text': 'Article content…’,
'date' : ISODate(...), 'title' : ’Intro to MongoDB’, 'author' : 'Dan Roberts’, 'tags' : [ 'mongodb',
'database', 'nosql’
]});
• Driver generates ObjectId() for _id – if not specified– 12 bytes - 4-byte epoch, 3-byte machine id, a 2-byte process id, and a 3-byte
counter.
Inserting documents
12
$gt, $gte, $in, $lt, $lte, $ne, $nin
• Use to query documents
• Logical: $or, $and, $not, $nor Element: $exists, $type
• Evaluation: $mod, $regex, $where Geospatial: $geoWithin, $geoIntersects, $near, $nearSphere
Comparison Operators
db.articles.find( { 'title' : ’Intro to MongoDB’ } )
db.articles.find( { ’date' : { ‘$lt’ : {ISODate("2014-02-
19T00:00:00.000Z") }} )
db.articles.find( { ‘tags’ : { ‘$in’ : [‘nosql’, ‘database’] } } );
13
• Find returns a cursor– Use to iterate over the results– cursor has many methods
Cursors
>var cursor = db.articles.find ( { ’author' : ’Dan Roberts’ } )>cursor.hasNext()true>cursor.next(){ '_id' : ObjectId(...),
'text': 'Article content…’, 'date' : ISODate(...), 'title' : ’Intro to MongoDB’, 'author' : 'Dan Roberts’, 'tags' : [ 'mongodb', 'database’, 'nosql’ ]
}
14
• Return only the attributes needed– Boolean 0 or 1 syntax select attributes– Improved efficiency
Projections
>var cursor = db.articles.find( { ’author' : ’Dan Roberts’ } , {‘_id’:0, ‘title’:1})>cursor.hasNext()true>cursor.next(){ "title" : "Intro to MongoDB" }
Updates
16
$each, $slice, $sort, $inc, $push$inc, $rename, $setOnInsert, $set, $unset, $max, $min
$, $addToSet, $pop, $pullAll, $pull, $pushAll, $push
$each, $slice, $sort
Update Operators
>db.articles.update(
{ '_id' : ObjectId(...)},
{ '$push' : {'comments' : ‘Great
article!’ }}
)
{ 'text': 'Article content…’ 'date' : ISODate(...),
'title' : ’Intro to MongoDB’,
'author' : 'Dan Roberts’,'tags' : ['mongodb',
'database’,'nosql’ ],’comments' :
[‘Great article!’ ]
}
17
Push to a fixed size array with…
$push, $each, $slice
Update Operators
>db.articles.update(
{ '_id' : ObjectId(...)},
{ '$push' : {'comments' :
{
'$each' : [‘Excellent’], '$slice' : -3}}, })
{ 'text': 'Article content…’ 'date' : ISODate(...),
'title' : ’Intro to MongoDB’,
'author' : 'Dan Roberts’,'tags' : ['mongodb',
'database’,'nosql’ ],’comments' :
[‘Great article!’, ‘More please’, ‘Excellent’ ]
}
18
• Push 10 comments to a document (bucket).
• Automatically create a new document.
• Use {upsert: true} instead of insert.
Update Operators - Bucketing
>db.comments.update(
{‘c’: {‘$lt’:10}}, {
‘$inc’ : {c:1}, '$push' : {
'comments' : ‘Excellent’}},{ upsert : true }
)
{‘_id’ : ObjectId( … )‘c’ : 3,’comments' :
[‘Great article!’,
‘More please’, ‘Excellent’ ]
}
19
Analytics – Pre-Aggregated reports
• Used for reporting on articles
• Create “pre-aggregated” reports
{‘_id’ : ObjectId(..),
‘article_id’ : ObjectId(..), ‘section’ : ‘schema’,
‘date’ : ISODate(..),‘daily’: { ‘views’ : 45,
‘comments’ : 150 } ‘hours’ : { 0 : { ‘views’ : 10 }, 1 : { ‘views’ : 2 }, … 23 : { ‘views’ : 14,
‘comments’ : 10 } }}
Interactions collections
METHOD def add_interaction(article_id, type):
20
• Use $inc to increment multiple counters.
• Single Atomic operation.
• Increment daily and hourly counters.
Incrementing Counters
>db.interactions.update({‘article_id’ : ObjectId(..)}, {
‘$inc’ : {
‘daily.views’:1,
‘daily.comments’:1
‘hours.8.views’:1
‘hours.8.comments’:1 })
{‘_id’ : ObjectId(..),
‘article_id’ : ObjectId(..), ‘section’ : ‘schema’,
‘date’ : ISODate(..),‘daily’: { ‘views’ : 45,
‘comments’ : 150 } ‘hours’ : { 0 : { ‘views’ : 10 }, 1 : { ‘views’ : 2 }, … 23 : { ‘views’ : 14,
‘comments’ : 10 } }}
21
• Increment new counters
Incrementing Counters 2
>db.interactions.update({‘article_id’ : ObjectId(..)}, {
‘$inc’ : {
‘daily.views’:1,
‘daily.comments’:1,
‘hours.8.views’:1,
‘hours.8.comments’:1,
‘referrers.bing’ : 1})
{‘_id’ : ObjectId(..),
‘article_id’ : ObjectId(..), ‘section’ : ‘schema’,
‘date’ : ISODate(..),‘daily’: { ‘views’ : 45,
‘comments’ : 150 } ‘hours’ : {
…..}‘referrers’ : {
‘google’ : 27}
}
22
• Increment new counters
Incrementing Counters 2
>db.interactions.update({‘article_id’ : ObjectId(..)}, {
‘$inc’ : {
‘daily.views’:1,
‘daily.comments’:1,
‘hours.8.views’:1,
‘hours.8.comments’:1,
‘referrers.bing’ : 1})
{‘_id’ : ObjectId(..),
‘article_id’ : ObjectId(..), ‘section’ : ‘schema’,
‘date’ : ISODate(..),‘daily’: { ‘views’ : 45,
‘comments’ : 150 } ‘hours’ : {
…..}‘referrers’ : {
‘google’ : 27,‘bing’ : 1
}}
Durability
24
Durability
• With MongoDB you get to choose• In memory• On disk• Multiple servers
• Write Concerns• Report on success of write operations• getLastError called from driver
• Trade off• Latency of response
25
Unacknowledged
26
MongoDB Acknowledged
Default Write Concern
27
Wait for Journal Sync
28
Replica Sets
• Replica Set – two or more copies
• “Self-healing” shard
• Addresses many concerns:
- High Availability
- Disaster Recovery
- Maintenance
29
Wait for Replication
Summary
31
• Interacting with the database
– Queries and projections– Inserts and Upserts
– Update Operators– Bucketing– Pre Aggregated reports
• basis for fast analytics
Summary
32
– Indexing• Indexing strategies• Tuning Queries
– Text Search
– Geo Spatial
– Query Profiler
Next Session – 6th March