MongoDB Europe 2016 - Advanced MongoDB Aggregation Pipelines
Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19
-
Upload
henrik-ingo -
Category
Technology
-
view
714 -
download
3
description
Transcript of Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19
![Page 1: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/1.jpg)
What's New in MongoDB 2.4
![Page 2: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/2.jpg)
Improvements made across…
• Development
• Operations
• Performance
• Security
• Enterprise Features
![Page 3: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/3.jpg)
MongoDB 2.4 Developer Improvements
![Page 4: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/4.jpg)
Geospatial
![Page 5: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/5.jpg)
New Geospatial Features
• Polygon intersections
• More accurate spherical model
• $near and $within work with Aggregation Framework
• Analytics now location-aware
![Page 6: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/6.jpg)
GeoJSON
• Geospatial interchange format based on JSON
• Supports Points, LineStrings & Polygons
• Complete spec available at http://geojson.org
![Page 7: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/7.jpg)
Point1 = {
type : "Point",
coordinates : [-73.947807, 40.663973]
}
somePoly = {
type : "Polygon",
coordinates : [[[40,5], [40,6],
[41,6], [41,5], [40,5]]]
}
GeoJSON
![Page 8: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/8.jpg)
$GeoIntersects
• Leverages GeoJSON
• Geospatial query operator that selects all locations that intersect with a GeoJSON object
• Uses spherical geometry
![Page 9: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/9.jpg)
point1 = { type: "Point", coordinates: [40, 5.5]}
line1 = { type: "LineString",
coordinates: [[40, 3.5], [40, 4.5]]};
somePoly = { type : "Polygon",
coordinates : [[[40,5], [40,6],
[41,6], [41,5], [40,5]]]};
res = c.find({ geo : {
"$geoIntersects" :
{"$geometry" : somePoly}
}});
$GeoIntersects
![Page 10: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/10.jpg)
Aggregation Framework
![Page 11: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/11.jpg)
New Aggregation Framework Features • Introduced in 2.2, came of age in 2.4
• Performance Improvements • 3 – 5 x faster
• Geo $near and $within support
• $concat support
• Support for Binary Data (pass through)
![Page 12: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/12.jpg)
Text Search (beta)
![Page 13: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/13.jpg)
Text Search (beta)
• Real-time indexes
• Case-insensitive
• Stemming, tokenization & stop words for 15 languages
• Index size is comparable with other full text search implementations (larger than standard MongoDB indexes)
• BETA status… please use in dev & give feedback
![Page 14: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/14.jpg)
Stemming Examples
• { walk, walked, walking, walks } ⇒ walk
• {magazine, magazines, magazine’s } ⇒ magazine
• {runs, running, run, ran } ⇒ { run, ran }
![Page 15: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/15.jpg)
Examples of English Stop Words
{ am, themselves, of, before, here, while, what's, myself, ought, me, the, into, about, this, do, can't, a, ... }
![Page 16: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/16.jpg)
Option 1: on the command line
$ mongod --setParameter textSearchEnabled=true
Option 2: as admin command to mongod or mongos
> db.adminCommand(
{ setParameter: 1, textSearchEnabled: true }
)
Enabling Text Search
![Page 17: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/17.jpg)
db.t.ensureIndex( { title: "text", post: "text" }, { weights: { title: 10, post: 5 }} )
Create a Text Search Index
![Page 18: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/18.jpg)
db.t.runCommand( "text", { search: "\"my first blog entry\"" } );
Run a Text Search Command
![Page 19: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/19.jpg)
{ "queryDebugString":"blog|entri|first||||my first blog entry||", "language" : "english", "results" : [{ "score" : 12, "obj" : { "_id" : 1, "title" : "1st post", "post" : "this is my first blog entry." }}], "stats" : { "nscanned" : 7, "nscannedObjects" : 0, "n" : 0, "nfound" : 1, "timeMicros" : 122 }, "ok" : 1 }
Text Search Result
Note: debug string shows the stemmed terms, stop-words deleted
![Page 20: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/20.jpg)
New Update Operators
![Page 21: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/21.jpg)
Capped Arrays
• Provides the ability to manipulate arrays with more control than ever before
• $push now supports the following:
• $each permits pushing multiple entries onto an array
• $slice maintains a fixed size array
• Can be stacked with $sort
![Page 22: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/22.jpg)
db.students.update(!
{ _id: 1 },!
{ $push: {!
scores: {!
$each: [!
{ attempt: 3, score: 7 },!
{ attempt: 4, score: 4 } ],!
$sort: { score: 1 },!
$slice: -3!
}!
}}!
)
Top 3 Scores
![Page 23: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/23.jpg)
New Upsert Operator
![Page 24: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/24.jpg)
db.products.update(!
{ _id: 1 },!
{ $setOnInsert: { defaultQty: 500, inStock: true },!
$set: { item: "apple" } },!
{ upsert: true }!
)
$setOnInsert
![Page 25: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/25.jpg)
MongoDB 2.4 Operational Improvements
![Page 26: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/26.jpg)
Hash-based Sharding
![Page 27: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/27.jpg)
Hash-based Sharding • Easier to manage clusters
• Less rebalancing • More event distribution for reads and writes
• Lower potential performance from range based, but more consistent.
• The hash stored in the hashed index is 64 bits of the 128 bit md5 hash
• MongoDB can use the hashed index to support equality queries, but hashed indexes do not support range queries.
![Page 28: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/28.jpg)
db.activeCollection.ensureIndex( { field_to_hash: "hashed" } )
Creating a Hashed Shard Key
![Page 29: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/29.jpg)
Working Set Analyzer
![Page 30: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/30.jpg)
Working Set Analyzer
• Working Set: The set of data kept in memory
• MongoDB performance best when working set < RAM
• Working set analyzer measures resources used over time
• Leads to more efficient MongoDB usage
![Page 31: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/31.jpg)
db.serverStatus( { workingSet: 1 } )
db.runCommand( { serverStatus: 1, workingSet: 1 } )
"workingSet" : {
"note" : "thisIsAnEstimate",
"pagesInMemory" : <num>,
"computationTimeMicros" : <num>,
"overSeconds" : num
},
MongoStatus
![Page 32: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/32.jpg)
Index Operation Management
![Page 33: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/33.jpg)
Index Operation Management
• A single mongod instance can build multiple indexes in the background at the same time.
• db.killOp() can now kill foreground index builds
• Improved validation of index types
![Page 34: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/34.jpg)
db.currentOp() { "inprog" : [ { "opid" : 45, "active" : true, "secs_running" : 2, "op" : "insert", "ns" : "test.system.indexes", ... }] } db.killOp(45)
Index Operation Management
![Page 35: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/35.jpg)
Replication Improvements
![Page 36: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/36.jpg)
Role Based Privileges
Replication Improvements
• Better detection of network hiccups (less false negatives)
• Faster initial sync when adding new secondary
![Page 37: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/37.jpg)
MongoDB 2.4 Performance Improvements
![Page 38: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/38.jpg)
V8 JavaScript Engine
• Affects all JS processing including MapReduce, the shell and $where queries
• Greater concurrency
• User feedback very positive indicating dramatic improvement in overall processing time with new V8
![Page 39: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/39.jpg)
Improvements All Over
• Aggregation 3x – 5x faster
• Faster Counting – Low-cardinality index-based counts up to 20x faster – Better performance on counting, e.g., count all the
males/females in my user list
• Faster $elemMatch
![Page 40: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/40.jpg)
MongoDB 2.4 Security Improvements
![Page 41: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/41.jpg)
Role Based Privileges
Role Based Privileges
• Builds on access controls introduced in 2.2
• Users granted roles that have specific privileges per database
• Users can have multiple roles
• Roles include – read – readWrite – userAdmin – dbAdmin – clusterAdmin
![Page 42: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/42.jpg)
Role Based Privileges
Enhanced SSL Support
• Supported in open source edition of MongoDB
• Must be compiled using --ssl flag
• Uses a standard .pem file that contains the SSL certificate and key
• Complete directions available in the MongoDB documentation http://docs.mongodb.org/manual/administration/ssl/
![Page 43: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/43.jpg)
Introducing MongoDB Enterprise 2.4
![Page 44: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/44.jpg)
MongoDB Enterprise • Advanced Security
– Kerberos authentication protocol – SSL support built in
• Monitoring – On-Prem Monitoring - visualization, alerts on 100+ system metrics – Includes same features as (MMS)
• Enterprise Software Integration – SNMP supports integration w/popular monitoring tools (e.g., Nagios )
• Certified OS Support – Red Hat/CentOS, Ubuntu and Amazon Linux
![Page 45: Whats new in mongoDB 2.4 at Copenhagen user group 2013-06-19](https://reader036.fdocuments.us/reader036/viewer/2022081400/55515b08b4c9059f768b4abf/html5/thumbnails/45.jpg)
Subscriptions
Basic Standard Enterprise
Edition MongoDB MongoDB MongoDB Enterprise
SLA 4 hours 1 Hour 30 Minutes
Support 9am – 9pm ET
M – F 24x7x365 24x7x365
License AGPL Commercial Commercial
Price per Host $2,500 $5,000 $7,500