MongoDB Days Silicon Valley: Introducing MongoDB 3.2
52
-
Upload
mongodb -
Category
Technology
-
view
898 -
download
2
Transcript of MongoDB Days Silicon Valley: Introducing MongoDB 3.2
Document-Level Concurrency
Document-Level
Concurrency
3.0: Document-Level Concurrency
0
50,000
100,000
150,000
200,000
250,000
300,000
Load 95/5read/write
50/50read/write
YCSB – Replica Set, Journaling Enabled
MongoDB 2.6 MongoDB 3.0
3.0: Document-Level Concurrency
3.0: Document-Level Concurrency
Storage Engine
API
3.0: Storage Engine API
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
On Disk DB Size
ENRON Email – Compression Performance
Original MMAPv1 WT - no compression WT - Snappy
3.0: Storage Engine API
One more storage
engine
3.0: Storage Engine API
3.0: Storage Engine API
/dev/null
Automation
3.0: Automation
Complete
Management
Toolkit
3.0: Cloud Manager
Rules
and
Tools
3.2: Rules and Tools
Schemas
db.allthethings.ensureIndex(
{”thing_id": 1},
{filter: {flags: "good”}}
)
3.2: Rules and Tools
Partial Indexes
db.runCommand({
collMod: "robots",
validator: {
laws_implemented: {$gte: 3}
}
})
3.2: Rules and Tools
Validation
3.2: Rules and Tools
> cursor.next()
{
"first_name" : "Rylee",
"last_name" : "Friesen",
"suffix" : "II"
"email" : {
"work" : "[email protected]"
},
"phone_number" : {
"work" : "(346) 182-8049 x3571"
},
"last_contact" : "Thu May 28 2015 18:21:45 GMT-0400 (EDT)",
number_of_licenses : 5
}
Compass
3.2: Rules and Tools
> cursor.next()
{
"first_name" : "Jasen",
"middle_name" : "Joseph",
"last_name" : "Jerde",
"phone_number" : {
"work" : "(901) 161-0043",
"cell" : "(534) 815-4550"
},
number_of_licenses : 1
}
> db.customers.find()
Compass
3.2: Rules and Tools
{
"first_name" : "Lavada",
"last_name" : "Padberg",
"email" : {
"work" : "[email protected]"
},
"title" : "National Markets Developer",
"last_contact" : "Fri May 29 2015 13:38:33 GMT-0400 (EDT)",
}
Compass
{
"first_name" : <string>,
"middle_name" : <string>,
"last_name" : <string>,
"suffix" : <string>,
"title" : <string>,
"phone_number" : {
"work" : <string>,
"cell" : <string>
},
"email" : {
"work" : <string>
}
"last_contact" : <date>,
"number_of_licenses" : <number>
}
Compass
3.2: Rules and Tools
DEMO
3.2: Rules and Tools
Rules
and
Tools
Join me in
querying
your data
3.2: Querying
grep -o "[0-9]\+ms" /var/log/mongodb/output.log |
sort -n | tail -n 20 |
xargs -n 1 -J XXX grep XXX/var/log/mongodb/output.log
Pipelines - unix
db.logs.find().sort( { time : -1 }).limit(20)
3.2: Querying
Pipelines - MongoDB
db.users.aggregate(
[
{ $unwind : "$likes" },
{ $group : { _id : "$likes" ,
number : { $sum : 1 } } },
{ $sort : { number : -1 } },
{ $limit : 5 }
]
)
3.2: Querying
Pipelines - MongoDB
Reports!
3.2: Querying
$stdDevPop
$stdDevSamp
$sumArray
$concatArrays
$arrayElemAt
$slice
$isArray
$filterArray
$avg
$min
$max
$sqrt
$log
$log10
$ln
$pow
$exp
$abs
$round
$trunc
$ceil
$floor
3.2: Querying
Reports!
“Show me the MongoDB Days
attendees who are active Cloud
Manager users, with MongoDB
hosts that have checked in with
Cloud Manager today, grouped by
MongoDB version.”
3.2: Querying
Reports!
mdbdays_attendees
{ first : "Eliot", last : "horowitz, email : "[email protected]", zip
: "10024", title : "CTO" }
cloud_users
{ first : "Eliot", last : "horowitz, email : "[email protected]",
groups : [ 12321, 51234 ]}
cloud_hosts
{ hostname: "foo.mongodb.com", group_id : 51234, last_ping : new
ISODate("2015-06-02Z05:42:12"), version : "3.0.3”}
3.2: Querying
Reports! – Example Schema
versions = {}
db.mdbdays_attendees.find().forEach(function(
user) {
var cloudUser = db.cloud_users.findOne({
email : user.email, active : true }); if
(!cloudUser ) {
return;
}
for (var i=0; i < cloudUser.groups.length;
i++) {
3.2: Querying
Reports! – Uh oh
db.mdbdays_attendeees.aggregate([
{ $lookup : { "from" : ”cloud_users", on: { email : "$email" }, as:
cloud_user_doc } },
{ $match : { cloud_user_doc.active : true } },
{ $unwind : ”$cloud_user_doc.groups"},
{ $lookup : { "from" : ”cloud_hosts", on: { group_id:
”$cloud_user_doc.groups"}, as: host}}
{ $match : { last_ping : {$gte : new ISODate("2015-06-01Z00:00:00") } } },
{ $group : { _id : "$version", count : { $sum : 1}}}
])
3.2: Querying
Reports!
3.2: Querying
DEMO
Join me in
querying
your data
1 picture >
1.0E3
queries
Data Insights
3.2
Encryption at Rest
Config Servers as Replica
Set
In Memory
SE
Raft
Read
Committed
Partial Index
WT as DefaultDoc Validation
General Purpose Auth
Networking Layer
clang-format
L D A P driver handshake
V8 -> SpiderMonkey
Per index stats Read after Optime
dump/restore archiving
Oplog Replication
RefactorBitwise query ops
Read Commited dump/restore compression
“Black Box” data file Journaling Perf in WT Oplog perf
open source evergreen
Data Replication
Refactoring
Collection Sampling
Misc Agg
Rules and Tools Aggregation Data Insights
Summary
Thank You
Thank You
