Document-Level Concurrency

Document-Level

Concurrency

3.0: Document-Level Concurrency

0

50,000

100,000

150,000

200,000

250,000

300,000

Load 95/5read/write

50/50read/write

YCSB – Replica Set, Journaling Enabled

MongoDB 2.6 MongoDB 3.0

3.0: Document-Level Concurrency

3.0: Document-Level Concurrency

Storage Engine

API

3.0: Storage Engine API

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

On Disk DB Size

ENRON Email – Compression Performance

Original MMAPv1 WT - no compression WT - Snappy

3.0: Storage Engine API

One more storage

engine

3.0: Storage Engine API

3.0: Storage Engine API

/dev/null

Automation

3.0: Automation

Complete

Management

Toolkit

3.0: Cloud Manager

Rules

and

Tools

3.2: Rules and Tools

Schemas

db.allthethings.ensureIndex(

{”thing_id": 1},

{filter: {flags: "good”}}

)

3.2: Rules and Tools

Partial Indexes

db.runCommand({

collMod: "robots",

validator: {

laws_implemented: {$gte: 3}

}

})

3.2: Rules and Tools

Validation

3.2: Rules and Tools

> cursor.next()

{

"first_name" : "Rylee",

"last_name" : "Friesen",

"suffix" : "II"

"email" : {

"work" : "rylee.ii.friesen@bartell.org"

},

"phone_number" : {

"work" : "(346) 182-8049 x3571"

},

"last_contact" : "Thu May 28 2015 18:21:45 GMT-0400 (EDT)",

number_of_licenses : 5

}

Compass

3.2: Rules and Tools

> cursor.next()

{

"first_name" : "Jasen",

"middle_name" : "Joseph",

"last_name" : "Jerde",

"phone_number" : {

"work" : "(901) 161-0043",

"cell" : "(534) 815-4550"

},

number_of_licenses : 1

}

> db.customers.find()

Compass

3.2: Rules and Tools

{

"first_name" : "Lavada",

"last_name" : "Padberg",

"email" : {

"work" : "padberg_lavada@harris.info"

},

"title" : "National Markets Developer",

"last_contact" : "Fri May 29 2015 13:38:33 GMT-0400 (EDT)",

}

Compass

{

"first_name" : <string>,

"middle_name" : <string>,

"last_name" : <string>,

"suffix" : <string>,

"title" : <string>,

"phone_number" : {

"work" : <string>,

"cell" : <string>

},

"email" : {

"work" : <string>

}

"last_contact" : <date>,

"number_of_licenses" : <number>

}

Compass

3.2: Rules and Tools

DEMO

3.2: Rules and Tools

Rules

and

Tools

Join me in

querying

your data

3.2: Querying

grep -o "[0-9]\+ms" /var/log/mongodb/output.log |

sort -n | tail -n 20 |

xargs -n 1 -J XXX grep XXX/var/log/mongodb/output.log

Pipelines - unix

db.logs.find().sort( { time : -1 }).limit(20)

3.2: Querying

Pipelines - MongoDB

db.users.aggregate(

[

{ $unwind : "$likes" },

{ $group : { _id : "$likes" ,

number : { $sum : 1 } } },

{ $sort : { number : -1 } },

{ $limit : 5 }

]

)

3.2: Querying

Pipelines - MongoDB

Reports!

3.2: Querying

$stdDevPop

$stdDevSamp

$sumArray

$concatArrays

$arrayElemAt

$slice

$isArray

$filterArray

$avg

$min

$max

$sqrt

$log

$log10

$ln

$pow

$exp

$abs

$round

$trunc

$ceil

$floor

3.2: Querying

Reports!

“Show me the MongoDB Days

attendees who are active Cloud

Manager users, with MongoDB

hosts that have checked in with

Cloud Manager today, grouped by

MongoDB version.”

3.2: Querying

Reports!

mdbdays_attendees

{ first : "Eliot", last : "horowitz, email : "eliot@mongodb.com", zip

: "10024", title : "CTO" }

cloud_users

{ first : "Eliot", last : "horowitz, email : "eliot@mongodb.com",

groups : [ 12321, 51234 ]}

cloud_hosts

{ hostname: "foo.mongodb.com", group_id : 51234, last_ping : new

ISODate("2015-06-02Z05:42:12"), version : "3.0.3”}

3.2: Querying

Reports! – Example Schema

versions = {}

db.mdbdays_attendees.find().forEach(function(

user) {

var cloudUser = db.cloud_users.findOne({

email : user.email, active : true }); if

(!cloudUser ) {

return;

}

for (var i=0; i < cloudUser.groups.length;

i++) {

3.2: Querying

Reports! – Uh oh

db.mdbdays_attendeees.aggregate([

{ $lookup : { "from" : ”cloud_users", on: { email : "$email" }, as:

cloud_user_doc } },

{ $match : { cloud_user_doc.active : true } },

{ $unwind : ”$cloud_user_doc.groups"},

{ $lookup : { "from" : ”cloud_hosts", on: { group_id:

”$cloud_user_doc.groups"}, as: host}}

{ $match : { last_ping : {$gte : new ISODate("2015-06-01Z00:00:00") } } },

{ $group : { _id : "$version", count : { $sum : 1}}}

])

3.2: Querying

Reports!

3.2: Querying

DEMO

Join me in

querying

your data

1 picture >

1.0E3

queries

Data Insights

3.2

Encryption at Rest

Config Servers as Replica

Set

In Memory

SE

Raft

Read

Committed

Partial Index

WT as DefaultDoc Validation

General Purpose Auth

Networking Layer

clang-format

L D A P driver handshake

V8 -> SpiderMonkey

Per index stats Read after Optime

dump/restore archiving

Oplog Replication

RefactorBitwise query ops

Read Commited dump/restore compression

“Black Box” data file Journaling Perf in WT Oplog perf

open source evergreen

Data Replication

Refactoring

Collection Sampling

Misc Agg

Rules and Tools Aggregation Data Insights

Summary

Thank You

Thank You