MongoDB London 2013: Basic Replication in MongoDB presented by Marc Schwering, 10gen
MongoDB Database Replication
-
Upload
mehdi-valikhani -
Category
Engineering
-
view
501 -
download
2
Transcript of MongoDB Database Replication
![Page 2: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/2.jpg)
Replication
Process of keeping IDENTICAL COPY of your data on different servers.
Replication offersMinimum Downtimein case of hardware failure, software failure or maintenance
![Page 3: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/3.jpg)
Replication is highly recommended for all databases in production from the very first day.
![Page 4: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/4.jpg)
Replica SetMongoDB’s Terminology for Replication
A replica set is a group of servers with one primary, the server taking client requests, and multiple secondaries, servers that keep copies of the primary’s data.
![Page 5: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/5.jpg)
Give A Go To MongoDB’s Replica SetIn60 Seconds!
#ToBeTriedOnlyAtHome
![Page 6: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/6.jpg)
Step 1: Setup Replica Set
$: mongo --nodb
$: myReplica = new ReplSetTest({nodes: 3});
$: myReplica.startSet();
$: myReplica.initiate();
![Page 7: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/7.jpg)
Congratulations!3 member of replica set running on ports 31000, 31001 and 31002
![Page 8: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/8.jpg)
Step 2: Connect To Primary
$: mongo --port 31000
testReplSet:PRIMARY> use surryhills_db
testReplSet:PRIMARY> db.offices.insert({name: ‘MongoDB’});
testReplSet:PRIMARY> show dbs
testReplSet:PRIMARY> show collections
testReplSet:PRIMARY> db.offices.find().pretty()
![Page 9: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/9.jpg)
Pro Tip :] Auto Format Query Response
Add DBQuery.prototype._prettyShell = true;to your ~/.mongorc.js
![Page 10: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/10.jpg)
Step 3: Connect To A Secondary
$: mongo --port 31001
testReplSet:SECONDARY> db.slaveOk()
testReplSet:SECONDARY> show dbs
testReplSet:SECONDARY> use surryhills_db
testReplSet:SECONDARY> show collections
testReplSet:SECONDARY> db.offices.find()
![Page 11: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/11.jpg)
Step 4: Shutdown Replica Set
$: myReplica.stopSet()
![Page 12: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/12.jpg)
Setup Your Replica SetPart 1: Run replica members
$: mongod --port 31000 --fork --logpath ./path/to/log --dbpath /path/to/srv1 --replSet myReplica
$: mongod --port 31001 --fork --logpath ./path/to/log --dbpath /path/to/srv2 --replSet myReplica
$: mongod --port 31002 --fork --logpath ./path/to/log --dbpath /path/to/srv3 --replSet myReplica
![Page 13: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/13.jpg)
Setup Your Replica SetPart 2: Configure the set
$: mongo --port 31000
$: rs.initiate();
$: rs.add(‘my-host:31001’);
$: rs.add(‘my-host:31002’);
![Page 14: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/14.jpg)
Setup Your Replica SetPart 2: Example Configuration$: rs.config();{ "_id" : "myReplica",
"version": 3, "members" : [ { "_id" : 0, "host" : "my-host:31000" }, { "_id" : 1, "host" : "my-host:31001" }, { "_id" : 2, "host" : "my-host:31002" } ]}
![Page 15: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/15.jpg)
Auto Connect To Current Primary
$: mongo --host myReplica/my-host:31000,my-host:31001,my-host:31002
![Page 16: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/16.jpg)
Replica Set Good To Knows
1. Always start your instance with `replSet`
2. You can’t initiate a replica set with more than one member with existing data on it.
3. If one of set members has data, you must run rs.initiate() against that member
4. `system.replset` collection on `local` database of all members contains set configuration.
5. Replica sets can have up to 50 members, 7 voting
6. By default, MongoDB reads from primary
![Page 17: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/17.jpg)
Replica Set ReconfigurationAdd new member
$: mongo --port 31000 # current primary
$: rs.add(‘my-host:31003’) # accepts object as well.
![Page 18: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/18.jpg)
Replica Set ReconfigurationRemove existing member
$: mongo --port 31000 # current primary
$: rs.remove(‘my-host:31003’)
![Page 19: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/19.jpg)
Replica Set ReconfigurationUpdate existing member
$: mongo --port 31000 # current primary
$: config = rs.config()
$: config.members[2].priority = 0;
$: rs.reconfig(config);
![Page 20: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/20.jpg)
How Replication WorksMongoDB keeps a log of every write operations that primary does in database
Logs are saved in “oplog.rs” collection of “local” db
Secondaries apply changes by reading from oplog.rs and applieing to their data
Secondaries maintain their own “oplog.rs” as well.
![Page 21: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/21.jpg)
“oplog.rs” Facts“oplog.rs” is a capped collection which means it can save limited amount of operation logs.
Bulk operations fill “oplog” quickly. A bulk removal of 1K documents adds 1K documents to “oplog”
It’s important to have a “oplog.rs” to keep logs for atleast last 24 hrs.
![Page 22: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/22.jpg)
“Stale” Secondary :(Secondary misses a copy of last operation it applied in syncing source.
Secondary queries other members to find another source to continue syncing.
Secondary missed some operations permanently. Admin must resyncs from scratch.
![Page 23: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/23.jpg)
HeartbeatMembers PING each other every 2 seconds.
Members should reply within 10 seconds otherwise they’re counted as unreachable.
![Page 24: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/24.jpg)
Change in replica set configuration
Current primary steps down
Replica Set ElectionWhen It Happens?
Current primary is unreachable
![Page 25: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/25.jpg)
MajorityThe Single Most Important Fact About Replica Set
More than half of All Voting Members in the set must be reachable.
![Page 26: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/26.jpg)
You need a majority of voting members to elect a primary.
A primary can only stay primary so long as it can reach a majority of voting members.
Majority
![Page 27: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/27.jpg)
Replica Set with two members.
Three Common Replica Set Design Mistakes :(
Even number of members in two data center.
A set with more than one arbiter
![Page 28: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/28.jpg)
Keep a majority of members in preferred data center and rest in the other one.
Two Common Replica Set Design Patterns :)
Keep even number of members in two data center and a tie breaker in third one.
![Page 29: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/29.jpg)
Meet Arbiter Budget Friendly Member Of Replica Set
Does not hold data
It’s a lightweight process
Can vote on elections
![Page 30: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/30.jpg)
Start An Arbiter Process
$: mongod --port 31005 --replSet myReplica --nojournal --smallfiles
Arbiter does not save data, no need for journaling
Don’t waste disk space and use --smallfiles
![Page 31: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/31.jpg)
Add An Arbiter To Replica Set
$: rs.addArb(‘arbiter-host-name:port’);
![Page 32: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/32.jpg)
ArbiterGood To Know
You can’t change an arbiter to non-arbiter member
If possible, use normal members instead of arbiters
At most you need one arbiter in your set
Do not add an arbiter to a set with odd members
![Page 33: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/33.jpg)
Replica SetUseful Commands
$: rs.config()
$: rs.reconfig()
$: rs.isMaster()
$: rs.status()
$: rs.stepDown()
![Page 34: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/34.jpg)
Rollback :(Primary does a write and goes down before at least one secondary has a chance to replicate the write.
![Page 35: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/35.jpg)
RollbackMain Reasons
Network Partitions
Different write rates between primary and secondary
![Page 36: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/36.jpg)
Rollback Process
1. Former primary reverts write operations
2. Saves affected documents into /rollback directory
3. Admin browses documents using mongorestore
4. Admin applies changes to the current primary
![Page 37: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/37.jpg)
DurabilityWrite Concerns And Replica Set
1. Unacknowledged
2. Acknowledged
3. Journaled
4. Replica Acknowledged
5. Custom Writer Concern
![Page 38: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/38.jpg)
Improve Durabilitydb.runCommand({getLastError:1}) options
Returns the error status of the preceding write operation on the current connection.
j: boolean
w: number | majority | custom role
wtimeout: number (milliseconds)
fsync: boolean
![Page 39: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/39.jpg)
“wtimeout” GOOD TO KNOW
MongoDB does not “rollback” or ”undo modifications” made before the “wtimeout” interval expired.
![Page 40: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/40.jpg)
Custom Write ConcernsStep 1: Tags
$: config = rs.config()
$: config.members[0].tags = {dc: ‘dc-1’};
$: config.members[3].tags = {dc: ‘dc-2’};
$: rs.reconfig(config);
![Page 41: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/41.jpg)
Custom Write ConcernsStep 2: Custom Last Error Mode
$: config = rs.config()
$: config.settings.lastErrorModes = [{
“each-dc”: {dc: 2}
}];
$: rs.reconfig(config);
![Page 42: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/42.jpg)
Custom Write ConcernsStep 3: Use it!
$: db.runCommand({getLastError:1, w: ‘each-dc’})
![Page 43: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/43.jpg)
Replica Set & Mixed Storage Engines
You can have both WiredTiger & MMAPv1
● Due to migration● Due to different use cases
![Page 44: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/44.jpg)
Keen To Go Further?
Register for M102: MONGODB FOR DBAS course!
Starts tomorrow!It’s awesome!
https://university.mongodb.com/courses/M102/about
![Page 45: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/45.jpg)
Keen To Go Further?docs.mongodb.org/manual/replication
MongoDBThe Definite Guide
![Page 46: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/46.jpg)
Stay In Touch● Follow me on Twitter
@mehdivk
● Add me to your Linkedin network https://au.linkedin.com/in/valikhani
● Subscribe to my blogblog.mehdivk.net
● Send me an [email protected]
![Page 47: MongoDB Database Replication](https://reader036.fdocuments.us/reader036/viewer/2022062300/55b6d401bb61eb8c418b491d/html5/thumbnails/47.jpg)
Thank You!