Sharding
-
Upload
mongodb -
Category
Technology
-
view
6.052 -
download
2
description
Transcript of Sharding
![Page 1: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/1.jpg)
Software Engineer, 10gen
Tyler Brock
Introduction to Sharding
Wednesday, March 27, 13
![Page 2: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/2.jpg)
Agenda
• Scaling Data• MongoDB's Approach• Architecture• Configuration• Mechanics
Wednesday, March 27, 13
![Page 3: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/3.jpg)
Scaling Data
Wednesday, March 27, 13
![Page 4: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/4.jpg)
Examining Growth
Wednesday, March 27, 13
![Page 5: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/5.jpg)
Examining Growth
• User Growth– 1995: 0.4% of the world’s population– Today: 30% of the world is online (~2.2B)– Emerging Markets & Mobile
Wednesday, March 27, 13
![Page 6: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/6.jpg)
Examining Growth
• User Growth– 1995: 0.4% of the world’s population– Today: 30% of the world is online (~2.2B)– Emerging Markets & Mobile
• Data Set Growth– Facebook’s data set is around 100 petabytes– 4 billion photos taken in the last year (4x a decade ago)
Wednesday, March 27, 13
![Page 7: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/7.jpg)
Working Set Exceeds Physical Memory
Wednesday, March 27, 13
![Page 8: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/8.jpg)
Read/Write Throughput Exceeds I/O
Wednesday, March 27, 13
![Page 9: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/9.jpg)
Vertical Scalability (Scale Up)
Wednesday, March 27, 13
![Page 10: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/10.jpg)
Horizontal Scalability (Scale Out)
Wednesday, March 27, 13
![Page 11: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/11.jpg)
Data Store Scalability
• Custom Hardware– Oracle
• Custom Software– Facebook + MySQL– Google
Wednesday, March 27, 13
![Page 12: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/12.jpg)
Data Store Scalability Today
• MongoDB Auto-Sharding• A data store that is
– Free– Publicly available– Open Source (https://github.com/mongodb/mongo)– Horizontally scalable– Application independent
Wednesday, March 27, 13
![Page 13: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/13.jpg)
MongoDB's Approach to Sharding
Wednesday, March 27, 13
![Page 14: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/14.jpg)
Partitioning
• User defines shard key• Shard key defines range of data• Key space is like points on a line• Range is a segment of that line
Wednesday, March 27, 13
![Page 15: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/15.jpg)
Data Distribution
• Initially 1 chunk• Default max chunk size: 64mb• MongoDB automatically splits & migrates chunks
when max reached
Wednesday, March 27, 13
![Page 16: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/16.jpg)
Routing and Balancing
• Queries routed to specific shards
• MongoDB balances cluster• MongoDB migrates data to
new nodes
Wednesday, March 27, 13
![Page 17: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/17.jpg)
MongoDB Auto-Sharding
• Minimal effort required– Same interface as single mongod
• Two steps– Enable Sharding for a database– Shard collection within database
Wednesday, March 27, 13
![Page 18: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/18.jpg)
Architecture
Wednesday, March 27, 13
![Page 19: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/19.jpg)
What is a Shard?
• Shard is a node of the cluster• Shard can be a single mongod or a replica set
Wednesday, March 27, 13
![Page 20: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/20.jpg)
• Config Server– Stores cluster chunk ranges and locations– Can have only 1 or 3 (production must have 3)– Not a replica set
Meta Data Storage
Wednesday, March 27, 13
![Page 21: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/21.jpg)
Routing and Managing Data
• Mongos– Acts as a router / balancer– No local data (persists to config database)– Can have 1 or many
Wednesday, March 27, 13
![Page 22: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/22.jpg)
Sharding infrastructure
Wednesday, March 27, 13
![Page 23: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/23.jpg)
Configuration
Wednesday, March 27, 13
![Page 24: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/24.jpg)
Example Cluster
• Don’t use this setup in production!- Only one Config server (No Fault Tolerance)- Shard not in a replica set (Low Availability)- Only one mongos and shard (No Performance Improvement)
Wednesday, March 27, 13
![Page 25: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/25.jpg)
Starting the Configuration Server
• mongod --configsvr• Starts a configuration server on the default port (27019)
Wednesday, March 27, 13
![Page 26: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/26.jpg)
Start the mongos Router
• mongos --configdb <hostname>:27019• For 3 configuration servers:
mongos --configdb <host1>:<port1>,<host2>:<port2>,<host3>:<port3>
• This is always how to start a new mongos, even if the cluster is already running
Wednesday, March 27, 13
![Page 27: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/27.jpg)
Start the shard database
• mongod --shardsvr• Starts a mongod with the default shard port (27018)• Shard is not yet connected to the rest of the cluster• Shard may have already been running in production
Wednesday, March 27, 13
![Page 28: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/28.jpg)
Add the Shard
• On mongos: - sh.addShard(‘<host>:27018’)
• Adding a replica set:
Wednesday, March 27, 13
![Page 29: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/29.jpg)
Verify that the shard was added
• db.runCommand({ listshards:1 }) { "shards" : ! [{"_id”: "shard0000”,"host”: ”<hostname>:27018” } ],
"ok" : 1 }
Wednesday, March 27, 13
![Page 30: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/30.jpg)
Enabling Sharding
• Enable sharding on a database
sh.enableSharding(“<dbname>”)
• Shard a collection with the given key
sh.shardCollection(“<dbname>.people”,{“country”:1})
• Use a compound shard key to prevent duplicates
sh.shardCollection(“<dbname>.cars”,{“year”:1, ”uniqueid”:1})
Wednesday, March 27, 13
![Page 31: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/31.jpg)
Tag Aware Sharding
• Tag aware sharding allows you to control the distribution of your data
• Tag a range of shard keys– sh.addTagRange(<collection>,<min>,<max>,<tag>)
• Tag a shard– sh.addShardTag(<shard>,<tag>)
Wednesday, March 27, 13
![Page 32: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/32.jpg)
Mechanics
Wednesday, March 27, 13
![Page 33: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/33.jpg)
Partitioning
• Remember it's based on ranges
Wednesday, March 27, 13
![Page 34: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/34.jpg)
Chunk is a section of the entire range
Wednesday, March 27, 13
![Page 35: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/35.jpg)
Chunk splitting
• A chunk is split once it exceeds the maximum size• There is no split point if all documents have the same shard key• Chunk split is a logical operation (no data is moved)
Wednesday, March 27, 13
![Page 36: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/36.jpg)
Balancing
• Balancer is running on mongos• Once the difference in chunks between the most dense shard
and the least dense shard is above the migration threshold, a balancing round starts
Wednesday, March 27, 13
![Page 37: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/37.jpg)
Acquiring the Balancer Lock
• The balancer on mongos takes out a “balancer lock”• To see the status of these locks:
use configdb.locks.find({ _id: “balancer” })
Wednesday, March 27, 13
![Page 38: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/38.jpg)
Moving the chunk
• The mongos sends a moveChunk command to source shard• The source shard then notifies destination shard• Destination shard starts pulling documents from source shard
Wednesday, March 27, 13
![Page 39: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/39.jpg)
Committing Migration
• When complete, destination shard updates config server- Provides new locations of the chunks
Wednesday, March 27, 13
![Page 40: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/40.jpg)
Cleanup
• Source shard deletes moved data- Must wait for open cursors to either close or time out- NoTimeout cursors may prevent the release of the lock
• The mongos releases the balancer lock after old chunks are
Wednesday, March 27, 13
![Page 41: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/41.jpg)
Routing Requests
Wednesday, March 27, 13
![Page 42: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/42.jpg)
Cluster Request Routing
• Targeted Queries• Scatter Gather Queries• Scatter Gather Queries with Sort
Wednesday, March 27, 13
![Page 43: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/43.jpg)
Cluster Request Routing: Targeted Query
Wednesday, March 27, 13
![Page 44: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/44.jpg)
Routable request received
Wednesday, March 27, 13
![Page 45: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/45.jpg)
Request routed to appropriate shard
Wednesday, March 27, 13
![Page 46: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/46.jpg)
Shard returns results
Wednesday, March 27, 13
![Page 47: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/47.jpg)
Mongos returns results to client
Wednesday, March 27, 13
![Page 48: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/48.jpg)
Cluster Request Routing: Non-Targeted Query
Wednesday, March 27, 13
![Page 49: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/49.jpg)
Non-Targeted Request Received
Wednesday, March 27, 13
![Page 50: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/50.jpg)
Request sent to all shards
Wednesday, March 27, 13
![Page 51: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/51.jpg)
Shards return results to mongos
Wednesday, March 27, 13
![Page 52: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/52.jpg)
Mongos returns results to client
Wednesday, March 27, 13
![Page 53: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/53.jpg)
Cluster Request Routing: Non-Targeted Query with Sort
Wednesday, March 27, 13
![Page 54: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/54.jpg)
Non-Targeted request with sort received
Wednesday, March 27, 13
![Page 55: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/55.jpg)
Request sent to all shards
Wednesday, March 27, 13
![Page 56: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/56.jpg)
Query and sort performed locally
Wednesday, March 27, 13
![Page 57: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/57.jpg)
Shards return results to mongos
Wednesday, March 27, 13
![Page 58: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/58.jpg)
Mongos merges sorted results
Wednesday, March 27, 13
![Page 59: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/59.jpg)
Mongos returns results to client
Wednesday, March 27, 13
![Page 60: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/60.jpg)
Shard Key
Wednesday, March 27, 13
![Page 61: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/61.jpg)
Shard Key
• Shard key is immutable• Shard key values are immutable• Shard key must be indexed• Shard key limited to 512 bytes in size• Shard key used to route queries
– Choose a field commonly used in queries
• Only shard key can be unique across shards– `_id ̀field is only unique within individual shard
Wednesday, March 27, 13
![Page 62: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/62.jpg)
Shard Key Considerations
• Cardinality• Write Distribution• Query Isolation• Reliability• Index Locality
Wednesday, March 27, 13
![Page 63: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/63.jpg)
Conclusion
Wednesday, March 27, 13
![Page 64: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/64.jpg)
Read/Write Throughput Exceeds I/O
Wednesday, March 27, 13
![Page 65: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/65.jpg)
Working Set Exceeds Physical Memory
Wednesday, March 27, 13
![Page 66: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/66.jpg)
Sharding Enables Scale
• MongoDB’s Auto-Sharding– Easy to Configure– Consistent Interface– Free and Open Source
Wednesday, March 27, 13
![Page 67: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/67.jpg)
• What’s next?– [ Insert Related Talks ]– [ Insert Upcoming Webinars ]– MongoDB User Group
• Resourceshttps://education.10gen.com/
http://www.10gen.com/presentations
Wednesday, March 27, 13
![Page 68: Sharding](https://reader033.fdocuments.us/reader033/viewer/2022060109/5558764dd8b42aaa7e8b546d/html5/thumbnails/68.jpg)
Software Engineer, 10gen
Tyler Brock
Thank You
Wednesday, March 27, 13