Download as PDF, PPTX
Uploaded byAntonios Giannopoulos
MongoDB Sharding Fundamentals
The document provides an overview of MongoDB sharding, including: - Sharding allows horizontal scaling of data by partitioning a database across multiple servers or shards. - The MongoDB sharding architecture consists of shards to hold data, config servers to store metadata, and mongos processes to route requests. - Data is partitioned into chunks based on a shard key and chunks can move between shards as the data distribution changes.
More Related Content
Similar to MongoDB Sharding Fundamentals
![[Redis Released]- FalkorDB - Redis + Graph Agentic Memory’s Secret Sauce](https://cdn.slidesharecdn.com/ss_thumbnails/redisreleased-falkordbslidedeck-1125-251115194922-e1c0046b-thumbnail.jpg?width=640&height=640&fit=bounds)
MongoDB Sharding Fundamentals
- 1. MongoDB Sharding fundamentals Antonios Giannopoulos Database Administrator at ObjectRocket by Rackspace
- 2. Background -‐ 14 years in databases and system engineering -‐ NoSQL DBA @ ObjectRocket by Rackspace -‐ MongoDB CerFfied DBA
- 3. What is sharding? -‐ A mechanism for horizontal scaling -‐ Distributes the dataset over mulFple servers (shards) -‐ Each shard is an independent database -‐ All shards consists a single logical database
- 4. Why Sharding? -‐ Increases cluster throughput – Read/Write Scaling -‐ Reduces costs -‐ Many small servers VS one big box -‐ Eliminates HW and SW hard limits
- 5. MongoDB Sharding -‐ Consists of three elements: Shards, Config Servers and Mongos -‐ Shards: Hold the cluster data, databases, collecFons, documents (Data nodes) -‐ Config Servers: Hold the cluster metadata, map the cluster architecture. -‐ Mongos: Serve all drivers requests. Route each request to a shard or shards (Router nodes)
- 6. ApplicaFon / Driver Layer Mongos01 Mongos02 MongosN ConfigSrv01 ConfigSrv02 ConfigSrv03 Shard02 ShardN MongoDB Sharded Cluster Shard01 … …
- 7. How Sharding works? -‐ Range parFFoning per collecFon (chunks) -‐ Shard key to define chunks (field(s)) -‐ Chunks are “metadata” on the config servers -‐ Chunks can move, split and merge
- 8. How Sharding works? -‐ Example { "name" : "Angelina", "surname" : "Jolie", "posiFon" : "Windows Eng.", "phone" : "555-‐5555" } { "name" : "Emma", "surname" : "Stone", "posiFon" : "Windows Eng.", "phone" : "555-‐5555" } { "name" : "Charlize", "surname" : "Theron", "posiFon" : "Linux Eng.", "phone" : "555-‐5555" } { "name" : "Olivia", "surname" : "Wilde", "posiFon" : "Linux Eng.", "phone" : "555-‐5555" } { "name" : "Jessica", "surname" : "Alba", "posiFon" : "Sr Linux Eng.", "phone" : "555-‐5555" } { "name" : "Scarlef", "surname" : "Johansson", "posiFon" : "Sr Windows Eng.", "phone" : "555-‐5555" } { "name" : "Megan", "surname" : "Fox", "posiFon" : "Networks Eng.", "phone" : "555-‐5555" } { "name" : "Mila", "surname" : "Kunis", "posiFon" : "Sr Networks Eng.", "phone" : "555-‐5555" } { "name" : "Natalie", "surname" : "Portman", "posiFon" : "Database Eng", "phone" : "555-‐5555" } { "name" : "Anne", "surname" : "Hathaway", "posiFon" : "Sr Database Eng", "phone" : "555-‐5555" } -‐ CollecFon employees for an IT company -‐ Shard key “posi-on”
- 9. How Sharding works? -‐ Example { "min" : { "posiFon" : { "$minKey" : 1 } }, "max" : { "posiFon" : "Database Eng" }, "shard" : ”Shard01" } { "min" : { "posiFon" : "Database Eng" }, "max" : { "posiFon" : "Sr Database Eng" }, "shard" : "Shard01" } { "min" : { "posiFon" : "Sr Database Eng" }, "max" : { "posiFon" : "Windows Eng." }, "shard" : "Shard02" } { "min" : { "posiFon" : "Windows Eng." }, "max" : { "posiFon" : { "$maxKey" : 1 } }, "shard" : "Shard02" } -‐ Lower/upper bound and shard (server)
- 10. Choose a shard key -‐ High Cardinality -‐ Not Null values -‐ Immutable field(s) -‐ Not Monotonically increased fields
- 11. Choose a shard key -‐ Even read/write distribuFon -‐ Even data distribuFon -‐ Read targeFng -‐ Read locality
- 12. Choose a shard key -‐ Hashed shard keys for randomness -‐ Compound shard keys for cardinality -‐ Unique indexes are good -‐ {_id:”hashed”} scales writes
- 13. LimitaFons of Sharding -‐ Unique indexes – Just one… -‐ IniFal collecFon size – Avoid collecFons > 256G, hard limit is a funcFon of key and chunk size , for 64MB chunk/512B key is more than 1TB -‐ Number of documents per chunk (250K)
- 14. LimitaFons of Sharding -‐ Shard key size < 512 bytes -‐ MulFkey,text, geo indexes are prohibited -‐ Some operaFons won’t run (for example group, db.eval(), $isolated, $snapshot, geoSearch)
- 15. “Sharding” – Other players -‐ ApplicaFon level sharding -‐ Mysql (MaxScale, Fabric,…) -‐ Postgres (pg_shard) -‐ ElasFcSearch (Document ID or rouFng) -‐ Cassandra (Hash-‐based -‐ Ring topology)
- 16. Contact www.objectrocket.com www.rackspace.co.uk/objectrocket/mongodb antonios.giannopoulos@rackspace.co.uk We are hiring! (DevOps, DBAs and more) hfp://objectrocket.com/careers
- 17. QuesFons? Thank you!!! MongoDB Meetup What's new in MongoDB 3.0 Tuesday, November 10 , 7:00 pm @ Harokopio University