An environment for experimenting with sharding and replication in MongoDB
4 min read.
When I am learning something new I like to get practical. Reading about something only takes me so far, at a certain point I need to DO something with it. I need to see it in action.
hello world !== hello world
Not too long ago I wanted to play around with replication and sharding in MongoDB. I began reading docs, then I read some more docs and after a while things went from exciting and interesting to kind of boring. My mind started to wander off and as I read the same sentence for the third time I realised that this approach (only reading before trying) wasn’t really working for me.
At this point I wanted to play with something concrete. But manually standing up a sharded cluster requires quite some time and effort and when you want to reset due to some failed experiment you have to do it all over again. However, thanks to some great tech like Docker and Node.js, I felt confident that a simple hello world example could not be far away… but there are hello world examples and then there are hello world examples!
procrastination => automation
What i really wanted was a quick and easy way to automagically and reproducibly spin up a pre-cooked sharded cluster to poke at and tweak in my experiments.
So I created dist-node-mongo - an environment for learning and experimenting with sharding and replication in mongodb (Yeah … naming is hard … could have just named it Fred).
dist-node-m….Fred … uses Node.js, ShellJS and Docker. With this learning environment (yes! Lets call it that!) installed you can easily stand up the following demo system:
You can clone this environment and, as you read the official docs, go straight to:
- (in the code) see what is needed step by step to get this up and running in docker containers
- trying out different shard keys (… and strategies)
- killing a primary in a replica set and see the election happen between the remaining nodes
- trying backups, failovers etc.. etc..
In this screenshot you can see me creating 1000 documents using Hulken (the environment comes with a simple smash.js script), checking the shard distribution in Mongos, stopping and starting a replica set primary and verifying replication to secondary nodes. (i.e playing around)
If you are learning sharding and replication in MongoDB I hope you will find this environment helpful!
If you improve this environment please share it with me! send a pull request or a link to your fork if you rather want to create an alternative version.