MongoDB : Step by Step sharding with replica set on local machine

Before we get started, let’s review the components of the setup we’ll be creating:

• Config Server - This stores metadata and configuration settings for the rest of the cluster.
• Query Router - The mongos acts as an interface between the client application and the cluster shards. Since data is distributed among multiple servers, queries need to be routed to the shard where a given piece of information is stored. The query router is run on the application server.
• Shard - A shard is simply a database server that holds a portion of your data. Items in the database are divided among shards either by range or hashing.

Initialize Config ServersPermalink

Rather than using a single config server, we’ll use a replica set to ensure the integrity of the metadata. This enables master-slave (primary-secondary) replication among the three servers and automates fail over so that if your primary config server is down, a new one will be elected and requests will continue to be processed.

The steps below should be performed on each config server individually, unless 

otherwise specified.

Config server 1 :- 

mongod –replSet mysetConfig --logpath "D:\ReplicaConfig\config1\Log\cfg-a.log" --dbpath D:\ReplicaConfig\config1\Data --port 57017 --configsvr --smallfiles

Config server 2: -

mongod –replSet mysetConfig --logpath "D:\ReplicaConfig\config2\Log\cfg-a.log" --dbpath D:\ReplicaConfig\config2\Data --port 57018 --configsvr --smallfiles

Config server 3:

mongod –replSet mysetConfig --logpath "D:\ReplicaConfig\config2\Log\cfg-a.log" --dbpath D:\ReplicaConfig\config2\Data --port 57019 --configsvr --smallfiles

Initialize Replica Sets :- 

We will mongod shell to run the replica set in local machina, We will config three node in each replica set. One node will be primary and other two would be secondary.

To enable the sharding, we need to use --shardsvr as below

Replic Set 1 with sharding :

mongod --dbpath D:\S1Replica\rs1\data -bind_ip 127.0.0.1 --port 27017 --logpath D:\S1Replica\rs1\log\mongod.log --replSet myset --shardsvr

mongod --dbpath D:\S1Replica\rs2\data -bind_ip 127.0.0.1 --port 27018 --logpath D:\S1Replica\rs2\log\mongod.log --replSet myset --shardsvr

mongod --dbpath D:\S1Replica\rs3\data -bind_ip 127.0.0.1 --port 27019 --logpath D:\S1Replica\rs3\log\mongod.log --replSet myset --shardsvr

Use config file to initiate member on replica set as mentioned in my previous post. 

Replica Set 2 with sharding :- 

mongod --dbpath D:\S2Replica\rs1\data -bind_ip 127.0.0.1 --port 27020 --logpath D:\S2Replica\rs1\log\mongod.log --replSet myset1 --shardsvr

mongod --dbpath D:\S2Replica\rs2\data -bind_ip 127.0.0.1 --port 27021 --logpath D:\S2Replica\rs2\log\mongod.log --replSet myset1 --shardsvr

mongod --dbpath D:\S2Replica\rs3\data -bind_ip 127.0.0.1 --port 27022 --logpath D:\S2Replica\rs3\log\mongod.log --replSet myset1 --shardsvr

Replica set 3 with sharding :- 

mongod --dbpath D:\S3Replica\rs1\data -bind_ip 127.0.0.1 --port 27023 --logpath D:\S3Replica\rs1\log\mongod.log --replSet myset2 --shardsvr

mongod --dbpath D:\S3Replica\rs2\data -bind_ip 127.0.0.1 --port 27024 --logpath D:\S3Replica\rs2\log\mongod.log --replSet myset2 --shardsvr

mongod --dbpath D:\S3Replica\rs3\data -bind_ip 127.0.0.1 --port 27025 --logpath D:\S3Replica\rs3\log\mongod.log --replSet myset2 --shardsvr

Configure Query Router

Since we’re only configuring one query router, we’ll only need to do this once. However, it’s also possible to use a replica set of query routers.

mongos --port 30020 --logpath "D:\ReplicaMongos\rs1\Log\mongos-1.log" --configdb mysetConfig/localhost:57017,localhost:57018,localhost:57019

Add Shards to the Cluster

Now that the query router is able to communicate with the config servers, we must enable sharding so that the query router knows which servers will host the distributed data and where any given piece of data is located.

Use mongo shell connect to the query router:

mongo localhost:37017  

From the mongos interface, add each shard individually:

sh.addShard("myset/127.0.0.1:27017,127.0.0.1:27018,127.0.0.1:27019")

sh.addShard("myset1/127.0.0.1:27020,127.0.0.1:27021,127.0.0.1:27022")

sh.addShard("myset2/127.0.0.1:27023,127.0.0.1:27024,127.0.0.1:27025")

Enable Sharding at Collection LevelPermalink

1. Connect to the mongo shell on your query router if you’re not already there:

   mongo localhost:37017  
   

2. Switch to the exampleDB database we created previously:

   use exampleDB
   

3. Create a new collection called exampleCollection and hash its _id key. The _id key is already created by default as a basic index for new documents:

   db.exampleCollection.ensureIndex( { _id : "hashed" } )
   

4. Finally, shard the collection:

   sh.shardCollection( "exampleDB.exampleCollection", { "_id" : "hashed" } )
   

   Add document in the collection and check shard distribution on the collection 

5. db.exampleCollection.getShardDistribution()

 

This will output information similar to the following:

Shard shard0000 at mongo-shard-1:27017
 data : 8KiB docs : 265 chunks : 2
 estimated data per chunk : 4KiB
 estimated docs per chunk : 132

Shard shard0001 at mongo-shard-2:27017
 data : 7KiB docs : 235 chunks : 2
 estimated data per chunk : 3KiB
 estimated docs per chunk : 117

Totals
 data : 16KiB docs : 500 chunks : 4
 Shard shard0000 contains 53% data, 53% docs in cluster, avg obj size on shard : 33B
 Shard shard0001 contains 47% data, 47% docs in cluster, avg obj size on shard : 33B

 

Using Polly to improve .NET applications resilience

This is an open source framework for that "allows developers to express transient exception and fault handling policies
such as Retry, Retry Forever, Wait and Retry, or Circuit Breaker in a fluent manner".

A policy basically defines which exceptions to handle, what to do when an exception occurs and you can tell Polly to retry the original method or break and stop the method being called again until a certain time span has passed.

A policy is created using a fluent style interface, so let’s take a look at some simple examples

1. Data base Connection retry policy :- Here is the example to set retry policy b/w c# and mongo db driver.

     public static RetryPolicy DbRetryPolicy()

        {

            return Policy.Handle<System.TimeoutException>()

                 .Or<MongoClientException>()

                 .Or<MongoConnectionException>()

                  .Retry(_maxRetryAttempts, (ex, retryCount) =>

            {

                _logger.Error(Environment.NewLine +

                string.Format("Exception caught on attempt {0} - will retry after delay {1}"

                ,retryCount, _pauseBetweenFailures), ex);  

              });

        }

      public IEnumerable<T> FindAll(TId id)
        {
            return PollyHelper.DbRetryPolicy()
                       .Execute(() =>
                        Collection.FindSync(f => f.Id.Equals(id)).ToEnumerable());
        }

 2. Http retry with policy :- If status code is not success, I have defined retry policy for http Response message.

      public static void ApiRetryPolicy()
        {
                return Policy.Handle<Exception>()
                      .OrResult<HttpResponseMessage>( r=> !r.IsSuccessStatusCode)
                      .Retry(_maxRetryAttempts, (ex, retryCount) =>
            {
                _logger.Error(Environment.NewLine +
                string.Format(" Exception caught on attempt {0} - will retry after delay {1}"
                ,retryCount, _pauseBetweenFailures), ex.Exception);           
            });       
        } 

       public async Task Get()
        {
              return await PollyHelper.ApiRetryPolicy()
                         .ExecuteAsync(() => client.GetAsync("client "));           
        }

Convert a Standalone Mongodb instance to a Replica Set on a single machine with multiple instances

This tutorial describes the process for converting a standalone instance into a replica set.

In this scenario, there is only a single machine, but we run multiple (for this example, two) mongod instances. One instance acts as PRIMARY and other instances act as SECONDARY.

Steps:- 
1. Shut down the standalone mongod instance.
2. Restart the instance. Use the --replSet option to specify the name of the new replica set.

For example, use the following command to create a new replica set 

mongod --dbpath c:\replica\rs1\data --port 27017 --dbpath c:\replica\rs1\log\mongodb.log --replSet myset

3. Connect a mongo shell to the mongod instance.

4. Use rs.initiate() to initiate the new replica set:

Start Secondary mongod instance.

We should use unique set of port and data path for each instance, otherwise we may get port binding error and data path issues. For this instance we shall use 27018 port and /replica/rs2/data data path.

mongod --dbpath c:\replica\rs2\data --port 27018 --dbpath c:\replica\rs2\log\mongodb.log --replSet myset

Add a MongoDB instance to the Replica Set.

To add the MongoDB instances, run the following command in the mongo shell

rs.add(<hostname:port>);

rs.add("localhost:27018")

Check the Status.

You may check the status of the Replica Set by running the following command

rs.status();

Now there are two members in the Replica Set, with localhost:27017 being PRIMARY and localhost:27018 being SECONDARY.

Check Replication.

Now we can check if the replication is happening correctly. Add document in primary instance, it should reflect in all replica set 

myset:PRIMARY> use Cars

switched to db Cars

myset:PRIMARY> db.Car.insertOne({ name: "Z4", brand: "BMW"})

myset:PRIMARY> 

Now check in SECONDARY instance, if this has replicated. To connect to MongoDB instance running at 27018, run the following command

mongo --port 27018

myset:SECONDARY> use Cars

switched to db Cars

myset:SECONDARY> db.Car.find();

error: { "$err" : "not master and slaveOk=false", "code" : 13435 }

myset:SECONDARY> 

Set "slave okay" mode to let the mongo shell know that We're allowing reads from a secondary. This is to protect applications from performing eventually consistent reads by accident. 

We can do this in the shell with:

rs.slaveOk()

Now find the document in secondary instance.

myset:SECONDARY> use Cars

switched to db fruits

myset:SECONDARY> db.Car.find();

{ "_id" : ObjectId("5a018ea7c89da78ba2076f25"), "name" : "Z4", "brand" : "BMW" }

myset:SECONDARY> 

Same we can do with another instance. 

Why you should use ConfigureAwait(false) in library code

Await task uses the sync context

1. it captures the current syncContext before awaiting.
2. Upon task completing, it calls syncContext.Post() to resume "Where you were before"


Problem :- 
1. DeadLock

The root cause of this deadlock is due to the way await handles contexts. By default, when an incomplete Task is awaited, the current “context” is captured and used to resume the method when the Task completes. This “context” is the current SynchronizationContext unless it’s null, in which case it’s the current TaskScheduler. GUI and ASP.NET applications have a SynchronizationContext that permits only one chunk of code to run at a time. When the await completes, it attempts to execute the remainder of the async method within the captured context. But that context already has a thread in it, which is (synchronously) waiting for the async method to complete. They’re each waiting for the other, causing a deadlock

  public static class DeadlockExample
    {
        private static async Task RunAsync()
        {
            await Task.Delay(1000);
        }
        // This method causes a deadlock when called in a GUI or ASP.NET context.
        public static void Test()
        {
            // Start the delay.
            var task = Runsync();
            // Wait for the delay to complete.
            task.Wait();
        }

    }

Solution :-  we can use "await task.configurationAwait(false).

  public ConfiguredTaskAwaitable ConfigureAwait(bool continueOnCapturedContext)
    {
      return new ConfiguredTaskAwaitable(this, continueOnCapturedContext);
    }

This suppresses step 2, instead if it resumes "on thread that completed  the tasks"

Principles :- UI messgage-queue is an app global resource. To much use will hurt UI responsiveness.

Guidance:- If your method calls chatty async apis. but doesn't touch the UI, then use configurationAwait(false)

2. Performance :- By using ConfigureAwait, This enable small amount of parallelism: Some asynchronous code can run in parallel with the GUI thread instead of constantly badgering it with bits of work to do.

Mongodb 1067 error : Service not starting

The Mongo DB service is starting.
The Mongo DB service could not be started.

A system error has occurred.

System error 1067 has occurred.

The process terminated unexpectedly.

Some blog suggest this error occure due to limitation disk space.

The Solution :- 

The error came from an unclean shutdown detected.

Below steps will fix it

1) Remove the file /data/db/mongod.lock

2) Run mongod.exe --repair

3) Start the mongod service net start MongoDB

Please visit http://dochub.mongodb.org/core/repair for complete recovery instructions.