Creating and maintaining MongoDB replica sets with pymongo
Question: I am trying to replicate (for a teaching activity) the [Docker and MongoDB
Sharded Cluster](https://sebastianvoss.com/docker-mongodb-sharded-
cluster.html) recipe in an IPython notebook using _pymongo_ to set up several
mongo replica sets.

The recipe suggests creating a replica set from the mongo prompt by connecting
to one member of the proposed replica set as follows:

    
    
    mongo --port <port>
    
    rs.initiate()
    rs.add("<IP_of_rs1_srv2>:27017")
    rs.add("<IP_of_rs1_srv3>:27017")
    rs.status()
    

The [_pymongo_
documentation](http://api.mongodb.org/python/current/examples/high_availability.html)
suggests the following initialisation route:

    
    
    $ hostname
    morton.local
    $ mongod --replSet foo/morton.local:27018,morton.local:27019 --rest
    $ mongod --port 27018 --dbpath /data/db1 --replSet foo/morton.local:27017 --rest
    $ mongod --port 27019 --dbpath /data/db2 --replSet foo/morton.local:27017 --rest
    
    from pymongo import MongoClient, ReadPreference
    c = MongoClient("morton.local:27017", read_preference=ReadPreference.SECONDARY)
    
    c.admin.command("replSetInitiate")
    

This requires knowing the port ids and using them in the initialisation of the
database servers via the `--replSet` flag, rather than just declaring a simple
single label for the replica set and then applying it to each member. How can
I script the initialisation in _pymongo_ to follow the original recipe?

The original also modifies the configuration by changing the hostname to
include the container IP address:

    
    
    cfg = rs.conf()
    cfg.members[0].host = "<IP_of_rs1_srv1>:27017"
    rs.reconfig(cfg)
    rs.status()
    

Again, _pymongo_ doesn't support the `rs` helpers, so how can I update the
replica set configuration?


Answer: One solution to this is to create a configuration for the replica set based on
the attributes of the mongodb containers and then use that to initiate the
cluster set.

So for example, a configuration might take the form:

    
    
    rsc = {'_id': 'rs4',
     'members': [{'_id': 0, 'host': '172.17.0.2:27017'},
      {'_id': 1, 'host': '172.17.0.3:27017'},
      {'_id': 2, 'host': '172.17.0.4:27017'}]}
    

and then be called with the `replSetInitiate` admin command.

    
    
    import docker
    
    #Connect to docker
    c = docker.Client(base_url='unix://var/run/docker.sock',
                      version='1.10',
                      timeout=10)
    

Create a set of database nodes to run as the replica set:

    
    
    def createReplicaSetNode(c,stub,num=0):
        ''' Create and run a specified number of mongo database servers as a replica set '''
        name='{stub}_srv{num}'.format(stub=stub,num=num)
        command='--replSet {stub}  --noprealloc --smallfiles'.format(stub=stub)
        c.create_container('dev24/mongodb',name=name,command=command)
        c.start(name,publish_all_ports=True)
        return name
    
    def createReplicaSetNodes(c,stub,numNodes):
        ''' Create and run a specified number of mongo database servers as a replica set '''
        names=[]
        for i in range(0,numNodes):
            name=createReplicaSetNode(c,stub,i)
            names.append(name)
        return names
    
    
    def getContainIPaddress(c,container):
        ''' Get the IP address of the container '''
        cConfig = c.inspect_container(container)
        return cConfig['NetworkSettings']['IPAddress']
    
    def rs_config(c,rsid,num=3):
        ''' Create a replica set of nodes and then define a configuration file for that replica set '''
        createReplicaSetNodes(c,rsid,num)
        _rs_config={"_id" : rsid, 'members':[] }
        #This is scrappy - should really return something better from the creation
        for i in range(0,num):
            name='{stub}_srv{num}'.format(stub=rsid,num=i)
            #c.inspect_container(name)
            #get IP and port
            _rs_config['members'].append({"_id":i,"host":'{0}:{1}'.format(getContainIPaddress(c,name),27017)})
        return _rs_config
    

We can then start the nodes and create a config file for the replica set:

    
    
    rsc=rs_config(c,'rs4')
    rsc
    
    '''
    {'_id': 'rs4',
     'members': [{'_id': 0, 'host': '172.17.0.2:27017'},
      {'_id': 1, 'host': '172.17.0.3:27017'},
      {'_id': 2, 'host': '172.17.0.4:27017'}]}
    '''
    

This configuration data is used to initialise the replica set:

    
    
    from pymongo import MongoClient
    
    #Find the local port bound for 27017/tcp for each server in the replica set
    def get27017tcp_port(c,container):
        cConfig = c.inspect_container(container)
        return int(cConfig['NetworkSettings']['Ports']['27017/tcp'][0]['HostPort'])
    
    #We'll use the 0th server in the set as a the node
    mc = MongoClient('localhost', get27017tcp_port(c,'rs4_srv0'))
    
    
    #In mongo console, we'd typically use command rs.config() to initialise replica set
    #Here, use replSetInitiate admin command, applying it with desired configuration
    mc.admin.command( "replSetInitiate",rsc);
    

