I'm creating a DocumentDB (MongoDB) cluster in AWS using Cloudformation. The documentation says that the DocumentDB instances WILL be spread across the subnet groups, in different AvailabilityZones (AZ), as defined here:
<https://docs.aws.amazon.com/documentdb/latest/developerguide/replication.html>

Excerpt:

> 
> "When the first instance in the cluster is created, it is the primary
> instance and is located in one of the Availability Zones. In this
> example, it's in AZ1. The second instance created is a replica
> instance and is located in one of the other two Availability Zones,
> say AZ2. The third instance created is a replica instance and is
> located in the remaining Availability Zone, AZ3. If you create more
> instances, they are distributed across the Availability Zones so that
> you achieve balance in the cluster."
> 
> 
> 

But I've done exactly that, and yet, it created two replica instances in the SAME AZ!

[![enter image description here](https://i.stack.imgur.com/wmzgi.png)](https://i.stack.imgur.com/wmzgi.png)

What's going on here? Why did AWS create my two READ REPLICAS in the same AZ? Shouldn't the last one have gone to us-east-1b ?!?!

* is there a race condition in Cloudformation? Cloudformation seems to create all three instances in parallel so maybe it doesn't have the smarts to wait until the other instances have been assigned to an AZ before determining which AZ each instance should go to? For example: maybe all three make a request to see what instances already exist and in which AZ they are located and then two notice that there's nothing in us-east-1a and both decide to create the instance there?
* should I create a "dependsOn" to avoid the race condition? I.e. instance2 depends on instance1, and instance3 depends on instance2? Ha, I'll try that and report back :)

See my Cloudformation template below. Note that the VPC, public and private subnets are defined in another Cloudformation template/stack but it's the usual VPC with 3 public and 3 private subnets, one NAT+EIP per AZ, and all the routes that go with it (outside the scope of this question). I just import the values exported from that other stack (VPC id, VPC CIDR, and subnets):

```
AWSTemplateFormatVersion: "2010-09-09"

Description: DocumentDB/MongoDB database server in the VPC, in all private subnets. And the database master username and password in secrets manager.

Parameters:
  MongoDBInstanceIdentifier:
    Description: The mongodb database identifier (ex dev, prod, etc).
    Type: String

Resources:
  # Create the MongoDB Master username password Secret with a randomly generated password
  MongoDBSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub 'docdb/masterpwd/${MongoDBInstanceIdentifier}'
      Description: !Sub 'MongoDB ${MongoDBInstanceIdentifier} master user credentials'
      GenerateSecretString:
        SecretStringTemplate: '{"username": "admin"}'
        GenerateStringKey: "password"
        PasswordLength: 16
        ExcludePunctuation: true
        ExcludeCharacters: "\"@/\\"
      Tags:
        - Key: Name
          Value: !Sub 'MongoDB ${MongoDBInstanceIdentifier} master user password secret'

  MongoDBSubnetGroup:
    Type: AWS::DocDB::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: MongoDB subnet group
      DBSubnetGroupName: !Sub "mongo-subnet-group-${MongoDBInstanceIdentifier}"
      SubnetIds:
        - !ImportValue PrivateSubnet1
        - !ImportValue PrivateSubnet2
        - !ImportValue PrivateSubnet3

  MongoDBSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: DocumentDB Security Group
      GroupName: !Sub docdb-sg-${MongoDBInstanceIdentifier}
      VpcId: !ImportValue VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          CidrIp: !ImportValue VpcCIDR
          FromPort: 27017
          ToPort: 27017
      Tags:
        - Key: Name
          Value: !Sub "MongoDB security group ${MongoDBInstanceIdentifier}"

  MongoDBCluster:
    Type: AWS::DocDB::DBCluster
    Properties:
      DBClusterIdentifier: !Sub "mongo-cluster-${MongoDBInstanceIdentifier}"
      DBClusterParameterGroupName: !Ref MongoDBClusterParameterGroup
      DeletionProtection: true
      BackupRetentionPeriod : 1
      DBSubnetGroupName : !Ref MongoDBSubnetGroup
      AvailabilityZones:
        - us-east-1a
        - us-east-1b
        - us-east-1c
      MasterUsername:
        Fn::Sub: "{{resolve:secretsmanager:${MongoDBSecret}::username}}"
      MasterUserPassword:
        Fn::Sub: "{{resolve:secretsmanager:${MongoDBSecret}::password}}"
      Port : 27017
      EngineVersion: 4.0.0
      PreferredBackupWindow : "23:00-23:59"
      PreferredMaintenanceWindow : "sun:00:00-sun:05:00"
      VpcSecurityGroupIds:
        - !Ref MongoDBSecurityGroup
      StorageEncrypted : true
      Tags:
        - Key: Name
          Value: !Sub "MongoDB cluster ${MongoDBInstanceIdentifier}"

  MongoDBClusterParameterGroup:
    Type: AWS::DocDB::DBClusterParameterGroup
    Properties:
      Description: !Sub "MongoDB cluster ${MongoDBInstanceIdentifier}"
      Family: "docdb4.0"
      Name: !Sub "mongo-cluster-params-${MongoDBInstanceIdentifier}"
      Parameters:
        audit_logs: "disabled"
        tls: "disabled"
        ttl_monitor: "enabled"
      Tags:
        - Key: Name
          Value: !Sub "MongoDB cluster parameters ${MongoDBInstanceIdentifier}"

  MongoDBInstance1:
    Type: AWS::DocDB::DBInstance
    Properties:
      DBInstanceClass: "db.r6g.large"
      DBClusterIdentifier: !Ref MongoDBCluster
      DBInstanceIdentifier: !Sub "mongodb-instance1-${MongoDBInstanceIdentifier}"
      PreferredMaintenanceWindow: "sun:00:00-sun:05:00"
      Tags:
        - Key: Name
          Value: !Sub "MongoDB Instance1 ${MongoDBInstanceIdentifier}"

  MongoDBInstance2:
    Type: AWS::DocDB::DBInstance
    Properties:
      DBInstanceClass: "db.r6g.large"
      DBClusterIdentifier: !Ref MongoDBCluster
      DBInstanceIdentifier: !Sub "mongodb-instance2-${MongoDBInstanceIdentifier}"
      PreferredMaintenanceWindow: "sun:00:00-sun:05:00"
      Tags:
        - Key: Name
          Value: !Sub "MongoDB Instance2 ${MongoDBInstanceIdentifier}"

  MongoDBInstance3:
    Type: AWS::DocDB::DBInstance
    Properties:
      DBInstanceClass: "db.r6g.large"
      DBClusterIdentifier: !Ref MongoDBCluster
      DBInstanceIdentifier: !Sub "mongodb-instance3-${MongoDBInstanceIdentifier}"
      PreferredMaintenanceWindow: "sun:00:00-sun:05:00"
      Tags:
        - Key: Name
          Value: !Sub "MongoDB Instance3 ${MongoDBInstanceIdentifier}"

Outputs:
  MongoDBClusterId:
    Description: DocumentDB/MongoDB cluster ID
    Value: !Ref MongoDBCluster
    Export:
      Name: !Sub "documentdb-cluster-id-${MongoDBInstanceIdentifier}"
  MongoDBClusterEndpoint:
    Description: DocumentDB/MongoDB cluster endpoint
    Value: !GetAtt MongoDBCluster.Endpoint
    Export:
      Name: !Sub "documentdb-cluster-endpoint-${MongoDBInstanceIdentifier}"
  MongoDBClusterReadEndpoint:
    Description: DocumentDB/MongoDB cluster READ endpoint
    Value: !GetAtt MongoDBCluster.ReadEndpoint
    Export:
      Name: !Sub "documentdb-cluster-read-endpoint-${MongoDBInstanceIdentifier}"
  MongoDBClusterPort:
    Description: DocumentDB/MongoDB cluster port
    Value: !GetAtt MongoDBCluster.Port
    Export:
      Name: !Sub "documentdb-cluster-port-${MongoDBInstanceIdentifier}"
  MongoDBSecret:
    Description: DocumentDB/MongoDB credentials secret ARN
    Value: !Ref MongoDBSecret
    Export:
      Name: !Sub "documentdb-${MongoDBInstanceIdentifier}-SecretARN"

```