title: Docker daemon crashes on EC2 instance
body: <p>So, I'm updating a cloudformation template so that it uses the Amazon Linux 2 AMI optimized for ECS [amzn2-ami-ecs-hvm-2.0.20190709-x86_64-ebs (ami-0fac5486e4cff37f4)]. Previously I was using ami-00129b193dc81bc31 which is an Amazon Linux 1 container.</p>
  
  <p>I initially found that simply changing out the AMI meant the EC2 instances no longer joined my ECS cluster. After a lot of digging (ensuring permissions, subnet, VPC, IAM were all right) I found that the docker daemon was crashing. The reason was traceable to what I was doing with the user data. I'll paste it below:</p>
  
  <pre><code>        "UserData": { "Fn::Base64" : { "Fn::Join" : ["", [
            "Content-Type: multipart/mixed; boundary=\"==BOUNDARY==\"\n",
            "MIME-Version: 1.0\n",
            "--==BOUNDARY==\n",
            "Content-Type: text/cloud-boothook; charset=\"us-ascii\"\n",
            "#!/bin/bash\n",
            "# Set Docker daemon options\n",
            "cloud-init-per once docker_debug echo 'OPTIONS=\"${OPTIONS} --storage-opt dm.basesize=10G\"' &gt;&gt; /etc/sysconfig/docker\n",
            "--==BOUNDARY==\n",
            "Content-Type: text/x-shellscript; charset=\"us-ascii\"\n",
            "#!/bin/bash -xe\n",
            "echo \"ECS_CLUSTER=",
            { "Ref" : "NovaProductionEcsCluster"},
            "\" &gt;&gt; /etc/ecs/ecs.config\n",
            "sudo mkdir /efs\n",
            "sudo mkdir /efs/nova_files\n",
            "sudo useradd -u 33 www-data\n",
            "sudo chown -R www-data /efs/nova_files/\n",
            "printf \"\nfunction novarun {\n docker exec -ti \\\"\\$(docker ps -qf name=ApacheTask)\\\" \\\"\\$([ \\$# -ne 1 ] &amp;&amp; echo \\\"bash\\\" || echo \\\"\\$1\\\")\\\" \n}\n\" &gt;&gt; /home/ec2-user/.bashrc\n",
            "--==BOUNDARY==--"
          ]]}}
        }
  </code></pre>
  
  <p>Basically, this combines this (<a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bootstrap_container_instance.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bootstrap_container_instance.html</a>) with this (<a href="https://aws.amazon.com/premiumsupport/knowledge-center/increase-default-ecs-docker-limit/" rel="nofollow noreferrer">https://aws.amazon.com/premiumsupport/knowledge-center/increase-default-ecs-docker-limit/</a>) to increase the disk space available to the docker container. This works perfectly with the old AMI, but the new AMI gives me <code>Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?</code> when I run <code>docker ps -a</code>.</p>
  
  <p>If I run <code>sudo journalctl -u docker</code> to get the error <code>Error starting daemon: error initializing graphdriver: overlay2: unknown option dm.basesize</code>.</p>
  
  <p>Can anyone tell me what's going wrong with this approach? Is there a way to fix this flag with ECS, or a different way to enlarge the amount of disk space available to the docker image?</p>
  
  <p>Thanks</p>