title: How to get file from S3 for bootstrapping application using CloudFormation template and without launching EC2 instances with a role?
body: <p>I need to download a file from S3 bucket which is a private location. I have created an IAM Role using AWS Console - hence an have an IAM Profile which I assign to instances launched. I can get the file from secure location by providing an Authentication Object in LaunchConfiguration. This only works if the instance is launched with an <code>IamInstanceProfile</code>. </p>
  
  <pre><code>   "LaunchConfiguration":{
          "Type":"AWS::AutoScaling::LaunchConfiguration",
          "Metadata":{
              "Comment":"Configure the Presentation Tier",
               "AWS::CloudFormation::Authentication": {
                  "S3Access" : {
                      "type" : "S3",
                      "buckets":["demo-upload"],
                      "roleName": "MYS3ROLE"
                  }
              },
              "AWS::CloudFormation::Init":{
                  "config":{
                      "packages":{
                          "yum":{
                              "tomcat6": [],
                              "java-1.6.0-openjdk-devel" : [],
                              "tomcat6-webapps" :[],
                              "tomcat6-admin-webapps" :[],
                              "tomcat6-docs-webapp" :[],
                              "curl":[]
                          }
                      },
                      "files":{
                          "/var/lib/tomcat6/webapps/demo.war" : { 
                              "source" : {"Ref":"WarLoc"},
                              "owner" : "tomcat",
                              "group" : "tomcat",
                              "authentication":"S3Access"
                          },
                          "/root/demo.war" : { 
                              "source" : {"Ref":"WarLoc"},
                              "owner" : "root",
                              "group" : "root",
                              "authentication":"S3Access"
                          },
                          "/etc/cfn/cfn-hup.conf" : {
                              "content" : { "Fn::Join" : ["", [
                              "[main]\n",
                              "stack=", { "Ref" : "AWS::StackId" }, "\n",
                              "region=", { "Ref" : "AWS::Region" }, "\n",
                              "interval=1"
                              ]]},
                          "mode"    : "000400",
                          "owner"   : "root",
                          "group"   : "root"
                          },
  
                        "/etc/cfn/hooks.d/cfn-auto-reloader.conf" : {
                          "content": { "Fn::Join" : ["", [
                            "[cfn-auto-reloader-hook]\n",
                            "triggers=post.update\n",
                            "path=Resources.LaunchConfiguration.Metadata.AWS::CloudFormation::Init\n",
                            "action=/opt/aws/bin/cfn-init -s ", { "Ref" : "AWS::StackId" },
                            "         -r LaunchConfiguration ",
                            "         --region     ", { "Ref" : "AWS::Region" }, "\n",
                            "runas=root\n"
                          ]]}
                        }
                      },
                      "services" : {
                          "sysvinit" : {
                              "tomcat6":{
                                  "files":["/var/lib/tomcat6/webapps/demo.war"],
                                  "enabled": "true",
                                  "ensureRunning":"true"
                              },
                              "cfn-hup":{
                                  "files":["/etc/cfn/cfn-hup.conf","/etc/cfn/hooks.d/cfn-auto-reloader.conf"],
                                  "enabled":"true",
                                  "ensureRunning":"true"
                              }
                          }
                      }
                  }
              }
          },
          "Properties":{
              "ImageId"        : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" }, { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
              "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
              "InstanceType"   : { "Ref" : "InstanceType" },
              "InstanceMonitoring": "false",
              "KeyName"        : { "Ref" : "KeyName" },
              "AssociatePublicIpAddress" : "true",
              "IamInstanceProfile":"arn:aws:iam::MY_ACCOUNT_ID:instance-profile/MYS3ROLE",
              "UserData"       : { "Fn::Base64" : { "Fn::Join" : ["", [
                "#!/bin/bash -v\n",
                "yum update -y\n",
                "# Install Presentation Software\n",
                "/opt/aws/bin/cfn-init -s ", { "Ref" : "AWS::StackId" }, " -r LaunchConfiguration ",
                "    --region ", { "Ref" : "AWS::Region" }, "\n",
                "service tomcat6 stop\n",
                "rm -rf /var/lib/tomcat6/webapps/demo\n",
                "sleep 3\n",
                "service tomcat6 start\n",
                "sleep 3\n",
                "service tomcat6 restart\n",
                "/opt/aws/bin/cfn-hup\n",
                "/opt/aws/bin/cfn-signal -e $? -r \"Setup complete\" '", { "Ref" : "WaitHandle" }, "'\n"
                ]]}}
          }
      },
  </code></pre>
  
  <p>Can I download the files from S3 which is not public without having to launch the machine with an <code>IamInstanceProfile</code>. ?</p>
  
  <p>If I delete the the <code>"IamInstanceProfile":"arn:aws:iam::MY_ACCOUNT_ID:instance-profile/s3access",</code> then in <code>/var/log/cfn-init.log</code> I find the following lines, which I don't think would be because of Authentication Faiures</p>
  
  <pre><code>ToolError: Failed to retrieve https://s3-us-west-2.amazonaws.com/demo-upload/0.0.1/demo.war: [Errno 404] HTTP Error 404 : &lt;?xml version="1.0" encoding="iso-8859-1"?&gt;
  &lt;!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
       "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt;
  &lt;html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"&gt;
  &lt;head&gt;
   &lt;title&gt;404 - Not Found&lt;/title&gt;
  &lt;/head&gt;
  &lt;body&gt;
     &lt;h1&gt;404 - Not Found&lt;/h1&gt;
   &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  
  <p>When I launch the Instance with IamProfile everything runs fine. So, is there a way to get the files - without scripts and storing credentials on machine - but not have instance launched with a role ?</p>