title: IntelliJ: setting AWS env vars for gradle run config
body: <p>I'm trying to set up my local dev env for a Java Spring Boot configured with gradle which needs a working AWS CLI environment. It's important to note that there are different profiles (<code>.aws/config</code>) present (<code>default</code> not configured) which are different role switches (<code>role_arn</code>) with MFA (<code>mfa_serial</code>).</p>
  <hr />
  <h2>Description</h2>
  <p>At the moment I'm using Windows 10 Build 18363 (&quot;19.09&quot;), IntelliJ Ultimate 20.02, gradle 5.6.4.
  In order to run this application I've set up a run configuration with the gradle <code>clean bootRun</code> task and the following env vars:</p>
  <ul>
  <li><code>AWS_SECRET_ACCESS_KEY</code></li>
  <li><code>AWS_ACCESS_KEY_ID</code></li>
  <li><code>AWS_REGION</code></li>
  <li><code>AWS_DEFAULT_REGION</code></li>
  <li><code>AWS_PROFILE</code></li>
  <li><code>AWS_DEFAULT_PROFILE</code></li>
  </ul>
  <pre><code>#.aws/config
  [profile prod]
  region=eu-central-1
  output=json
  role_arn=&lt;role_arn_prod&gt;
  source_profile=site-iam
  mfa_serial=&lt;arn_iam_user_mfa_1&gt;
  
  
  [profile sit]
  region=eu-central-1
  output=json
  role_arn=&lt;role_arn_sit&gt;
  source_profile=site-iam
  mfa_serial=&lt;arn_iam_user_mfa_1&gt;
  
  [profile dev]
  region=eu-central-1
  output=json
  role_arn=&lt;role_arn_dev&gt;
  source_profile=site-iam
  mfa_serial=&lt;arn_iam_user_mfa_1&gt;
  
  [profile site-iam]
  region=eu-central-1
  output=json
  
  [default]
  
  </code></pre>
  <pre><code>#.aws/credentials
  [default]
  aws_access_key_id = &lt;access_key_id_1&gt;
  aws_secret_access_key = &lt;secret_access_key_1&gt;
  
  [site-iam]
  aws_access_key_id = &lt;access_key_id_1&gt;
  aws_secret_access_key = &lt;secret_access_key_1&gt;
  </code></pre>
  <hr />
  <h2>Behaviour</h2>
  <p>The request inside of this java application is quite simple.</p>
  <pre class="lang-java prettyprint-override"><code>GetObjectRequest getObjectRequest = GetObjectRequest.builder()
      .bucket(this.bucket)
      .key(this.key)
      .build();
  String response = this
      .s3Client
      .getObjectAsBytes(getObjectRequest).asString(Charset.forName(StandardCharsets.UTF_8.name()));
  </code></pre>
  <p>The error below suggests that there was no role switch and / or the authentication failed (because we need to submit the TOTP in order to authenticate.</p>
  <pre><code>Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403, Request ID: &lt;REQUEST_ID&gt;)
      at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:123) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:79) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:40) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:189) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:121) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:147) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52) ~[sdk-core-2.13.13.jar:na]
      at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:62) ~[aws-core-2.13.13.jar:na]
      at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:3606) ~[s3-2.13.13.jar:na]
      at software.amazon.awssdk.services.s3.S3Client.getObjectAsBytes(S3Client.java:7563) ~[s3-2.13.13.jar:na]
  </code></pre>
  <p>A similar behaviour is reproduciblevia the cmd</p>
  <pre><code>aws s3 ls
  
  An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied
  </code></pre>
  <hr />
  <h2>Different approaches</h2>
  <h3>STS get-session-token | env var</h3>
  <p>I've obtained the session token  via</p>
  <pre class="lang-sh prettyprint-override"><code>aws sts get-session-token --serial &lt;arn_iam_user_mfa_1&gt; --token-code &lt;TOTP&gt; --duration-seconds 129600
  </code></pre>
  <p>as json</p>
  <pre><code>{
      &quot;Credentials&quot;: {
          &quot;AccessKeyId&quot;: &quot;&lt;access_key_id_2&gt;&quot;,
          &quot;SecretAccessKey&quot;: &quot;&lt;secret_access_key_2&gt;&quot;,
          &quot;SessionToken&quot;: &quot;&lt;session_token_2&gt;&quot;,
          &quot;Expiration&quot;: &quot;2020-08-02T22:57:12+00:00&quot;
      }
  }
  </code></pre>
  <p>and set updated the following env vars to the obtained values:</p>
  <ul>
  <li><code>AWS_ACCESS_KEY_ID</code>: <code>&lt;access_key_id_2&gt;</code></li>
  <li><code>AWS_SECRET_ACCESS_KEY</code>: <code>&lt;secret_access_key_2&gt;</code></li>
  <li><code>AWS_SESSION_TOKEN</code>: <code>&lt;session_token_2&gt;</code></li>
  <li><code>AWS_SECURITY_TOKEN</code>: <code>&lt;session_token_2&gt;</code></li>
  </ul>
  <p>but this is returning the same error</p>
  <pre><code>Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403, Request ID: &lt;request_id&gt;)
  </code></pre>
  <hr />
  <h3>Quick and dirty <code>.aws/config</code> fix</h3>
  <p>After several time of debugging the <code>default</code> profile was changed in the following way</p>
  <pre><code>[default]
  region=eu-central-1
  output=json
  role_arn=&lt;role_arn_prod&gt;
  source_profile=site-iam
  mfa_serial=&lt;arn_iam_user_mfa_1&gt;
  </code></pre>
  <p>which resulted in</p>
  <pre><code>Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): To use assumed roles in the '&lt;ROLE_PROFILE&gt;' profile, the 'sts' service module must be on the class path., ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.]
  </code></pre>
  <hr />
  <h3>Setting <code>AWS_WEB_IDENTITY_TOKEN_FILE</code></h3>
  <p>The <code>AWS_WEB_IDENTITY_TOKEN_FILE</code> env var was set to an existing token file located in <code>~/.aws/cli/cache</code> via a WSL sytle path (<code>\wsl$\...</code>)
  which leads to the same error as above even though the the env vars are present and / or updated</p>
  <pre><code>Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): To use assumed roles in the '&lt;ROLE_PROFILE&gt;' profile, the 'sts' service module must be on the class path., ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.]
  </code></pre>
  <hr />
  <h2>Debugging</h2>
  <p>There are no permission problems.
  The download of this specific file works with the correct role arn.</p>
  <pre><code>
  C:\Users\tunnelblick&gt;aws s3 ls
  
  An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied
  
  C:\Users\tunnelblick&gt;aws s3 ls --profile prod
  Enter MFA code for &lt;arn_iam_user_mfa_1&gt;:
  2020-02-05 09:28:41 &lt;bucket_1&gt;
  2020-06-23 05:16:07 &lt;bucket_2&gt;
  
  C:\Users\tunnelblick&gt;aws s3 cp s3://&lt;bucket_1&gt;/&lt;file&gt;.json .
  fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
  
  C:\Users\tunnelblick&gt;aws s3 cp s3://&lt;bucket_1&gt;/&lt;file&gt;.json . --profile prod
  download: s3://&lt;bucket_1&gt;/&lt;file&gt;.json to .\&lt;file&gt;.json
  </code></pre>
  <p>Setting these env vars in my WSL 1 terminal and executing <code>aws s3 ls</code> resulted in the same <code>Access Denied</code> error
  (<code>An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied</code>)</p>