title: aws s3a read is ok but facing errors with write operation
body: <p>I am using spark 2.4 and I am running spark app in ec2. I am able to read but not write to s3. </p>
  
  <p>I have <code>/opt/spark/jars/aws-java-sdk-1.7.4.jar</code> and <code>/opt/spark/jars/hadoop-aws-2.7.3.jar</code> in classpath.
  I am able to read s3 files but write gives AWS Error Message:</p>
  
  <pre><code>Forbidden, S3 Extended Request ID: XXXXXXXXX
  </code></pre>
  
  <p>I got the same error too for reading, then I removed access keys from spark properties and managed s3 access via roles and it worked. S3 Policy gives full permission on bucket--* </p>
  
  <pre><code>ppl10.write.format("csv").mode("Overwrite").save("s3a://mybucket/data/proto_ds_out")
  </code></pre>
  
  <p>Detail error:</p>
  
  <pre><code>Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: 
  Status Code: 403, 
  AWS Service: Amazon S3, 
  AWS Request ID: 5E4B3CE8F0940E35, 
  AWS Error Code: null, 
  AWS Error Message: Forbidden, 
  S3 Extended Request ID: Mq6rwJGff8UONQHsBU+mlIggUCUyUYuzYOtyCj7o/7icK6FDw/y9plPkHgwpBUbedUwFIE0eTsU=
  </code></pre>