title: PySpark Writing DataFrame Partitions to S3
body: <p>I've been trying to partition and write a spark dataframe to S3 and I get an error.</p>
  
  <pre><code>df.write.partitionBy("year","month").mode("append")\
      .parquet('s3a://bucket_name/test_folder/')
  </code></pre>
  
  <p>Error message is:</p>
  
  <pre><code>Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: 
  Status Code: 403, AWS Service: Amazon S3, AWS Request ID: xxxxxx, 
  AWS Error Code: SignatureDoesNotMatch, 
  AWS Error Message: The request signature we calculated does not match the signature you provided. Check your key and signing method.
  </code></pre>
  
  <p>However, when I simply write without partitioning <strong>it does work</strong>.</p>
  
  <pre><code>df.write.mode("append").parquet('s3a://bucket_name/test_folder/')
  </code></pre>
  
  <p>What could be causing this problem?</p>