title: Obtaining Invalid S3 URI error when reading from Redshift
body: <p>I am trying to read data from a Redshift table into a Spark 2.0 dataframe. 
  My call looks like this:</p>
  
  <pre><code> df = spark.read \
   .format("com.databricks.spark.redshift") \
   .option("url", "jdbc:redshift://hostname:5439/dbname?user=myuser&amp;password=pwd&amp;ssl=true&amp;sslfactory=com.amazon.redshift.ssl.NonValidatingFactory") \
   .option("dbtable", "myschema.mytable") \
   .option('forward_spark_s3_credentials',"true") \
   .option("tempdir", "s3a://mybucket/tmp2") \
   .option("region", "us-east-1") \
   .load()
  </code></pre>
  
  <p>This returns ok without errors.
  However, when I run </p>
  
  <pre><code>df.collect()
  </code></pre>
  
  <p>I obtain the error below:</p>
  
  <pre><code>17/02/07 17:37:36 WARN Utils$: An error occurred while trying to read 
  the S3 bucket lifecycle configuration
  java.lang.IllegalArgumentException: Invalid S3 URI: hostname does not 
  appear to be a valid S3 endpoint: s3://mybucket/tmp2
  at com.amazonaws.services.s3.AmazonS3URI.&lt;init&gt;(AmazonS3URI.java:65)
  at com.amazonaws.services.s3.AmazonS3URI.&lt;init&gt;(AmazonS3URI.java:42)
  at com.databricks.spark.redshift.Utils$.checkThatBucketHasObjectLifecycleConfiguration(Utils.scala:72)
  at com.databricks.spark.redshift.RedshiftRelation.buildScan(RedshiftRelation.scala:76)
  at org.apache.spark.sql.execution.datasources.DataSourceStrategy$anonfun$11.apply(DataSourceStrategy.scala:336)
  at org.apache.spark.sql.execution.datasources.DataSourceStrategy$anonfun$11.apply(DataSourceStrategy.scala:336)
  at org.apache.spark.sql.execution.datasources.DataSourceStrategy$anonfun$pruneFilterProject$1.apply(DataSourceStrategy.scala:384)
  at ...
  </code></pre>
  
  <p>The data is then subsequently returned...</p>
  
  <pre><code>Out[2]: [Row(col1=1, col2=u'yyyyy', col3=datetime.date(2015, 1, 6), col4=datetime.date(2017, 1, 6), col5=Decimal('21'), col6=u'ABCDEF',...)]
  </code></pre>
  
  <p>Points to note:</p>
  
  <ul>
  <li>This error occurs for both spark-submit and pyspark </li>
  <li><p>The version of
  Spark is 2.1 and the jars directory contains these pertinent files:</p>
  
  <p><em>RedshiftJDBC4-1.2.1.1001.jar</em></p>
  
  <p><em>aws-java-sdk-1.7.4.jar</em></p>
  
  <p><em>spark-redshift_2.11-0.5.0.jar</em></p>
  
  <p><em>hadoop-aws-2.7.3.jar</em></p></li>
  </ul>
  
  <p>I have tried other combinations esp of the aws-java but in such cases, I don't even get the dataframe to return. I get an an error from the spark.read call.</p>
  
  <ul>
  <li>The tmp2 bucket directory in S3 exists and is written to with a split
  file containing the results from Redshift. </li>
  <li>This is being run under a federated login and there is no need to supply credentials
  explicitly.</li>
  </ul>
  
  <p>Any help/suggestions would be greatly appreciated.</p>