title: Loading data from GCS using Spark Local
body: <p>I am trying to read data from GCS buckets on my local machine, for testing purposes. I would like to sample some of the data in the cloud
  I have downloaded the <a href="https://cloud.google.com/dataproc/docs/concepts/connectors/install-storage-connector" rel="nofollow noreferrer">GCS Hadoop Connector JAR</a>.</p>
  
  <p>And setup the <code>sparkConf</code> as follow:</p>
  
  <pre class="lang-py prettyprint-override"><code>conf = SparkConf() \
      .setMaster("local[8]") \
      .setAppName("Test") \
      .set("spark.jars", "path/gcs-connector-hadoop2-latest.jar") \
      .set("spark.hadoop.google.cloud.auth.service.account.enable", "true") \
      .set("spark.hadoop.google.cloud.auth.service.account.json.keyfile", "path/to/keyfile")
  
  sc = SparkContext(conf=conf)
  
  spark = SparkSession.builder \
      .config(conf=sc.getConf()) \
      .getOrCreate()
  
  spark.read.json("gs://gcs-bucket")
  </code></pre>
  
  <p>I have also tried to set the conf like so:</p>
  
  <pre class="lang-py prettyprint-override"><code>sc._jsc.hadoopConfiguration().set("fs.AbstractFileSystem.gs.impl",  "com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS")
  sc._jsc.hadoopConfiguration().set("fs.gs.auth.service.account.json.keyfile", "path/to/keyfile")
  sc._jsc.hadoopConfiguration().set("fs.gs.auth.service.account.enable", "true")
  </code></pre>
  
  <p>I am using PySpark install via PIP and running the code using the unit test module from IntelliJ</p>
  
  <pre><code>py4j.protocol.Py4JJavaError: An error occurred while calling o128.json.
  : java.io.IOException: No FileSystem for scheme: gs
  </code></pre>
  
  <p>What should I do?</p>
  
  <p>Thanks!</p>