title: Configure standalone spark for azure storage access
body: <p>I have a need to be able to run spark on my local machine to access azure wasb and adl urls, but I can't get it to work.  I have a stripped down example here:</p>
  
  <p>maven pom.xml (Brand-new pom, only the dependencies have been set):</p>
  
  <pre><code>&lt;dependencies&gt;
  &lt;dependency&gt;
    &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
    &lt;artifactId&gt;spark-core_2.11&lt;/artifactId&gt;
    &lt;version&gt;2.3.0&lt;/version&gt;
  &lt;/dependency&gt;
    &lt;dependency&gt;
        &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
        &lt;artifactId&gt;spark-sql_2.11&lt;/artifactId&gt;
        &lt;version&gt;2.3.0&lt;/version&gt;
    &lt;/dependency&gt;
  &lt;dependency&gt;
    &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
    &lt;artifactId&gt;hadoop-common&lt;/artifactId&gt;
    &lt;version&gt;2.8.0&lt;/version&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
    &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
    &lt;artifactId&gt;hadoop-azure-datalake&lt;/artifactId&gt;
    &lt;version&gt;3.1.0&lt;/version&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
    &lt;groupId&gt;com.microsoft.azure&lt;/groupId&gt;
    &lt;artifactId&gt;azure-storage&lt;/artifactId&gt;
    &lt;version&gt;6.0.0&lt;/version&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
    &lt;groupId&gt;com.microsoft.azure&lt;/groupId&gt;
    &lt;artifactId&gt;azure-data-lake-store-sdk&lt;/artifactId&gt;
    &lt;version&gt;2.2.3&lt;/version&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
    &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
    &lt;artifactId&gt;hadoop-azure&lt;/artifactId&gt;
    &lt;version&gt;3.1.0&lt;/version&gt;
  &lt;/dependency&gt;
  &lt;dependency&gt;
    &lt;groupId&gt;com.microsoft.azure&lt;/groupId&gt;
    &lt;artifactId&gt;azure-storage&lt;/artifactId&gt;
    &lt;version&gt;7.0.0&lt;/version&gt;
  &lt;/dependency&gt;
  </code></pre>
  
  <p></p>
  
  <p>Java code (Doesn't need to be java - could be scala):</p>
  
  <pre><code>import org.apache.spark.SparkConf;
  import org.apache.spark.SparkContext;
  import org.apache.spark.sql.SparkSession;
  
  public class App {
      public static void main(String[] args) {
          SparkConf config = new SparkConf();
          config.setMaster("local");
          config.setAppName("app");
          SparkSession spark = new SparkSession(new SparkContext(config));
          spark.read().parquet("wasb://container@host/path");
          spark.read().parquet("adl://host/path");
      }
  }
  </code></pre>
  
  <p>No matter what I try I get:</p>
  
  <pre><code>Exception in thread "main" java.io.IOException: No FileSystem for scheme: wasb
  </code></pre>
  
  <p>Same for adl.  Every doc I can find on this either just says to add the azure-storage dependency, which I have done, or says to use HDInsight.</p>
  
  <p>Any thoughts?</p>