title: NativeAzureFileSystem not recognizing other containers
body: <p>My objective is to access, from spark-shell of an HD Insight instance, blobs that are located in a container inside the storage account over which the cluster was created.</p>
  
  <p>These are the steps I took:</p>
  
  <ol>
  <li>Created an HD Insight cluster over the container <a href="https://mystorage.blob.core.windows.net:443/" rel="nofollow noreferrer">https://mystorage.blob.core.windows.net:443/</a><strong>maincontainer</strong>.</li>
  <li>Created another container on the same storage account: <a href="https://mystorage.blob.core.windows.net:443/" rel="nofollow noreferrer">https://mystorage.blob.core.windows.net:443/</a><strong>extracontainer</strong>.</li>
  <li>Created a file named <em>person.json</em> inside the <strong>extracontainer</strong>: <a href="https://mystorage.blob.core.windows.net:443/extracontainer/data/person.json" rel="nofollow noreferrer">https://mystorage.blob.core.windows.net:443/extracontainer/data/person.json</a></li>
  <li>Opened a spark-shell session</li>
  </ol>
  
  <p>Then I executed the following code:</p>
  
  <pre><code>scala&gt; import org.apache.hadoop.fs._
  
  scala&gt; val conf = sc.hadoopConfiguration
  conf: org.apache.hadoop.conf.Configuration = Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml
  
  scala&gt; val fs: FileSystem = FileSystem.newInstance(conf)
  fs: org.apache.hadoop.fs.FileSystem = org.apache.hadoop.fs.azure.NativeAzureFileSystem@417e5282
  
  scala&gt; val files = fs.listFiles(new Path("wasbs://extracontainer@mystorage.blob.core.windows.net/data"), true)
  java.io.FileNotFoundException: Filewasbs://extracontainer@mystorage.blob.core.windows.net/data does not exist.
  </code></pre>
  
  <p>Then I created the same folder and file on the <strong>maincontainer</strong>:
  <a href="https://mystorage.blob.core.windows.net:443/maincontainer/data/person.json" rel="nofollow noreferrer">https://mystorage.blob.core.windows.net:443/maincontainer/data/person.json</a> and I got the following result:</p>
  
  <pre><code>scala&gt; val files = fs.listFiles(new Path("wasbs://extracontainer@mystorage.blob.core.windows.net/data"), true)
  scala&gt; while( files.hasNext() ) { println(files.next().getPath) }
  wasb://maincontainer@mystorage.blob.core.windows.net/data/person.json
  </code></pre>
  
  <p>It shows me the file in the <strong>maincontainer</strong> and not the one in the <strong>extracontainer</strong>.</p>
  
  <p>Does any one knows what's happening?</p>
  
  <p>I also tried creating the FileSystem object using <code>new Configuration()</code> and I got the same behavior.</p>
  
  <p>The correct behavior is obtained when using <code>hadoop fs</code> command line:</p>
  
  <pre><code>&gt; hadoop fs -ls wasbs://extracontainer@mystorage.blob.core.windows.net/data/
  Found 1 item
  -rwxrwxrwx   1        977 2017-02-27 08:46 wasbs://extracontainer@mystorage.blob.core.windows.net/data/person.json
  </code></pre>