title: spark resolve external packages behind corporate artifactory
body: <p>I want to resolve packages in spark when behind a corporate Artifactory:</p>
  <pre><code>spark-shell --repositories https://bin.mycorp.com/artifactory/maven/ --packages io.delta:delta-core_2.12:0.7.0
  </code></pre>
  <p>simply doesn't resolve for me. In the logs I find:</p>
  <pre><code>found io.delta#delta-core_2.12;0.7.0 in repo-1
  found org.antlr#antlr4;4.7 in repo-1 https://bin.mycorp.com/artifactory/maven/ --packages io.delta:delta-core_2.12:0.7.0                                                                      [https://found org.antlr#antlr4-runtime;4.7 in repo-1a remote repository with the name: repo-1
  Ivy Default Cache set to: ~/.ivy2/cache
  The jars for the packages stored in: ~/.ivy2/jars
  :: loading settings :: url = jar:file:/usr/hdp/3.1.x.x-xxx/spark2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
  io.delta#delta-core_2.12 added as a dependency
  :: resolving dependencies :: org.apache.spark#spark-submit-parent-ae4e5fbb-afc7-4c44-8262-b5c40ee2a1d5;1.0
          confs: [default]
  </code></pre>
  <p>no output.</p>
  <p>The ivy settings from the JAR (<code>/usr/hdp/3.1.4.0-315/spark2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml</code>) are:</p>
  <pre><code>&lt;ivysettings&gt;
          &lt;settings defaultResolver=&quot;default&quot;/&gt;
          &lt;include url=&quot;${ivy.default.settings.dir}/ivysettings-public.xml&quot;/&gt;
          &lt;include url=&quot;${ivy.default.settings.dir}/ivysettings-shared.xml&quot;/&gt;
          &lt;include url=&quot;${ivy.default.settings.dir}/ivysettings-local.xml&quot;/&gt;
          &lt;include url=&quot;${ivy.default.settings.dir}/ivysettings-main-chain.xml&quot;/&gt;
          &lt;include url=&quot;${ivy.default.settings.dir}/ivysettings-default-chain.xml&quot;/&gt;
  &lt;/ivysettings&gt;
  </code></pre>