title: How to start graphframes on spark on pyspark on juypter on docker?
body: <p>Been playing with pyspark on juypter all day with no issues. Just by simply using the docker image <code>juypter/pyspark-notebook</code>, 90% of everything I need is packaged (YAY!)</p>
  <p>I would like to start exploring using GraphFrames, which sits on top of GraphX which sits on top of Spark. Has anyone gotten this combination to work?</p>
  <p>Essentially, according to the documentation, I just need to pass &quot;--packages graphframes:xxyyzz&quot; when running pyspark to download and run graphframes. Problem is that juypter is already running as soon as the container comes up.</p>
  <p>I've tried passing the &quot;--packages&quot; line as an environment variable (-e) for both JUYPTER_SPARK_OPTS and SPARK_OPTS when running docker run and that didn't work. I found that I <em>can</em> do <code>pip install graphframes</code> from a terminal, which gets me part of the way -- the python libraries are installed, but the java ones are not &quot;java.lang.ClassNotFoundException: org.graphframes.GraphFramePythonAPI&quot;.</p>
  <p>The <a href="https://jupyter-docker-stacks.readthedocs.io/en/latest/using/specifics.html#apache-spark" rel="nofollow noreferrer">image specifics documentation</a> does not appear to offer any insights on how to deploy a Spark Package to the image.</p>
  <p>Is there a certain place to throw the graphframes .jar? Is there a command to install a spark package post-docker? Is there a magic argument to <code>docker run</code> that would install this?</p>
  <p>I bet there's a really simple answer to this --Or am I in high cotton here?</p>
  <p>References:</p>
  <ul>
  <li><a href="https://stackoverflow.com/questions/50286139/no-module-named-graphframes-jupyter-notebook">No module named graphframes Jupyter Notebook</a></li>
  <li><a href="https://stackoverflow.com/questions/48915274/how-do-i-run-pyspark-with-jupyter-notebook">How do I run pyspark with jupyter notebook?</a></li>
  </ul>