title: Zeppelin: Scala Dataframe to python
body: <p>If I have a Scala paragraph with a DataFrame, can I share and use that with python. (As I understand it pyspark uses <a href="https://www.py4j.org/advanced_topics.html#array">py4j</a>) </p>
  
  <p>I tried this: </p>
  
  <p>Scala paragraph: </p>
  
  <pre><code>x.printSchema
  z.put("xtable", x )
  </code></pre>
  
  <p>Python paragraph: </p>
  
  <pre><code>%pyspark
  
  import numpy as np
  import pandas as pd
  
  import matplotlib.pyplot as plt
  import seaborn as sns
  
  the_data = z.get("xtable")
  
  print the_data
  
  sns.set()
  g = sns.PairGrid(data=the_data,
                   x_vars=dependent_var,
                   y_vars=sensor_measure_columns_names +  operational_settings_columns_names,
                   hue="UnitNumber", size=3, aspect=2.5)
  g = g.map(plt.plot, alpha=0.5)
  g = g.set(xlim=(300,0))
  g = g.add_legend()
  </code></pre>
  
  <p>Error :</p>
  
  <pre><code>Traceback (most recent call last):
    File "/tmp/zeppelin_pyspark.py", line 222, in &lt;module&gt;
      eval(compiledCode)
    File "&lt;string&gt;", line 15, in &lt;module&gt;
    File "/usr/local/lib/python2.7/dist-packages/seaborn/axisgrid.py", line 1223, in __init__
      hue_names = utils.categorical_order(data[hue], hue_order)
  TypeError: 'JavaObject' object has no attribute '__getitem__'
  </code></pre>
  
  <p>Solution: </p>
  
  <pre><code>%pyspark
  
  import numpy as np
  import pandas as pd
  
  import matplotlib.pyplot as plt
  import seaborn as sns
  
  import StringIO
  def show(p):
      img = StringIO.StringIO()
      p.savefig(img, format='svg')
      img.seek(0)
      print "%html &lt;div style='width:600px'&gt;" + img.buf + "&lt;/div&gt;"
  
  df = sqlContext.table("fd").select()
  df.printSchema
  pdf = df.toPandas()
  
  g = sns.pairplot(data=pdf,
                   x_vars=["setting1","setting2"],
                   y_vars=["s4", "s3", 
                           "s9", "s8", 
                           "s13", "s6"],
                   hue="id", aspect=2)
  show(g)   
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/fjMyv.png"><img src="https://i.stack.imgur.com/fjMyv.png" alt="cluster visualisation"></a></p>