title: Spark java - How to store java object in spark dataframe?
body: <p>I'm using spark in java and I want to store java objects in spark dataframe. I've searched many blogs and links but couldn't find a way to do this. Let's say my object class is -</p>
  <pre><code>class Temporary{
      Integer id;
      String name;
      String another;
  
      public Temporary(Integer id, String name, String another){
          this.id = id;
          this.name = name;
          this.another = another;
      }
  }
  </code></pre>
  <p>I have a dataframe df and I want to store an object of this Temporary class in a column. Assume I am adding a column <code>Temporary</code> of array type that will have objects of Temporary class. I want to do something like-</p>
  <pre><code>df = df.withColumn(&quot;Temporary&quot;, functions.array(
                                 functions.expr(&quot;CASE WHEN something_is_true THEN Temporary(1,&quot;john&quot;,&quot;tempstring&quot;) ELSE empty_object END&quot;),
                                 functions.expr(&quot;CASE WHEN something_else_is_true THEN Temporary(2,&quot;johny&quot;,&quot;tempstring&quot;) ELSE empty_object END&quot;)
                                ));
  
  // expected result would be 
  
  +------------------------------------------+
  |                Temprorary                |
  +------------------------------------------+
  |  [Temproary_class_object, empty_object]  |
  +------------------------------------------+
  
  </code></pre>
  <p>I know this will not work because expr only accepts a SQL string and not objects. Any way to do this? Any help would be appreciated.</p>