title: Spark Row to JSON
body: <p>I would like to create a JSON from a Spark v.1.6 (using scala) dataframe. I know that there is the simple solution of doing <code>df.toJSON</code>.</p>
  
  <p>However, my problem looks a bit different. Consider for instance a dataframe with the following columns:</p>
  
  <pre><code>|  A  |     B     |  C1  |  C2  |    C3   |
  -------------------------------------------
  |  1  | test      |  ab  |  22  |  TRUE   |
  |  2  | mytest    |  gh  |  17  |  FALSE  |
  </code></pre>
  
  <p>I would like to have at the end a dataframe with </p>
  
  <pre><code>|  A  |     B     |                        C                   |
  ----------------------------------------------------------------
  |  1  | test      | { "c1" : "ab", "c2" : 22, "c3" : TRUE }    |
  |  2  | mytest    | { "c1" : "gh", "c2" : 17, "c3" : FALSE }   |
  </code></pre>
  
  <p>where C is a JSON containing <code>C1</code>, <code>C2</code>, <code>C3</code>. Unfortunately, I at compile time I do not know what the dataframe looks like (except the columns <code>A</code> and <code>B</code> that are always "fixed").</p>
  
  <p>As for the reason why I need this: I am using Protobuf for sending around the results. Unfortunately, my dataframe sometimes has more columns than expected and I would still send those via Protobuf, but I do not want to specify all columns in the definition.</p>
  
  <p>How can I achieve this?</p>