title: Creating Dataframe out of two different arrays spark
body: <p>I am using spark streaming and am creating this dataframe from the kafka message:</p>
  
  <pre><code>|customer|initialLoadComplete|initialLoadRunning|      messageContent|        tableName|
  +--------+-------------------+------------------+--------------------+-----------------+
  |   A|              false|              true|TEFault_IdReason...|Timed_Event_Fault|
  |   A|              false|              true|TEFault_IdReason...|Timed_Event_Fault|
  +--------+-------------------+------------------+--------------------+-----------------+
  </code></pre>
  
  <p>Now I want to extract out messageContent, messageContent is basically like a CSV that includes the raw data and the first line is the columns.
  I can extract out the headers in the following way from the messageContent field.</p>
  
  <pre><code> val Array1 = ssc.sparkContext.parallelize(rowD.getString(2).split("\u0002")(0))
  </code></pre>
  
  <p>So Array1 looks like this: </p>
  
  <pre><code> Array1: col1^Acol2^Acol3
  </code></pre>
  
  <p>Array2 is basically the raw data, each column value seperated by ^A and record seperated by ^B. </p>
  
  <p>^A is a column seperator. ^B is record seperator</p>
  
  <p>So this is what array2 could look like:</p>
  
  <pre><code>Array2 = value1^Avalue2^Avalue3^Bvalue4^Avalue5^Avalue6^Bvalue7^Avalue8^Avalue9
  </code></pre>
  
  <p>Basically I want to create a dataframe out of this so it looks like this:</p>
  
  <pre><code>col1   | col2   | col3
  -------------------------
  value1 | value2 | value3
  value4 | value5 | value6
  value7 | value8 | value9
  </code></pre>
  
  <p>^B is the record delimiter. </p>
  
  <p>When we were reading from a hdfs file, we created a dataframe via this command:</p>
  
  <pre><code>  val df = csc.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").option("delimiter", "\u0001").load(hdfsFile)
  </code></pre>
  
  <p>But this time I am creating a dataframe from two arrays from memory. Array1 is the headers for the values in array2 and array2 is record seperated by ^B.</p>
  
  <p>What could be the equivalent of creating a dataframe in this approach as I did for creating a dataframe from a file.</p>