title: Convert byte array to string spark
body: <p>I have a csv file which contains multiple fields. There are few fields for which data is coming in byte array format(b'1234'). I want to remove these b'(convert from byte array to string). </p>
  
  <p>I came to know that we can convert byte array to string in two ways.</p>
  
  <pre><code>&gt;&gt;&gt; s1 = b'Hi'
  &gt;&gt;&gt; s2 = s1.decode('utf-8') 
  &gt;&gt;&gt; print(s2)
  Hi
  
  
  &gt;&gt;&gt; s1 = b'Hi'
  &gt;&gt;&gt; s2 = str(s1, 'utf-8')
  &gt;&gt;&gt; print(s2)
  Hi
  </code></pre>
  
  <p>As there so many fields in CSV out of which only few fields contains byte array, I can't directily apply the function to each and every field. I don't have idea of which are are byte array fields and which ate string, int fields.</p>
  
  <p>Any suggestions to convert byte array to string in csv file. I'm trying to do this in spark.</p>
  
  <p>My code snippet :</p>
  
  <pre><code>df = sqlContext.read.format('com.databricks.spark.csv').options(header='true', inferschema='true') \
      .option("delimiter", ",")\
      .option("multiLine", "true")\
      .load("file.csv")
  </code></pre>
  
  <p><strong>Input Data:</strong></p>
  
  <pre><code>b'1234',b'123',Hi,"Hello",b'2FB',b'272',b'4943',b'786',2018-02-19,,Out,768,"Data",b'502',351,
  </code></pre>
  
  <p>As schema changes dynamically we don't have control of knowing which are byte array and which are string. I tried <a href="https://stackoverflow.com/questions/40389764/how-to-translate-bytes-objects-into-literal-strings-in-pandas-dataframe-pytho">this</a> solution, however it didn't worked for me (it converted all the fields to nan).</p>