title: How Can I find all the Null fields in Dataframe
body: <p>I am trying to find out the null fields present in a dataframe and concatenate all the fields in a new field in the same dataframe.</p>
  <p>Input dataframe looks like this</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th style="text-align: left;">name</th>
  <th style="text-align: left;">state</th>
  <th style="text-align: left;">number</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td style="text-align: left;">James</td>
  <td style="text-align: left;">CA</td>
  <td style="text-align: left;">100</td>
  </tr>
  <tr>
  <td style="text-align: left;">Julia</td>
  <td style="text-align: left;">Null</td>
  <td style="text-align: left;">Null</td>
  </tr>
  <tr>
  <td style="text-align: left;">Null</td>
  <td style="text-align: left;">CA</td>
  <td style="text-align: left;">200</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>Expected Output</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th style="text-align: left;">name</th>
  <th style="text-align: left;">state</th>
  <th style="text-align: left;">number</th>
  <th style="text-align: left;">Null Fields</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td style="text-align: left;">James</td>
  <td style="text-align: left;">CA</td>
  <td style="text-align: left;">100</td>
  <td style="text-align: left;"></td>
  </tr>
  <tr>
  <td style="text-align: left;">Julia</td>
  <td style="text-align: left;">Null</td>
  <td style="text-align: left;">Null</td>
  <td style="text-align: left;">state,number</td>
  </tr>
  <tr>
  <td style="text-align: left;">Null</td>
  <td style="text-align: left;">CA</td>
  <td style="text-align: left;">200</td>
  <td style="text-align: left;">name</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>My code looks like this but it is failing. Please help me here.</p>
  <pre><code>from pyspark.sql import functions as F
  from pyspark.sql.types import StructType,StructField, StringType, IntegerType
  data2 = [(&quot;James&quot;,&quot;CA&quot;,&quot;100&quot;),
  (&quot;Julia&quot;,None,None),
  (None,&quot;CA&quot;,&quot;200&quot;)]
  
  schema = StructType([ \
  StructField(&quot;name&quot;,StringType(),True), \
  StructField(&quot;state&quot;,StringType(),True), \
  StructField(&quot;number&quot;,StringType(),True)
  ])
  
  df = spark.createDataFrame(data=data2,schema=schema)
  cols = [&quot;name&quot;,&quot;state&quot;,&quot;number&quot;]
  df.show()
  
  def null_constraint_check(df,cols):
   df_null_identifier = df.withColumn(&quot;NULL Fields&quot;,\
                                      [F.count(F.when(F.col(c).isNull(), c)) for c in cols])
   return df_null_identifier
  df1 = null_constraint_check(df,cols)
  </code></pre>
  <p>Error I am getting</p>
  <pre><code>AssertionError: col should be Column
  </code></pre>