title: Remove struct from Array column in PySpark dataframe
body: <p>I want to remove an array from an array of structs (in Array column) in a dataframe (PySpark).</p>
  <pre><code>import pyspark.sql.functions as F
  from pyspark.sql.types import StructType, StructField, StringType, IntegerType
  
  data = [(&quot;1&quot;, &quot;A&quot;, 2), (&quot;1&quot;, None, 0), (&quot;1&quot;, &quot;B&quot;, 3), (&quot;2&quot;, None, 0), (&quot;2&quot;, &quot;C&quot;, 4), (&quot;2&quot;, &quot;D&quot;, 1), (&quot;2&quot;, None, 0)]
  dfschema = StructType([StructField(&quot;id&quot;, StringType()), StructField(&quot;value&quot;, StringType()), StructField(&quot;amount&quot;, IntegerType())])
  df = spark.createDataFrame(data, schema=dfschema)
  
  grouped = (
      df.
      groupby(&quot;id&quot;).
      agg(
          F.collect_list(
              F.struct(
                  F.col(&quot;value&quot;), 
                  F.col(&quot;amount&quot;)
              )
          ).alias(&quot;collected&quot;)
      )
  )
  grouped.show(truncate=False)
  
  +---+------------------------------+                                            
  |id |collected                     |
  +---+------------------------------+
  |1  |[[A, 2], [, 0], [B, 3]]       |
  |2  |[[, 0], [C, 4], [D, 1], [, 0]]|
  +---+------------------------------+
  </code></pre>
  <p>Here's the result I want</p>
  <pre><code>+---+-----------------------+
  |id |collected              |
  +---+-----------------------+
  |1  |[[A, 2], [B, 3]]       |
  |2  |[[C, 4], [D, 1]]       |
  +---+-----------------------+
  </code></pre>
  <p>I tried using <code>F.array_remove(..., [, 0])</code> but that gives an error. Not quite sure how I can define the element I want to remove. Thanks!</p>