title: How to transform a data frame in spark structured streaming using python?
body: <p>I am testing structured streaming using localhost from which it reads a stream of data. Input streaming data from localhost:  </p>
  
  <pre><code>ID   Subject  Marks
  --------------------
  1    Maths    85  
  1    Physics  80  
  2    Maths    70  
  2    Physics  80  
  </code></pre>
  
  <p>I would like to get the average marks for each unique ID's.</p>
  
  <p>I tried this but not able to transform the DF which is a single value.</p>
  
  <p>Below is my code:</p>
  
  <pre><code>from pyspark.sql import SparkSession
  from pyspark.sql.functions import *  
  from pyspark.sql.types import *
  spark = SparkSession.builder.appName("SrteamingAge").getOrCreate()
  
  schema = StructType([StructField("ID", IntegerType(), \  
  True),StructField("Subject", StringType(), True),StructField("Marks", \
  IntegerType(), True)])
  
  marks = spark.readStream.format("socket").option("host", 
  "localhost").option("port", 9999).schema(schema).load()
  marks.printSchema()
  result = marks.groupBy("ID").agg(avg("Marks").alias("Average Marks"))
  </code></pre>
  
  <p>But I am getting the below error:</p>
  
  <pre><code>    root
        |-- value: string (nullable = true)
  
  Pyspark.sql.utils.Analysisexception: "u can not resolve 'ID' given input columns: [value];"
  </code></pre>
  
  <p>I am creating a schema for the same but no luck. Any help would be appreciated.</p>
  
  <p>My expected output is just 2 columns (ID and Average Marks)</p>
  
  <pre><code>ID  Average Marks  
  1     82.5  
  2     75  
  </code></pre>