title: How to write a parquet file using Spark df.write.parquet with defined schema. - pyspark
body: <p>I am converting JSON to parquet file conversion using df.write. In my JSON file all my columns are the string, so while reading into dataframe I am using schema to infer and the reason for that no of columns in the file also keep changing. While writing to parquet I do not want to write them as the string instead I want some columns to change to date and decimal. I know we can select and do casting in the column level in the df select before capturing to another df. I do not want to do it since I have 300 columns in the file and program work based on parameter. Is there a way we can have them in the file as schema and do it. </p>
  
  <p>example: </p>
  
  <p>JSON read schema:</p>
  
  <pre><code>read_schema = StructType([
      StructField("event_name"              , StringType(), True),
      StructField("dollar_balance"          , StringType(), True),
      ])
  </code></pre>
  
  <p>Parquet write schema:</p>
  
  <pre><code>write_schema = StructType([
      StructField("event_name"              , StringType(), True),
      StructField("dollar_balance"          , DoubleType(), True),
      ])
  </code></pre>
  
  <p>Maybe a long question. Help would be appreciated. Thanks</p>