title: Can't import lzo files in pyspark
body: <p>I have a csv file compressed in lzo format and I want to import it into a pyspark dataframe. Were the file not compressed, I would simply do:</p>
  
  <pre><code>import pyspark as ps
  
  spark = ps.sql.SparkSession.builder.master("local[2]").getOrCreate()
  data = spark.read.csv(fp, schema=SCHEMA, sep="\t")
  </code></pre>
  
  <p>where the file path <code>fp</code> and schema <code>SCHEMA</code> are properly defined elsewhere. When the file is compressed with lzo, however, this returns a dataframe filled with <code>null</code> values.</p>
  
  <p>I have installed lzop on my machine and can decompress the file from the terminal then import it using pyspark. However, that's not a feasible solution due to hard disk space and time constraints (I have tons of lzo files).</p>