title: Read only specific rows of .parquet files matching criteria?
body: <p>I'm working against a filesystem filled with .parquet files. One of the columns, 'id', uniquely identifies a machine. I was able to use pyspark to open all .parquet files in a certain directory path, then create a set([]) of the values from the 'id' column. I'd like to open all other rows in all other files, where the 'id' matches one of the values in the previously calculated set.</p>
  
  <p>I was able to do this via pyspark, but it's quite complex and requires me to instantiate a local spark server. I'm trying to find a way to do this via pyarrow, but it seems that it's read_pandas / read methods 'filters' argument can only filter on partition data, and not arbitrary column data.</p>
  
  <p>Is there a way to achieve what I'm looking for here? I can't open the entire dataset and then use Python to filter out rows where the 'id' doesn't match, because it doesn't fit in memory.</p>