title: Dynamic partitioning in google cloud dataflow?
body: <p>I'm using dataflow to process files stored in GCS and write to Bigquery tables. Below are my requirements:</p>
  
  <ol>
  <li>input files contain events records, each record pertains to one eventType;</li>
  <li>need to partition records by eventType;</li>
  <li>for each eventType output/write records to a corresponding Bigquery table, one table per eventType. </li>
  <li>events in each batch input files vary;</li>
  </ol>
  
  <p>I'm thinking of applying transforms such as "groupByKey" and "partition", however seems that I have to know number of (and type of) events at the development time which is needed to determine the partitions. </p>
  
  <p>Do you guys have a good idea to do the partitioning dramatically? meaning partitions can be determined at run time?</p>