title: Dataflow/apache beam - how to access current filename when passing in pattern?
body: <p>I have seen this question answered before on stack overflow (<a href="https://stackoverflow.com/questions/29983621/how-to-get-filename-when-using-file-pattern-mat">https://stackoverflow.com/questions/29983621/how-to-get-filename-when-using-file-pattern-match-in-google-cloud-dataflow</a>), but not since apache beam has added splittable dofn functionality for python. How would I access the filename of the current file being processed when passing in a file pattern to a gcs bucket? </p>
  
  <p>I want to pass the filename into my transform function: </p>
  
  <pre><code>with beam.Pipeline(options=pipeline_options) as p:                              
      lines = p | ReadFromText('gs://url to file')                                        
  
  
      data = (                                                                    
          lines                                                                   
          | 'Jsonify' &gt;&gt; beam.Map(jsonify)                                        
          | 'Unnest' &gt;&gt; beam.FlatMap(unnest)                                      
          | 'Write to BQ' &gt;&gt; beam.io.Write(beam.io.BigQuerySink(                  
              'project_id:dataset_id.table_name', schema=schema,                     
              create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED,    
              write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND)       
          )                                                   
  </code></pre>
  
  <p>Ultimately, what I want to do is pass the filename into my transform function when I transform each row of the json (see <a href="https://stackoverflow.com/questions/29983621/how-to-get-filename-when-using-file-pattern-match-in-google-cloud-dataflow">this</a> and then use the filename to do a lookup in a different BQ table to get a value). I think once I manage to know how to get the filename I will be able to figure out the side input part in order to do the lookup in the bq table and get the unique value. </p>