title: How can I incorporate the current input filename into my Pig Latin script?
body: <p>I am processing data from a set of files which contain a date stamp as part of the filename. The data within the file does not contain the date stamp. I would like to process the filename and add it to one of the data structures within the script. Is there a way to do that within Pig Latin (an extension to PigStorage maybe?) or do I need to preprocess all of the files using Perl or the like beforehand?</p>
  
  <p>I envision something like the following:</p>
  
  <pre><code>-- Load two fields from file, then generate a third from the filename
  rawdata = LOAD '/directory/of/files/' USING PigStorage AS (field1:chararray, field2:int, field3:filename);
  
  -- Reformat the filename into a datestamp
  annotated = FOREACH rawdata GENERATE
    REGEX_EXTRACT(field3,'*-(20\d{6})-*',1) AS datestamp,
    field1, field2;
  </code></pre>
  
  <p>Note the special "filename" datatype in the LOAD statement. Seems like it would have to happen there as once the data has been loaded it's too late to get back to the source filename.</p>