title: Pig - String extraction using regex
body: <p>Iam using <strong>pig</strong> to analyze log files.</p>
  
  <p>My input log file('/user/586376/Pig_C') looks like this. </p>
  
  <blockquote>
    <p><strong>12901890</strong>   \t <strong>rosies blog</strong> \t    <strong>2006-05-15 21:42:19</strong>    \t <strong>1</strong>    \t <strong><a href="http://www.rosie.com" rel="nofollow">http://www.rosie.com</a></strong></p>
  </blockquote>
  
  <pre><code>A = load '/user/586376/Pig_C' as (ID:int, query:chararray, time:chararray, rank:int, url:chararray);
  </code></pre>
  
  <p>Here,i want to list the frequently visited sites(<strong><em>top 3</em></strong>) during the following time of the day : <strong>10 A.M. to 11 A.M.</strong></p>
  
  <p>Presently, Iam using STRSPLIT on the time field to extract the required hours.</p>
  
  <pre><code>Timesplit = FOREACH A GENERATE url, STRSPLIT(time,' ') as time_split;
  
  B = FOREACH Timesplit GENERATE url, FLATTEN(time_split) as (date,time1); 
  
  C = FOREACH B GENERATE url, STRSPLIT(time1,':') as h;
  
  final = FOREACH C GENERATE url,flatten(h) AS (hour,min,sec); 
  </code></pre>
  
  <p>After splitting the time field, Iam using <strong>FILTER</strong> , <strong>GROUP BY</strong> and <strong>COUNT UDF</strong> to find the frequently visited top 3 url's during 10 A.M. and 11 A.M.</p>
  
  <p><strong>Output:</strong></p>
  
  <pre><code>(http://www.google.com,5)
  
  (http://finance.yahoo.com,2)
  
  (http://www.nada.com,2)
  </code></pre>
  
  <p>But, i feel there must be an easier way to obtain the hours field using PIG UDF's as <strong>regex_extract</strong> and <strong>regex_extract_all</strong> rather than using STRSPLIT. But Iam unable to write a regex to extract required hours from the timestamp given.</p>
  
  <p>Any ideas now to write the regular expression to find the hours from the given timestamp?</p>
  
  <p>EDITED:</p>
  
  <p>Wrote the following regex in pig to extract the hours:</p>
  
  <pre><code>data = FOREACH A GENERATE url, FLATTEN(EXTRACT(time,'\\d{4}-\\d{2}-\\d{2}\\s(\\d{2}):\\d{2}:\\d{2}')) AS (hour:chararray) ;
  </code></pre>
  
  <p>But getting the following error,</p>
  
  <p>Error:</p>
  
  <blockquote>
    <p>2013-08-19 18:20:28,745 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1045: Could not infer the matching function for org.apache.pig.piggybank.evaluation.string.RegexExtract as multiple or none of them fit. Please use an explicit cast.</p>
  </blockquote>
  
  <p>Any suggestions, where iam going wrong???</p>