title: How to debug Pig being stuck after job submission
body: <p>I have a map-reduce job written in Pig that is doing the following.</p>
  
  <p>Given a set of apache log files representing visits to a certain resource on a website</p>
  
  <ul>
  <li>clean the logs from the robots and from the unwanted log lines</li>
  <li>produce the tuples (ip, resource_id) found in the logs</li>
  </ul>
  
  <p>For example this log:<br>
  <code>1.1.1.1 - [14/Jun/2014:06:26:27 +0000] "GET /path/to/resource/&lt;resource_id&gt;" "Agent"</code></p>
  
  <p>will be transalted into (if it's not a robot):<br>
  <code>(1.1.1.1, &lt;resource_id&gt;)</code></p>
  
  <p>This is done with a simple UDF that parse the log with a regex + a library for robot detection.<br>
  Starting from this step the script continues with a couple of additional map-reduce operations. </p>
  
  <p>The issue is the following:</p>
  
  <ul>
  <li>I am able to complete the pig job locally.  </li>
  <li>I uploaded the script to Amazon Elastic Map Reduce with 5GB of logs to be processed.  </li>
  <li>I ran the script for 1 hours with 10 <code>m1.large</code> instances.   </li>
  <li>The job didn't finished and I terminated it.   </li>
  </ul>
  
  <p>The hadoop logs that have been produced does not show much progress and seems to be stuck in the initial preparation phase described previously.  </p>
  
  <p><code>2014-07-07 06:31:17,609 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: pre1[4,7],pre2[-1,-1],pre3[7,7],pre4[8,7],r2[13,5] C:  R: r5[-1,-1]
  2014-07-07 06:31:17,661 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
  </code></p>
  
  <p>How would you suggest to debug the issue starting from here?<br>
  Do you think that given the data size the number of machine is reasonable?<br>
  I was really expecting to finish the job in that amount of time.</p>
  
  <p>Thank you</p>