title: Error handling in hadoop map reduce
body: <p>Based on the documentation, there are a few ways, how the error handling is performed in map reduce. Below are the few:</p>
  
  <p>a. Custom counters using enum - increment for every failed record.</p>
  
  <p>b. Log error and analyze later.</p>
  
  <p>Counters give the number of failed records. However to get the identifier of the failed record(may be its unique key), and details of the exception occurred, node on which the error occurred - we need to perform centralized log analysis and there are many nodes running. Logstash is on which is available. </p>
  
  <p>Apart from these, are there any other ways to handle the error scenarios, without manual intervention. Any tools, references, best practices are welcome.</p>
  
  <p>I think the same technique applies to any distributed applications, with minor changes.</p>