title: What is the Best Logging method in Hadoop MapReduce java classes?
body: <p>I wrote classes which run on hadoop (2.1 or 2.2) as part of hive UDF/UDTF query. Eventually it runs as a Map/Reduce job.
  The application performs various tasks, and I want to add logging based on log level.</p>
  
  <p>I would like to know what is the best way (or the proper way) to do so.
  Clearly I DO NOT want the logs to reside in some file on the data node that is not getting collected. I want hadoop to collect the logs so that we will be able to review all logs from a central location like the job tracker.</p>
  
  <p><strong>Option-1:</strong> the straight forward logging solution - print to system.out / system.err
  I see lots of forums mentioning printing to system out or system err, but it will mean developing our own logging mechanism with debug levels etc...</p>
  
  <pre><code>System.out.println("my log message");
  </code></pre>
  
  <p><strong>Option-2:</strong> using hadoop’s log4j commons logging
  Other posts mention using apache commons logging.</p>
  
  <pre><code>import org.apache.commons.logging.Log;
  import org.apache.commons.logging.LogFactory;
  private static final Log LOG = LogFactory.getLog(MgrsIndexCommon.class);
  LOG.trace(“my logging message”);
  </code></pre>
  
  <p>I used it, and it works, but I don’t have permissions to modify the logger configuration file. 
  I am thinking of adding a feature to have log level in my applications configuration file, and modifying the log level from the java code. 
  By the way I didn’t add a dependency to log4j in my pom.xml, and I used the one that comes with hadoop libraries.</p>
  
  <p>If someone used this option, than how did you configure the logger?</p>
  
  <p><strong>Option-3:</strong> <a href="http://logging.apache.org/log4j/2.x/" rel="nofollow">log4j 2</a>
  Some architect mentioned that we should look into adding log4j 2 dependency, and use it. Any suggestions in this area? If hadoop is using log4j can I bring my own log4j 2 and use it to write to hadoops logs?</p>
  
  <p>Before I continue, I wanted to see if someone has good suggestions.</p>
  
  <p>Thanks.</p>