title: Configuring Hadoop logging to avoid too many log files
body: <p>I'm having a problem with Hadoop producing too many log files in $HADOOP_LOG_DIR/userlogs (the Ext3 filesystem allows only 32000 subdirectories) which looks like the same problem in this question: <a href="https://stackoverflow.com/questions/2091287/error-in-hadoop-mapreduce">Error in Hadoop MapReduce</a></p>
  
  <p>My question is: does anyone know how to configure Hadoop to roll the log dir or otherwise prevent this? I'm trying to avoid just setting the "mapred.userlog.retain.hours" and/or "mapred.userlog.limit.kb" properties because I want to actually keep the log files.</p>
  
  <p>I was also hoping to configure this in log4j.properties, but looking at the Hadoop 0.20.2 source, it writes directly to logfiles instead of actually using log4j. Perhaps I don't understand how it's using log4j fully.</p>
  
  <p>Any suggestions or clarifications would be greatly appreciated.</p>