title: Retaining logs from Hadoop job after it's executed
body: <p>I'm wondering if there's an easy way to grab all the job logs / task attempt logs of a particular run, and persist them somewhere (HDFS, perhaps)?</p>
  
  <p>I know that the logs are on the local filesystem at <code>/var/log/hadoop-0.20-mapreduce/userlogs</code> for any particular job's task attempts, and that I could write a script to SSH to each of the slave nodes and scoop them all up.  However, I'm trying to avoid that if it makes sense to - perhaps there's some built-in function of Hadoop that I'm not aware of?</p>
  
  <p>I did find <a href="http://blog.cloudera.com/blog/2010/11/hadoop-log-location-and-retention/" rel="nofollow">this link</a>, which is old, but contains some helpful information -- but did not include the answer I'm looking for.</p>
  
  <p><code>mapreduce.job.userlog.retain.hours</code> is set to 24 by default, so any job's logs will be automatically purged after 1 day.  Is there anything I can do besides increasing the value of the <code>retain.hours</code> parameter to get these to persist?</p>