title: How to track lots of statistics with map-reduce cascading?
body: <p>I have a series map-reduce jobs to process user data (implemented using the Cascading framework), and I would like to track lots of fine-grained statistics (I can have between 100 and 1000 users and 20 statistics per user, so, possibly between 5000 and 10.000 statistics in total).
  I wanted to use map-reduce counters to build those stats because it is very convenient to use them in the code, but there is a limit to the number of map-reduce counters (120 by default), and according to this post: <a href="http://developer.yahoo.com/blogs/hadoop/posts/2010/08/apache_hadoop_best_practices_a/" rel="nofollow">http://developer.yahoo.com/blogs/hadoop/posts/2010/08/apache_hadoop_best_practices_a/</a>
  I should not use them if i have more than 20/50 custom counters.</p>
  
  <p>Question: is there a proper way to track my statistics in this map-reduce context, using a counter-like pattern ? by counter-like, i mean, to have access to counters everywhere in my code and be able to increment them where needed.</p>
  
  <p>thanks by advance
  register</p>