title: 2 Hdfs files comparison
body: <p>I have 6000+ .csv files in <code>/hadoop/hdfs/location1</code> and 6100+ .csv files in <code>/hadoop/hdfs/location2</code>.</p>
  
  <p>I want to compare these two hdfs directories and find the diff of files. The diff .csv files(non-similar) should be reflected in a 3rd hdfs directory(<code>/hadoop/hdfs/location3</code>). I am not sure we can use <em>diff command</em> as in unix to hdfs file system. </p>
  
  <p>Any idea on how to resolve this would be appreciable.</p>
  
  <p>Anshul</p>