title: Using Hadoop to find files that contain a particular string
body: <p>I have around <code>1000 files</code> and each file is of the size of <code>1GB</code>. And I need to find a String in all these <code>1000 files</code> and also which files contains that particular String. I am working with Hadoop File System and all those <code>1000 files</code> are in Hadoop File System.</p>
  
  <p>All the <code>1000 files</code> are under real folder, so If I do like this below, I will be getting all the <code>1000 files</code>. And I need to find which files contains a particular String <strong>hello</strong> under real folder.</p>
  
  <pre><code>bash-3.00$ hadoop fs -ls /technology/dps/real
  </code></pre>
  
  <p>And this is my data structure in hdfs-</p>
  
  <pre><code>row format delimited 
  fields terminated by '\29'
  collection items terminated by ','
  map keys terminated by ':'
  stored as textfile
  </code></pre>
  
  <p>How I can write MapReduce jobs to do this particular problem so that I can find which files contains a particular string? Any simple example will be of great help to me.</p>
  
  <p><strong>Update:-</strong></p>
  
  <p>With the use of grep in Unix I can solve the above problem scenario, but it is very very slow and it takes lot of time to get the actual output-</p>
  
  <pre><code>hadoop fs -ls /technology/dps/real | awk '{print $8}' | while read f; do hadoop fs -cat $f | grep cec7051a1380a47a4497a107fecb84c1 &gt;/dev/null &amp;&amp; echo $f; done
  </code></pre>
  
  <p>So that is the reason I was looking for some MapReduce jobs to do this kind of problem...</p>