title: How to import nltk corpus in HDFS when I use hadoop streaming
body: <pre><code> I got a little problem I want to use nltk corpus in hdfs,But failed.For example I want to load nltk.stopwords in my python code.
   I use this http://eigenjoy.com/2009/11/18/how-to-use-cascading-with-hadoop-streaming/
  </code></pre>
  
  <p>I do all that say,but I don't know how to transform it in my work. My nltk file name is nltk-2.0.1.rc1 my pyam file name is PyYAML.3.0.1 so my commad is:</p>
  
  <pre><code>zip -r nltkandyaml.zip nltk-2.0.1.rc1 PyYAML.3.0.1
  </code></pre>
  
  <p>then it said "mv ntlkandyaml.zip /path/to/where/your/mapper/will/be/nltkandyaml.mod"</p>
  
  <p>My mapper.py save in /home/mapreduce/mapper.py  so my command is:</p>
  
  <pre><code>mv ntlkandyaml.zip /home/mapreduce/nltkandyaml.mod
  </code></pre>
  
  <p>is that right?</p>
  
  <p>then i zip my corpus stopwords:</p>
  
  <pre><code>zip -r /nltk_data/corpora/stopwords-flat.zip *
  </code></pre>
  
  <p>In my code I use:</p>
  
  <pre><code>importer = zipimport.zipimporter('nltkandyaml.mod')
  yaml = importer.load_module('PyYAML-3.09')
  nltk = importer.load_module('nltk-2.1.0.1rc1')
  from nltk.corpus.reader import stopwords
  from nltk.corpus.reader import StopWordsCorpusReader
  nltk.data.path+=["."]
  stopwords = StopWordsCorpusReader(nltk.data.find('lib/stopwords-flat.zip'))
  </code></pre>
  
  <p>finally  I use command:</p>
  
  <pre><code>bin/hadoop jar /home/../streaming/hadoop-0.21.0-streaming.jar -input  
  /user/root/input/voa.txt -output /user/root/output -mapper /home/../mapper.py -reducer  
  /home/../reducer.py -file /home/../nltkandyaml.mod -file /home/../stopwords-flat.zip
  </code></pre>
  
  <p>please tell me where I'm wrong </p>
  
  <p>thank you all</p>