title: Hadoop Streaming Job failed error in python
body: <p>From <a href="http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-program-in-python/#running-the-python-code-on-hadoop" rel="noreferrer">this guide</a>, I have successfully run the sample exercise. But on running my mapreduce job, I am getting the following error<br>
  <code>ERROR streaming.StreamJob: Job not Successful!<br>
  10/12/16 17:13:38 INFO streaming.StreamJob: killJob...<br>
  Streaming Job Failed!</code><br>
  Error from the log file  </p>
  
  <pre><code>java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 2
  at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311)
  at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545)
  at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
  at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
  at org.apache.hadoop.mapred.Child.main(Child.java:170)
  </code></pre>
  
  <p>Mapper.py</p>
  
  <pre><code>import sys
  
  i=0
  
  for line in sys.stdin:
      i+=1
      count={}
      for word in line.strip().split():
          count[word]=count.get(word,0)+1
      for word,weight in count.items():
          print '%s\t%s:%s' % (word,str(i),str(weight))
  </code></pre>
  
  <p>Reducer.py  </p>
  
  <pre><code>import sys
  
  keymap={}
  o_tweet="2323"
  id_list=[]
  for line in sys.stdin:
      tweet,tw=line.strip().split()
      #print tweet,o_tweet,tweet_id,id_list
      tweet_id,w=tw.split(':')
      w=int(w)
      if tweet.__eq__(o_tweet):
          for i,wt in id_list:
              print '%s:%s\t%s' % (tweet_id,i,str(w+wt))
          id_list.append((tweet_id,w))
      else:
          id_list=[(tweet_id,w)]
          o_tweet=tweet
  </code></pre>
  
  <p>[edit] command to run the job:  </p>
  
  <pre><code>hadoop@ubuntu:/usr/local/hadoop$ bin/hadoop jar contrib/streaming/hadoop-0.20.0-streaming.jar -file /home/hadoop/mapper.py -mapper /home/hadoop/mapper.py -file /home/hadoop/reducer.py -reducer /home/hadoop/reducer.py -input my-input/* -output my-output
  </code></pre>
  
  <p>Input is any random sequence of sentences.</p>
  
  <p>Thanks,</p>