title: Reading file in hadoop streaming
body: <p>I am trying to read an auxiliary file in my mapper and here are my codes and commands.</p>
  
  <p>mapper code:</p>
  
  <pre><code>#!/usr/bin/env python
  
  from itertools import combinations
  from operator import itemgetter
  import sys
  
  storage = {}
  
  with open('inputData', 'r') as inputFile:
      for line in inputFile:
           first, second = line.split()
           storage[(first, second)] = 0
  
  for line in sys.stdin:
      do_something()
  </code></pre>
  
  <p>And here is my command:</p>
  
  <pre><code>hadoop jar hadoop-streaming-2.7.1.jar \
  -D stream.num.map.output.key.fields=2 \
  -D mapred.output.key.comparator.class=org.apache.hadoop.mapred.lib.KeyFieldBasedComparator \
  -D mapred.text.key.comparator.options='-k1,1 -k2,2' \
  -D mapred.map.tasks=20 \
  -D mapred.reduce.tasks=10 \
  -partitioner org.apache.hadoop.mapred.lib.KeyFieldBasedPartitioner \
  -mapper mapper.py -file mapper.py \
  -reducer reducer.py -file reducer.py \
  -file inputData \
  -input /data \
  -output /result
  </code></pre>
  
  <p>But I keep getting this error, which indicates that my mapper fails to read from stdin. After deleting the read file part, my code works, So I have pinppointed the place where the error occurs, but I don't know what should be the correct way of reading from it. Can anyone help?</p>
  
  <pre><code> Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads():
  </code></pre>