title: Hadoop MapReduce job I/O Exception due to premature EOF from inputStream
body: <p>I ran a MapReduce program using the command <code>hadoop jar &lt;jar&gt; [mainClass] path/to/input path/to/output</code>. However, my job was hanging at: <code>INFO mapreduce.Job:  map 100% reduce 29%</code>.</p>
  
  <p>Much later, I terminated and checked the datanode log (I am running in pseudo-distributed mode). It contained the following exception:</p>
  
  <pre><code>java.io.IOException: Premature EOF from inputStream
  at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:201)
  at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:213)
  at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)
  at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)
  at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:472)
  at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:849)
  at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:804)
  at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:137)
  at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:74)
  at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:251)
  at java.lang.Thread.run(Thread.java:745)
  </code></pre>
  
  <p>5 seconds later in the log was <code>ERROR DataXceiver error processing WRITE_BLOCK operation</code>.</p>
  
  <p>What problem might be causing this exception and error?</p>
  
  <p>My NodeHealthReport said:</p>
  
  <pre><code>1/1 local-dirs are bad: /home/$USER/hadoop/nm-local-dir; 
  1/1 log-dirs are bad: /home/$USER/hadoop-2.7.1/logs/userlogs
  </code></pre>
  
  <p>I found <a href="http://mail-archives.apache.org/mod_mbox/hadoop-user/201312.mbox/%3CCAF4PTbxFR5AaGJcHBREUSZNgs-qLo2_kyjcYHTMC6FiABGf+fg@mail.gmail.com%3E" rel="noreferrer">this</a> which indicates that <code>dfs.datanode.max.xcievers</code> may need to be increased. However, it is deprecated and the new property is called <code>dfs.datanode.max.transfer.threads</code> with default value  4096. If changing this would fix my problem, what new value should I set it to?</p>
  
  <p><a href="http://mail-archives.apache.org/mod_mbox/hadoop-user/201303.mbox/%3CCADE3u=buJ6V0CAHxBfxkV_EW-rTSU3gUphvsZA6_awcJBYH8fw@mail.gmail.com%3E" rel="noreferrer">This</a> indicates that the <code>ulimit</code> for the datanode may need to be increased. My <code>ulimit -n</code> (open files) is 1024. If increasing this would fix my problem, what should I set it to?</p>