title: Checksum Exception when reading from or copying to hdfs in apache hadoop
body: <p>I am trying to implement a parallelized algorithm using Apache hadoop, however I am facing some issues when trying to transfer a file from the local file system to hdfs. A <strong>checksum exception</strong> is being thrown when trying to read from or transfer a file. </p>
  
  <p>The strange thing is that some files are being successfully copied while others are not (I tried with 2 files, one is slightly bigger than the other, both are small in size though). Another observation that I have made is that the Java <em>FileSystem.getFileChecksum</em> method, is returning a <strong>null</strong> in all cases.</p>
  
  <p>A slight background on what I am trying to achieve: I am trying to write a file to hdfs, to be able to use it as a distributed cache for the mapreduce job that I have written.</p>
  
  <p>I have also tried the <em>hadoop fs -copyFromLocal</em> command from the terminal, and the result is the exact same behaviour as when it is done through the java code.</p>
  
  <p>I have looked all over the web, including other questions here on stackoverflow however I haven't managed to solve the issue. Please be aware that I am still quite new to <em>hadoop</em> so any help is greatly appreciated.</p>
  
  <p>I am attaching the stack trace below which shows the exceptions being thrown. (In this case I have posted the stack trace resulting from the <em>hadoop fs -copyFromLocal</em> command from terminal)</p>
  
  <pre><code>name@ubuntu:~/Desktop/hadoop2$ bin/hadoop fs -copyFromLocal ~/Desktop/dtlScaleData/attr.txt /tmp/hadoop-name/dfs/data/attr2.txt
  
  13/03/15 15:02:51 INFO util.NativeCodeLoader: Loaded the native-hadoop library
      13/03/15 15:02:51 INFO fs.FSInputChecker: Found checksum error: b[0, 0]=
      org.apache.hadoop.fs.ChecksumException: Checksum error: /home/name/Desktop/dtlScaleData/attr.txt at 0
          at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:219)
          at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
          at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
          at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
          at java.io.DataInputStream.read(DataInputStream.java:100)
          at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
          at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:47)
          at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
          at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:230)
          at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:176)
          at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1183)
          at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:130)
          at org.apache.hadoop.fs.FsShell.run(FsShell.java:1762)
          at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
          at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
          at org.apache.hadoop.fs.FsShell.main(FsShell.java:1895)
      copyFromLocal: Checksum error: /home/name/Desktop/dtlScaleData/attr.txt at 0
  </code></pre>