title: Comparing images to find duplicates
body: <p>I have a few (38000) picture/video files in a folder. Approximately 40% of these are duplicates which I'm trying to get rid of. My question is, how can I tell if 2 files are identical? So far I tried to use a SHA1 of the files but it turns out that many duplicates files had different hashes. This is the code I was using:</p>
  
  <pre><code>public static String getHash(File doc) {
      MessageDigest md = null;
      try {
          md = MessageDigest.getInstance("SHA1");
          FileInputStream inStream = new FileInputStream(doc);
          DigestInputStream dis = new DigestInputStream(inStream, md);
          BufferedInputStream bis = new BufferedInputStream(dis);
          while (true) {
              int b = bis.read();
              if (b == -1)
                  break;
          }
  
          inStream.close();
          dis.close();
          bis.close();
      } catch (NoSuchAlgorithmException | IOException e) {
          e.printStackTrace();
      }
  
      BigInteger bi = new BigInteger(md.digest());
  
      return bi.toString(16);
  }
  </code></pre>
  
  <p>Can I modify this in any way? Or will I have to use a different method?</p>