Is there really no use for MD5 anymore? I read an article about password schemes that makes two seemingly conflicting claims:

MD5 is broken; it’s too slow to use as a general purpose hash; etc
The problem is that MD5 is fast

I know that MD5 should not be used for password hashing, and that it also should not be used for integrity checking of documents.  There are way too many sources citing MD5 preimaging attacks and MD5s low computation time.
However, I was under the impression that MD5 still can be used as a non-cryptgraphic hash function:

Identifying malicious files, such as when Linux Mint's download servers were compromised and an ISO file was replaced by a malicious one; in this case you want to be sure that your file doesn't match; collision attacks aren't a vector here.
Finding duplicate files.  By MD5-summing all files in a directory structure it's easy to find identical hashes.  The seemingly identical files can then be compared in full to check if they are really identical.  Using SHA512 would make the process slower, and since we compare files in full anyway there is no risk in a potential false positive from MD5. (In a way, this would be creating a rainbow table where all the files are the dictionary)

There are checksums of course, but from my experience, the likelihood of finding two different files with the same MD5 hash is very low as long as we can rule out foul play.
When the password scheme article states that &quot;MD5 is fast&quot;, it clearly refers to the problem that hashing MD5 is too cheap when it comes to hashing a large amount of passwords to find the reverse of a hash.  But what does it mean when it says that &quot;[MD5 is] too slow to use as a general purpose hash&quot;?  Are there faster standardized hashes to compare files, that still have a reasonably low chance of collision?
I know that MD5 should not be used for password hashing, and that it also should not be used for integrity checking of documents. There are way too many sources citing MD5 preimaging attacks and MD5s low computation time.


There is no published preimage attack on MD5 that is cheaper than a generic attack on any 128-bit hash function.  But you shouldn't rely on that alone when making security decisions, because cryptography is tricky and adversaries are clever and resourceful and can find ways around it!


  
  Identifying malicious files, such as when Linux Mint's download servers were compromised and an ISO file was replaced by a malicious one; in this case you want to be sure that your file doesn't match; collision attacks aren't a vector here.
  


The question of whether to publish known-good vs. known-bad hashes after a compromise is addressed elsewhere—in brief, there's not much that publishing known-bad hashes accomplishes, and according to the citation, Linux Mint published known-good, not known-bad, hashes.  So what security do you get from known-good MD5 hashes?

There are two issues here:


If you got the MD5 hash from the same source as the ISO image, there's nothing that would prevent an adversary from replacing both the MD5 hash and the ISO image.

To prevent this, you and the Linux Mint curators need two channels: one for the hashes which can't be compromised (but need only have very low bandwidth), and another for the ISO image (which needs high bandwidth) on which you can then use the MD5 hash in an attempt to detect compromise.

There's another way to prevent this: Instead of using the uncompromised channel for the hash of every ISO image over and over again as time goes on—which means more and more opportunities for an attacker to subvert it—use it once initially for a public key, which is then used to sign the ISO images; then there's only one opportunity for an attacker to subvert the public key channel.
Even if you got the MD5 hash from a known-good source, collision attacks may still be a vector in cases like this.  Consider the following scenario:


I am an evil developer.  I write two software packages, whose distributions collide under MD5.  One of the packages is benign and will survive review and audit.  The other one will surreptitiously replace your family photo album by erotic photographs of sushi.
The Linux Mint curators carefully scrutinize and audit everything they publish in their package repository and publish the MD5 hashes of what they have audited in a public place that I can't compromise.
The Linux Mint curators cavalierly administer the package distributions in their package repository, under the false impression that the published MD5 hashes will protect users.


In this scenario, I can replace the benign package by the erotic sushi package, pass the MD5 verification with flying colors, and give you a nasty—and luscious—surprise when you try to look up photos of that old hiking trip you took your kids on.



  
  Finding duplicate files.  By MD5-summing all files in a directory structure it's easy to find identical hashes.  The seemingly identical files can then be compared in full to check if they are really identical.  Using SHA512 would make the process slower, and since we compare files in full anyway there is no risk in a potential false positive from MD5.
  


When I put my benign software package and my erotic sushi package, which collide under MD5, in your directory, your duplicate-detection script will initially think they are duplicates.  In this case, you absolutely must compare the files in full.  But there are much better ways to do this!


If you use SHA-512, you can safely skip the comparison step.  Same if you use BLAKE2b, which can be even faster than MD5.
You could even use MD5 safely for this if you use it as HMAC-MD5 under a uniform random key, and safely skip the comparison step.  HMAC-MD5 does not seem to be broken, as a pseudorandom function family—so it's probably fine for security, up to the birthday bound, but there are better faster PRFs like keyed BLAKE2 that won't raise any auditors' eyebrows.
Even better, you can choose a random key and hash the files with a universal hash under the key, like Poly1305.  This is many times faster than MD5 or BLAKE2b, and the probability of a collision between any two files is less than $1/2^{100}$, so the probability of collision among $n$ files is less than $\binom n 2 2^{-100}$ and thus you can still safely skip the comparison step until you have quadrillions of files.
You could also just use a cheap checksum like a CRC with a fixed polynomial.  This will be the fastest of the options—far and away faster than MD5—but unlike the previous options you still absolutely must compare the files in full.


So, is MD5 safe for finding candidate duplicates to verify, if you subsequently compare the files bit by bit in full?  Yes.  So is the constant zero function.


  (In a way, this would be creating a rainbow table where all the files are the dictionary)


This is not a rainbow table.  A rainbow table is a specific technique for precomputing a random walk over a space of, say, passwords, via, say, MD5 hashes, in a way that can find MD5 preimages for hashes that aren't necessarily in your table with fewer trial hash evaluations than a naive search.  (It can also be done in parallel with the search to not only speed it up but reduce the price/performance ratio.)  It is not simply a list of precomputed hashes on a dictionary of inputs.

(The blog post by tptacek in 2007 that you cited, and the blog post by Jeff Atwood that it was a response to, both misrepresent what rainbow tables are.  tptacek claims in 2019 that he DGAF about rainbow tables, perhaps because modern password hashes thwart them by the mere use of salts.  But password search under a fixed, unsalted hash function is only one application of the idea, for which the term was popularized.  Rainbow tables are relevant to any preimage search as noted in the original paper, such as AES keys given known plaintext/ciphertext pairs.  And it's not just a trivial time/memory tradeoff—a rainbow table preimage search can reduce price/performance ratio if parallelized.  This is why, for instance, AES-128 should not be construed to give ‘128-bit security’.)


  When the password scheme article states that "MD5 is fast", it clearly refers to the problem that hashing MD5 is too cheap when it comes to hashing a large amount of passwords to find the reverse of a hash.  But what does it mean when it says that "[MD5 is] too slow to use as a general purpose hash"?  Are there faster standardized hashes to compare files, that still have a reasonably low chance of collision?


I don't know what tptacek meant—you could email and ask—but if I had to guess, I would guess this meant it's awfully slow for things like hash tables, where you would truncate MD5 to a few bits to determine an index into an array of buckets or an open-addressing array.