Is there are a metric based on collisions to compare bad hash functions? Bad hash functions are not so perfect as in the "general collision probability"  hypothesis... And a general concept of "collision resistence"  not need the constraint  of independence between the hash function and the sample set.

Let $H$ be a hash function with digest of $k$ bits, and $S$ a input sample set with size $s=|S|$.$H$ is collision resistant if it is "hard to find" two inputs that hash to the same output, &nbsp;
$\exists x\ne y\in S|H(x)=H(y)$

But "hard to find" is not a good definition: to compare the performance of two hash functions $H_1$ and $H_2$ we need a precise metric... Wikipedia  use $Pr[H]$, the probability concept to define this metric... But, in real-life tests or benchmarking, we have only the number of collisions $c$, and a big standard deviation to compare $c$ values when change $S$ or $H$.

Puting the question in other words:  &nbsp; there are a "standard way" to obtain the real-life $Pr[H]$ of bad hash functions?



Context. Let me be a programmer that buy low quality proprietary software that implement exotic hash functions: the hash function $H$ for me is a black box, and I need to test it.



So, if the seller of the function says it is resistant, I need to check it by experiments and some statistical analysis of the experimental results.



Back to the question... If this standard exists, I am supposing that is a kind of probability definition, so,  will be someting like (is it? need only to check the number $c$ of collisions) $$Pr[H](S)=\frac{c}{|S|}=\frac{c}{s}$$

Bad hash functions (example) have characteristic values of $Pr$ that not depends only on $|S|$...  We can use  $c$ to compare the efficiency of the bad hash  with other hashes. I am imagining that  I can  use $c$ or an averaged $\overline{c}$ based on many $c_i$ tests as a metric of collision rates.



NOTES

Let $c_1$ the number of collisions in $H_1(S)$ and  $c_2$ the number of collisions in $H_2(S)$, them we can compare the two hash functions.

To avoid bias we can do an average about many sample sets $S_1$, $S_2$, etc. So there are averaged $\overline{c}_1$ and  $\overline{c}_2$ as good estimators.

Theoretically we can also estimate the minimum $\overline{c}$ of a hash function with digests of $k$ bits by   $${Pr}_{min}[H](S) ~=~  1 - \frac{2^k!}{(2^k)^s (2^k - s)!} ~\approx~ 1 - exp({-s^2/2^{k + 1}})$$

Conclusion, a good score to compare hash functions is
$$q ~=~ Pr[H](S) - Pr_{min}[H](S) ~\approx~ \frac{\overline{c}}{s} -1+exp({-s^2/2^{k + 1}})$$

When $q=0$ we have "perfect hash function", that have an expected behaviour about collisions... Imagining that hashes like SHA1 are always perfect because is impossible to compute $\overline{c}$, but  truncated SHA1 or "bad hashes" can be tested about big $S$ samples. PS: negative $q$ is possible when  $H$ and $S$ are not independent. 

My interest is the behaviour of these truncated or bad hashes, and to compare them.
There are a few standard quantities related to families of hash functions $H_k\colon \{0,1\}^m \to \{0,1\}^h$ for a uniform random key $k$.  You might call them metrics.  They came to prominence in Carter and Wegman's research program on universal hash families (paywall-free), though their first use in cryptography for one-time authenticators, by Gilbert, MacWilliams, and Sloane (paywall-free), predated the Carter–Wegman program by several years.


An upper bound on $\Pr[H_k(x) = u]$ for any $x \in \{0,1\}^m$ and $u \in \{0,1\}^h$.  If it is $1/2^h$, then we say the family is equidistributed.
An upper bound on $\Pr[H_k(x) = H_k(y)]$ for any distinct $x, y \in \{0,1\}^m$.  For example, if it's at most $1/2^h$, we say the family is universal; if it's at most $\varepsilon = O(1/2^h)$, we say it's $\varepsilon$-almost universal.  The quantity $\Pr[H_k(x) = H_k(y)]$ is sometimes called the collision probability.
An upper bound on $\Pr[H_k(x) \oplus H_k(y) = u]$ for any distinct $x, y, \in \{0,1\}^m$ and any $u \in \{0,1\}^h$.  If it's at most $\varepsilon = O(1/2^h)$, we say it's $\varepsilon$-almost xor-universal.
An upper bound on $\Pr[H_k(x) = u, H_k(y) = v]$ for any distinct $x, y \in \{0,1\}^m$ and any $u, v \in \{0,1\}^h$.  If it's at most $\varepsilon = O(1/2^{2h})$, we say it's $\varepsilon$-almost strongly universal or $\varepsilon$-almost pairwise independent.
An upper bound on $\Pr[H_k(x_1) = u_1, \dots, H_k(x_n) = u_n]$ for any distinct $x_i \in \{0,1\}^m$ and any $u_i \in \{0,1\}^h$.  If it's at most $\varepsilon = O(1/2^{nh})$, we say it's $\varepsilon$-almost $n$-independent or $n$-universal.


There are standard provable examples of functions attaining optimal or near-optimal values of the above bounds.  In cryptography, these bounds are relevant to one-time authenticators such as Poly1305 and GHASH, which are based on equidistributed $\varepsilon$-almost xor-universal hash families for $\varepsilon \approx \lceil m/128\rceil/2^{128}$, as well as to other applications such as hash tables with non-adaptive adversaries.  The quantity $\Pr[H_k(m') = a' \mathrel| H_k(m) = a]$ for $m' \ne m$ is sometimes called the forgery probability.

An $n$-independent hash family has the property that the probability of a collision $H_k(x) = H_k(y)$ for distinct $x, y \in S$ in a subset $S \subseteq \{0,1\}^m$ of $n$ elements is $$1 - \frac{2^h!}{(2^h)^n (2^h - n)!} \approx 1 - e^{-n^2/2^{h + 1}}.$$  However, the key must be enormous to guarantee this.  Further, none of these bounds guarantees anything about the difficulty of deliberately computing a collision knowing the key.

What you are probably getting at is not collision probabilities for unknown keys, but rather the collision resistance of a hash family:


An upper bound on $\Pr[x \ne y, H_k(x) = H_k(y)]$ where $(x, y) = A(k)$ and $A$ is any random algorithm.  If this negligible when the expected cost of $A$ is below $\sqrt{2^h} = 2^{h/2}$, we say the family is collision-resistant.

In particular, if $B_t(f)$ is a random algorithm that applies the birthday paradox to $t$ uniform random inputs to $f\colon \{0,1\}^m \to \{0,1\}^h$ as an oracle and returns a collision if there is one (for example, using van Oorschot and Wiener's parallel collision search machine), then for any equidistributed $f$, $$\Pr[x \ne y, f(x) = f(y)] = 1 - \frac{2^h!}{(2^h)^t (2^h - t)!} \approx 1 - e^{-t^2/2^{h + 1}},$$ where $(x, y) = B_t(f)$.  When $t^2 \lll 2^h$, so that $t \lll \sqrt{2^h} = 2^{h/2}$, this is negligible in $h$.  The best generic attack on $H_k$ with cost $t$ is $A(k) = B_t(H_k)$; if there is no known attack better than this we say that $H_k$ is collision-resistant.


We don't know how to prove this for all random algorithms $A$, however, so this property can only be conjectured of hash families for which we haven't figured out a cheaper collision search algorithm.  For example, we are quite confident that Poly1305 or GHASH is not collision-resistant; it is trivial to compute two-block collisions under any key with one field multiplication and one field addition.  We conjecture that keyed BLAKE2 is collision-resistant because nobody has figured out how to make it collide.

It is of little use to empirically study collision resistance by sampling outputs of a generic hash function.  If you try to use the generic birthday algorithm on Poly1305 or GHASH, it won't do much better than it will on keyed BLAKE2.  But it doesn't take a genius of a cryptanalyst to immediately recognize there is a trivial algorithm to find Poly1305 and GHASH collisions, given the key, much faster than the generic birthday algorithm.  The best we can do is to empirically study it by sampling algorithms for finding collisions—asking a lot of smart cryptanalysts to try to think of them.  So far, empirically, nobody has thought of a way to find collisions in keyed BLAKE2.

Since sampling outputs is not useful for studying collision resistance, when studying simplified versions of hash families, we don't normally study a progression of truncated output sizes. Rather, we structure the hash family to iterate some internal scrambling function for a variable number of rounds, and study a progression of numbers of rounds.  For example, the best known collision attacks on reduced-round versions of BLAKE apply only to 5 rounds, which is why the BLAKE2 designers felt comfortable recommending only 10 rounds.

Loosely, we sometimes also pick a fixed hash function such as SHA3-256, and say that it is collision-resistant if nobody has found a way to compute a collision under that fixed hash function.  Of course, if $h \leq m$, we are guaranteed that a collision exists, so there must exist a trivial random algorithm of negligible cost to compute it: namely an algorithm that simply returns the collision without doing any computation.

So there's no nice way to formalize this, but in practice, it seems to be hard.  Twenty years ago, it was conjectured that SHA-1 was collision-resistant in this loose sense, until someone figured out a way to compute SHA-1 collisions at lower cost than the generic attack in 2004, and then someone published an actual collision last year.  Today, it is still conjectured that SHA-256 is still collision-resistant in this loose sense.