Compare two methods of random permutations
Your method of comparison is very easy to defeat. I assume that by "Hamming distance from the regular configuration" you mean the number of fixed points. Make a table of the number of permutations in $S_n$ having $k$ fixed points (this can be computed recursively), and fix a single permutation with $k$ fixed points for every $k$. This way you can sample a permutation in a ways that matches this particular statistic of random permutations, but probably performs rather differently compared to a truly random permutation.