Why does appending permutations of servers at the end of hash table avoid bottlenecks?
Adding permutations isn't about preventing slow servers from becoming bottlenecks, rather it's about dispersing a convoy once one forms behind a slow server.  Because of the way tract locations are hashed, sequential reads of any blob always walk the tract locator table in the same order.  Suppose you have six tract servers and your tract locator table looks like this

123456

and server 3 is slower than the others.  Since sequential readers walk the table in the same order the slowness of server 3 will cause a convoy of sequential clients to form waiting for server 3.  Once server 3 serves them, they all move on server 4 which may also become overloaded.  Then 5, then 6, and before long the convoy is big enough to hammer each server in turn instead of spreading the load evenly across all of them.

If your tract locator table looks like this

643512623154125364213546

i.e. four permutations of 123456, events transpire differently if there is a slow server.  If server 3 is slow, satisfied sequential readers will move from server 3 to servers 5, 1 or 6 depending on where they are in the table.  This instead of all sequential I/O clients turning their attention to the same server after visiting server 3.  The extra permutations have the effect of breaking up convoys if they form and lessening the likelihood of convoys forming at all.