1. A method of processing client requests received in a server system over a communication network, the method comprising the steps of: determining a distribution of a set of documents over a plurality of servers based at least in part on access rates of at least a subset of the documents; computing a set of redirection probabilities based on the distribution; routing a client request to a redirection server; and redirecting the client request from the redirection server to one of the plurality of document servers based on the set of redirection probabilities.