OOP Decision Maker Class - ServerArbiter
Why does the decide() method do more work? It's the problem of the object constructing your Rules, which is not necessarily the ServerArbiter.
Isn't that already a part of the Server entity? Why do you need to inject yet another dependency. If the server's address isn't in the Server entity, why the hell the server's address isn't in the Server entity?!
Your architecture is fine. Easily separated, Servers don't know about the Rules.
Yes. Since querying all the servers to get their downloads count, it makes sense to remember the best server in the ServerDownloadsComparisonRule the first time resolve is called, and then just reference it from the cache.

if ($bestServer == null) {
    $bestServer = $this-&gt;findServerWithLeastDownloads();
}
return $server === $bestServer;