getreceivedbyaddress without knowing the private key
Bitcoin Core 0.10 was just released with support for watch-only wallets.

You can use the importaddress command to make a wallet consider the address to be belonging to it, and track incoming payments to it. getreceivedbyaddress will work as expected in that case. In fact, Bitcoin Core does not support an "address balance" feature at all.

It does indeed require you to import the address in advance (or go through a lengthy rescanning procedure). The upside is that this approach scales much better (it doesn't require a fully indexed blockchain, and is compatible with future pruning features).