Bitcoind JSON/RPC extremely slow under load
Bitcoind can be quite memory hungry and bandwidth hungry. If you're swapping a lot, definitely add more memory.

You may also consider using an external API like:

http://dev.blockcypher.com/#blockchain-api

Bitcoind itself is actually quite hard to scale out.