Where should I store the private key (RSA) in a web application?
If you are in AWS, well and paranoid enough, you might want to use KMS. Google has a similar service. As for non-cloud environments, you can use either an HSM or a KMIP key server. Both HSMs and KMIP servers can be built in an High Availability way, and Amazon offer a good option to use SafeNet HSMs in the cloud, the service is called CloudHSM.

I am not a .Net expert, any more, but my understanding is that it has some adapters to plug in an PKCS#11 engine (HSM) transparently, so that you don't have to write any code for this to work. But it's probably a lot of configuration ugly work to get it to work.