How can a key pair be derived from an arbitrary hash?
Bitcoin doesn't use RSA, it uses ECDSA. Every 256-bit value is a valid private key. (Though a very small fraction of them have to be folded.)

But even if the numbers had to be special, it still wouldn't matter. You could use every 256-bit value as a seed to a pseudo-random number generator which you could use to deterministically generate numbers that had whatever properties you need.

You can also trivially create schemes to generate numbers that do have needed properties from passphrases. For example, Ripple uses a scheme like the following:


Set a counter to zero.
Hash the passphrase together with the counter.
Does the hash have the needed properties? If so, stop. We're done.
Increment the counter.
Go to step 2.