How difficult is it to get a key using simple XOR
That "algorithm" is fine, as long as the random values cannot be guessed by an attacker. The scheme is known as (trivial) secret sharing.

XOR with completely random data is called a one time pad or OTP. The security of the algorithm therefore depends on the random number generator. Of course there are additional operational requirements to obtain system security (if somebody just distributes his share of the key ...).