If the sender address is known is it possible to find the recipient address?
The person you gave your address to wouldn't know if the address you subsequently sent the funds to was actually your cold storage or a merchant, or friend, etc. However if you did business with this person again in the future, and he sees that you again sent it to the same address you sent it to before, then he could assume it might be your address.    

With that said, there are BIP32 wallets that allow you to only store one extended public/private key pair that will allow to restore the millions of private keys and address that you may ever generate. This way you wouldn't have to keep all funds in a single address, instead just a single extended BIP32 key chain.