Bruce Schneier, encryption, public keys and digital-signature. In his book "Applied Cryptography" Bruce Schneier, when talking about public key cryptography, says that:


  If the same algorithm is used for both encryption and digital-signature verification there is a possible attack. In these cases, the digital signature operation is the inverse of the encryption operation: Vx = Ex and Sx = DX.


I'm not sure how that is possible, could you further explain what Bruce is proposing here?
Say you ask the holder of the private key to decrypt something that you've encrypted for him. You've however lost the original data. So the holder of the private key decrypts the seemingly random data and returns the result.

Now imagine that instead of a ciphertext you send a message to be signed or a hash over the message to be signed. The holder of the private key will try to decrypt it. However, because $D(x) = S(x)$ what is returned is a valid signature over the message. This could be the case for, for instance, raw RSA.

However, for most encryption algorithms this won't be possible in real life. For instance, raw RSA is insecure, so PKCS#1 padding is used. Not only is PKCS#1 padding different for encryption and signature generation, but remember that we have to unpad for decryption and pad for signature generation.

So it depends on the algorithm used, but for often-used encryption primitives the problem is usually not applicable. Still, it is always preferable to use a different key pair for authentication, signing (or rather, non-repudiation) and encryption.

If you have to create your own protocol for, for instance, homomorphic encryption, where raw primitives are more common then sure, remember Bruce's advice.