question about RSA sign/verify scheme could you please explain what is the problem here?:

I'm trying to simulate sign/verify scenario with RSA keys. For simplicity I use 64bits keys:

PK = {BE1EE24B5ACD9DE5, 10001}
SK = {BE1EE24B5ACD9DE5, 15D231A2C5B605C1}


and do the calculations using http://extranet.cryptomathic.com/rsacalc/index

scenario 1 - Ok:

m = 0123456789ABCDEF


using Decrypt method (SIGN) I'm getting signature: 43437cd86b6a2cce

then using Encrypt method (VERIFY) on signature I'm getting back original message (m): 0123456789abcdef

scenario 2 - something goes wrong:

m = FEDCBA9876543210


SIGN = b3baca40fe26f5a1

VERIFY = 40bdd84d1b86942b

why VERIFY method doesn't produce original message here?

Thank you,
The RSA operation always produces an answer modulo the modulus.

Normally, we are careful to have a 'plaintext' which is smaller than the modulus, and hence the end result is the same.  After all, the fundamental identity for RSA is:

$$(M^d \bmod N)^ e \bmod N = M \bmod N$$

If $M &lt; N$, then the right hand side simplifies to $M$

However, in this case, you have an initial input FEDCBA9876543210 which is actually larger than the modulus BE1EE24B5ACD9DE5.  When you do this, you end up with an end result which is FEDCBA9876543210 mod BE1EE24B5ACD9DE5 = 40BDD84D1B86942B

That answers the immediate question you asked; however as you dig further into RSA, you should be aware of the following:


When we use RSA, we (almost) never apply the RSA operation directly to the value given by the application (hash result, plaintext); instead, we apply a 'padding' operation first, to avoid potential attacks that exploit the homomorphic properties (and determinism in the case of encryption) that the raw RSA operation has
During the signature operation, we really can't call the RSA operation 'decryption' (as we're not actually decrypting something that has been previously encrypted).  Normally, it is called a 'signing' or 'signature generation'.