Brute force attack on RSA - success criteria?
Well, typically for RSA, we have the public key.  That makes the verification process easy:


If you're trying to guess the private key (factoring or equivilent), that's easy; you just verify the factorization; "does the two primes we get, multiplied together, give us the modulus in the public key?"
If you're trying to guess the padded plaintext that corresponds to a ciphertext, that's easy; you encrypt the padded plaintext with the public key, and if it results in the ciphertext, you guessed it correctly.  Note that, for nondetermanistic padding methods (which is what ought to be used for RSA encryption), you have to guess the nondetermanistic padding bytes as well as the message bytes.
If you're trying to forge an RSA signature, that's easy; you just verify the forgery with the public key.