Can CTR mode be used with public key cryptography? Can counter mode be used in some way in public key cryptography? Can we decrypt when counter mode is used if 1. we have the key and 2. we have the cipher text? Is there any other mode in which this is possible?
I interpret your question to be asking "Can we directly use public key encryption algorithms with a CTR-like mode?", as opposed to "is it possible to perform a key exchange via RSA and use AES-CTR?" (which is obviously possible).

Let's define a "block cipher" as a deterministic algorithm that produces a permutation (the 
"cipher" part) on a fixed-length block of bits (the "block" part). Non-randomized RSA meets this definition, though there are technically variants with different block sizes, determined by the modulus size. 

The question then becomes "Can we apply textbook RSA to a public stream of incrementing numbers?", which is obviously a yes. However, doing so will not be secure. Non-randomized RSA is understood to be insecure. More importantly here, you will also be sending the nonce, and the adversary is assumed to possess your public key. Thus, nothing will prevent an adversary from applying $E(seed, public\ key)$ and generating the key stream himself. 

"Randomizing" the above construction by using a random nonce will not work - if you are sending the nonce alongside the ciphertext, and the adversary has the key, the adversary can trivially recover the message. 

So a secure CTR-mode using deterministic public key encryption such as RSA appears to be doomed.

Can we decrypt when counter mode is used if 1. we have the key and 2. we have the cipher text?

No, CTR mode generates a key stream by encrypting a counter, and adds the key stream to the message. Thus, if you do not have the seed for the key stream, you will not be able to regenerate the key stream and recover the message (assuming you do not resort to cryptanalysis to recover the message, anyways). While not sending the seed will disallow an adversary from recovering the message, it will also disallow the intended recipient from recovering the message, as both of them possess identical keys.