Are there transitive ciphers (either symmetric and asymmetric)
Actually, there's the Pohlig-Hellman cipher (dating back to the 70's) that can do precisely this; it works like this:


There's a large public prime $p$ (for which the DLOG problem is hard)
The secret key is an integer $x$ that is relatively prime to $p-1$
To encrypt a message $M$ with key $x$, we compute $C = M^x \bmod p$
To decrypt a message $C$ with key $x$, we compute $M = C^{x^{-1} \bmod p-1} \bmod p$


It should be clear that the encryption and decryption processes (for a fixed key) are inverses of each other, and that this allows Alice and Bob to do precisely what you asked for; if Alice has $M^b$, she can give Bob , $(M^b)^a = M^{ab}$; Bob computes $(M^{ab})^{b^{-1}} = M^a$, and send that back to Alice, who can then recover $M$