How can you calculate the inverse of S component of signature, while you cannot do it in ECC to calculate private key from public key? When verifying the signature in bitcoin, we calculate S^(-1) * hash(m) * G + S^(-1) * R * pub_key and verify that this is equal to the temporary public key. Given the same logic, why can't we use it to calculate private key from public key?

for example: pub_key = priv_key * G. Modify it to: priv_key^(-1) = pub_key^(-1)*G and then take get the priv_key by taking inverse of LHS.

I just want to understand the math that makes us calculate S^(-1) in the first case, but does not allow us to do it in the second case.
The semantics of multiplication are different in your two equations. 

As for the first equation. The s-part of a signature is an element in a group (represented by a number) so the inverse exists and is easy to compute (or even well known)

In the second case you take the numerical value of the private key and compute the multiple of this with a base point. With that notation the "inverse" becomes equivalent to finding the discrete logarithm in this additive group. In ecc it is not known how to compute the discrete log efficiently 

I guess this answer is somewhat frustrating. I recommend learning about Groupe theory to understand the differences in notation.