Normal equation result simplification
Only square matrices have an inverse, and neither $X$ nor $X^T$ here are (necessarily) square. I guess that's the most direct answer. $X$ may have a left-inverse $A$, such that $AX = I$. In that case, if $X\theta = y$ then $AX\theta = Ay$ so $\theta = Ay$. 

The direct way to construct that $A$ is just where you started though; it's $(X^TX)^{-1}X^T$. The Gramian matrix $X^TX$ is (square, and) invertible if the features (columns) of $X$ are linearly independent, and, $X^TX$ is more importantly relatively quite small and so it's feasible to invert.

You normally wouldn't compute the inverse anyway, but try to solve the system $X^TX\theta = X^Ty$ with QR decomposition or something, but that's not what you're asking.