Derivation of an RC Low Pass Filter's Cutoff Frequency
Remember from mathematics that a complex number can be written as: \$z=a+j\,b\$.
Ok, once you know that, the magnitude of this number is easily calculated as \$\left|z\right| = \sqrt{a^2 + b^2}\$. Plug it into your transfer function and you'll find that \$\left|1+j\omega RC\right| = \sqrt{1^2 + (\omega RC)^2 }\$.
For the cutoff frequency of your RC low-pass filter, the cutoff frequency is the one when the output power has been reduced to half the passband power, which means a 3dB decrease in gain, thus : \$\implies \frac{\left|V_{out}\right|}{\left|V_{in}\right|} = \frac{1}{\sqrt{2}}\left|H(j0)\right|\$ where \$\left|H(j0)\right|\$ is the gain in the passband of your RC low-pass filter, which in this case is equal to 1.
Set this value equal to the transfer function value and you'll find the cutoff frequency to be \$\omega = \frac{1}{RC}\$