Transfer Function Magnitude, how to deal with 'j'?
Given what you already developed:

$$\begin{align*}
H\left(s\right)\quad &amp;=\quad \frac{s R C}{1 + s R C}\\\\
\mid\: H\left(s\right)\: \mid^{\:2}\quad &amp;=\quad H\left(s\right)\cdot H\left(s^*\right)\\\\
&amp;=\quad\frac{s R C}{1 + s R C}\cdot\frac{s^* R C}{1 + s^* R C}\\\\
&amp;=\quad\frac{\left(\sigma^2+\omega^2\right) R^2 C^2}{1 + 2\sigma R C+\sigma^2 R^2 C^2+\omega^2 R^2 C^2}
\end{align*}$$

Laplace uses \$s=\sigma + j \omega\$. But this collapses to Fourier along the imaginary axis portion, or the \$j \omega\$ part of that. Since you want the periodic and non-decaying part of the system response -- the frequency response -- you set \$\sigma=0\$ in \$s\$.

So the above devolves to:

$$\begin{align*}
\mid\:H\left(j\omega\right)\:\mid\quad &amp;=\sqrt{ \frac{\omega^2 R^2 C^2}{1 + \omega^2 R^2 C^2}}\\\\
&amp;=\quad\frac{\omega R C}{\sqrt{1+\omega^2 R^2 C^2}}
\end{align*}$$

I hope that helps.