Names of system functions in frequency domain
$\Omega$ and $\omega$ are frequencies (in radians). Usually, $\omega$ is used, but if you need to deal with continuous-time as well as discrete-time systems, it's common to use one for discrete time and one for continuous time in order to distinguish the two. There is no real standard as to which one is which. I've seen $\Omega$ used for continuous time as well as for discrete time. Having said that, the standard text book Discrete-Time Signal Processing by Oppenheim and Schafer uses $\Omega$ for continuous-time systems and $\omega$ for the normalized frequency of discrete-time systems.
$H(\Omega)$ and $H(\omega)$ are (continuous-time and discrete-time) frequency responses. Note that in discrete time it's also common to use $e^{j\omega}$ (instead of $\omega$) as the argument of the frequency response. This is discussed in the answers to this question. $H(s)$ is the transfer function in continuous time, and $H(z)$ is the transfer function in discrete time. $H(s)$ is the Laplace transform of the continuous-time impulse response $h(t)$, and $H(z)$ is the $\mathcal{Z}$-transform of the discrete-time impulse response $h[n]$. So with transfer functions it's always clear that the variable $s$ (or $p$ in some older texts) is used in continuous time and $z$ is used in discrete time.
The magnitudes $|H(\Omega)|$ and $|H(\omega)|$ are simply called what they are: magnitudes of frequency responses, or sometimes magnitude responses.
The magnitudes $|H(s)|$ and $|H(z)|$ don't have any special names as far as I know, and they are also rarely used with general complex arguments.