What is the relation between the PSDs of filter input and output called? $R_Y = |H|^2R_X$
I don't know the name of the relationship, but $\vert H(f)\vert^2$ is
called the power transfer function of the LTI system.  The output
power spectrum is the input power spectrum multiplied by the
power transfer function, just as for deterministic signals, the output
spectrum is the input spectrum multiplied by the transfer function
$H(f)$.