How do restricted open shell calculations mathematically proceed in the context of the self consistent field iterations?
The paper by Tsuchimochi and Scuseria describes it quite well.  

Briefly, Roothan-Hall equation is the same but the ROHF matrix is constructed from the $\alpha$ and $\beta$ Fock matrices from UHF.

$$\mathbf{F}_\mathrm{ROHF} =
\begin{bmatrix}
    \mathbf{R}_{cc}       &amp; \mathbf{F}_{co}^\beta &amp; \mathbf{F}_{cv}^{cs} \\
    \mathbf{F}_{oc}^\beta &amp; \mathbf{R}_{oo} &amp; \mathbf{F}_{ov}^\alpha \\
    \mathbf{F}_{vc}^{cs}  &amp; \mathbf{F}_{vo}^\alpha &amp; \mathbf{R}_{cc} \\
\end{bmatrix}$$

where $c$ is doubly occupied, $o$ is singly occupied, $v$ is virtual and 

$$\begin{align}
\mathbf{R}_{cc} &amp;= A_{cc}\mathbf{F}_{cc}^\alpha + B_{cc}\mathbf{F}_{cc}^\beta\\
\mathbf{R}_{oo} &amp;= A_{oo}\mathbf{F}_{oo}^\alpha + B_{oo}\mathbf{F}_{oo}^\beta\\
\mathbf{R}_{vv} &amp;= A_{vv}\mathbf{F}_{vv}^\alpha + B_{vv}\mathbf{F}_{vv}^\beta
\end{align}$$


  Different values for these parameters have been suggested in the literature. Although they do not affect the ROHF wave function and energy, they affect orbital energies whose physical meaning is obscured because of this dependence.





Takashi Tsuchimochi and Gustavo E. Scuseria, J. Chem. Phys. 2010, 133, 141102. DOI: 10.1063/1.3503173; available as ePrint: arXiv:1008.1607 [cond-mat.str-el].