Can we relate approximate Hartree-Fock orbitals to true solutions in the basis-set limit?
In quantum chemistry numerical experimentation is the norm and rigorous mathematical proof the rare exception. As a result, not many rigorous results are know, not even for the good-old Hartree-Fock method. For a review of mathematical results I recommend you have a look at these references:

Claude Le Bris, Computational chemistry from the perspective of numerical analysis, Acta Numerica (2005), pp. 363–444
Pablo Echenique and J. L. Alonso, A mathematical and computational review of Hartree–Fock SCF methods in quantum chemistry, Molecular Physics, Vol. 105, Nos. 23–24, 10 December–20 December 2007, 3057–3098

In particular, section 3.3 of the first reference is relevant, but don't expect to find mathematically strong results. For example, here's a quote:

The choice of an AO basis for solving
a given problem mostly relies upon some practical know-how. The lack of
rigorous understanding is a pity, because the output of the calculations
(typically some molecular property) might be very sensitive to the choice
of the basis set. The only available measures of the quality of the basis
set are obtained, in the chemistry literature, by choosing test cases, i.e.,
reference systems, where the solution of the exact Schrödinger equation may
be computed, mostly through numerical computations and, when possible,
with the help of an analytic calculation.

From a practical point of view I don't see why occupied Hartree-Fock orbitals expanded in the usual Gaussian basis sets should not converge (in the integral norm sense, ie leaving aside the lack of cusp at the nucleus) to the exact solutions of the Hartree-Fock equations. I have little idea as to what happens to the virtual orbitals.