CI without Hartree-Fock SCF
If you were to perform Full CI (within a finite basis) then I don't think that it matters whether you start with SCF orbitals or some other choice.  This solution will have the lowest conceivable energy no matter what.

But if you are doing truncated (less than full) CI, then you are very much at the mercy of your starting orbitals. In general, one would want to start with orbitals that resemble those of the actual state that you are trying to describe via CI.

Otherwise, your precious excitations (singles and doubles) may be consumed correcting the "slop" that you provided, rather than moving toward the full CI limit.