No Systems Course CS Master's
If you do really have everything else, then I doubt that it would have much effect for most MS programs. There are probably exceptions, but, generally, admission to a MS program doesn't imply the need for a BS in the same field.
But only by applying (or otherwise asking a university) can you be sure. You may need to take a systems course as part of your MS, though. With a math degree, I doubt you'd have much trouble with it.