Can not having any research papers be a reason for rejection for an MS program?
There is no way to predict whether you will be accepted or rejected from any graduate program.  Graduate admission is a random process.  You have some influence on the expected outcome, but the variance is set differently by each department, and you only get one trial per department per year.

That said, as far as I know, there are no CS MS programs that restrict admission to applicants with prior publications.  There are PhD programs where most students in some research areas had published (or at least publishable) results as undergraduates—algorithms at Princeton, for example. But even in those programs, and in those research areas, publication is not an actual requirement for admission.