Going for PhD or MS in research right after under graduation?
In certain disciplines it's very common to enter a Ph.D/MS+thesis program straight out of an undergraduate degree. Computer science is one such discipline. Usually you spend the first year or two in the Ph.D program getting up to speed and doing research. For an MS+Thesis program, things are more compressed, but you also aren't expected to produce research of the same scope. 

While it's always better to have some research experience as an undergraduate (and this helps your grad application) it's not a requirement for success at the graduate level.