Sanded wall rough after brushing / rolling Kilz Original -- should I sand or is this normal?
Could it be that your roller is a TEXTURED roller and not a smooth roller. 

Many rollers that are available for purchase can have a texture by design and others by fault that they will leave on a smooth surface even 3/8 knap rollers, and some paints because of how they are made might be more susceptible to roller defects especially primers as the intent for a primer is coverage. 

Search for a roller that has a very small knap or if the area is small enough prime it with a spray paint or a smooth flat style brush - typically a flat style brush is used for knockout work. I have used it on a small wall before with nice results.