Why won't primer adhere to my walls?
Both answers so far are correct to a point. Older oil based enamels can be very difficult to cover. NOTE OF CAUTION, many older enamels contain LEAD and should not be sanded without first testing for lead and if positive,  proper precautions taken. Lead testing kits can be purchased at most paint departments. Read the directions carefully, but they are really quite easy to use and are very accurate if used properly. 

As mentioned, cleaning the walls is very important. Use TSP and scrub with something like Scotchbrite pads or green scrubbies. Use an oil based primer or sometimes pigmented shellac (Bulls Eye Primer) works well. If you still have an adhesion problem, there are liquid sandpaper prep liquids available that are good to use if you have lead based paint or high gloss finishes. They are kinda nasty to use, stink and are caustic, so follow the directions carefully and wear protective eye wear and gloves.  If the surface tests negative for lead, sanding may be the best way to prep the surface. Using a DA sander with 100 to 150 grit paper, then cleaning the surface should do the trick.  Be careful and Good luck.