How to keep track of current physics after graduating?
I am in this same situation. Here are some thoughts I have: 

In order to retain the skills we learned as undergraduates, we have to practice them much like we had to 'practice' problem sets for assignments. I think that the MIT open courseware is an excellent tool for reminding and practicing.

Physicists don't write papers about stuff other physicists know. Most (or all) of what's in a paper could be new and unfamiliar to the majority. A BS does not bring you up to speed with the physics community (my degree barely got me into the 20th century) so there is still a good deal of study before you could pick a paper and read it with ease. Continuing with your own studies by advancing your knowledge of physics will do a lot to help you understand what is in these papers. Reading the titles of recent Arxiv papers is a fun pass time; you can sit down and read any that perk your interest.