Is there legal recourse to drivers passing too close?
I can relate to your disdain for being passed to close. Preferably, everyone would give you at least 3 feet. But as you mentioned, as more and more people ride their bikes and drivers get used to your presence and others', it will get better.

The problem with your wish is, it's kind of like prosecuting someone for stealing five cents from you. Sure, it's against the law to steal five cents from you, but no judge is going to hear the case. It's small potatoes. Too small.

You could get one of those shirts that has the 3 foot rule on the back though. http://www.3footrule.com/