How to have a proper alphabetic pager with number support?
I recently started putting together a blog post showing how I did exactly this a few months ago using a custom module to create a new filter handler for views. I finished it up and posted here: http://garandwebtech.com/blog/2011/creating-alphabetical-filter-numeric-category-drupal-views-using-custom-filter-handler

Once you set up the new filter you just need to link # to pass the right parameter.