Has anyone seen a Micro-USB socket which installs with the socket perpendicular to the PCB?
Micro USB (as also any other USB) is available in vertical board mounted form. Those words in italics should provide you some suitable search results. 

For instance, this one from Global Connector Technology, part number USB3105:



The GCT USB connector page for Micro USB has a few other vertical board mounted ones as well.