How to mount component perpendiculary to PCB
I have seen this done by using two PCBs: a smaller one to hold the component is mounted 90° perpendicular to the other one. Pads on the edge of the small board (both sides) soldered to the main PCB are used as support:



These pads can consist of electrical connections, with additional redundant pads for strength. An alternative would be to use your choice of connector to interface between the two PCBs: