Do "standing"/90° MicroSD card slots exist?
If you cant find one, you could try making a very small pcb that holds a SMD SD card holder, with an edge connector on the PCB that can orient it vertically.