How else can I design a USB switch?
There are dedicated ICs designed for proper usb slave switching. While usb is complex, you can often get away without all the extra passives and esd circuitry, just make a pcb with tight control of the usb data pairs. The USB switching ICs will come with datasheets that show typical application schematics and you can get the board layouts from the demo boards. Without looking into a specific one, google usb switching ICs and demo boards. Read the datasheet for a good rundown of what needs to be done. You may also need a power switch. And some time with pcb design software and a pcb manufacturer. It's a bit of an endeavor.