Switching USB data lines with SSR doable?
Switching D+/D- USB lines with SSRs will mess up with the signals, especially if you're using high-speed USB (480MB/s). The bandwidth is not sufficient and the impedance will not be maintained, that won't work.

There are specialty chips that do just what you want, for example TS3USB30. Unfortunately, they are all very hard to solder by hand (QFN packages for most of them) because they are designed for mobile devices and avoid gull-wing leads to better maintain the impedance.

Also, if you build such a device, you need to really be careful about the layout of the traces (impedance matching, again), and you'll probably need a four-layer board. This is not straightforward.

The conclusion is: this is impractical (for a hobbyist, at least).