Designing a USB switch, schematic done, wanting a second opinion
This is a pretty standard idea to switch a device between two USB hosts. The implementation however is unnecessary convoluted.

As I understand, the Pi and Flash drive are contained in one box. In this application both Pi (inside the "box") and external cable are USB hosts. The flash drive needs to be switched between them. 

The solution is simple, you need to start with upper part of your schematics (FSUSB30 switch and three USB connectors), but throw the bottom away, completely. Some fundamental modifications need to be made for ground and power.

First, all grounds MUST BE CONNECTED at all times, it is a very poor idea to have them switched. No "magic ground", ever.

Second, the VBUS to "USB_PEN_DATA" port can be permanently connected to +5V from box power, for simplicity.

The USB_INT_DATA can be wired permanently to Pi data port, and VBUS ("VCC") from Pi side (USB_INT_DATA) must be disconnected from VCC_USB_COMMON. There should no "common VCC_USB" because the VCC from USB_EXT_DATA port will conflict with internal +5V power rail.

The switch should be controlled by VCC that will come (or will not come) from the "USB_EXT_DATA" port in the following way:


when the "external" cable is not plugged, VCC on USB_EXT_DATA will be zero (some pull-down would be needed here). The switch should be controlled (by Pi) in direction from Flash to USB_INT_DATA.
When an external host is connected, it will apply VBUS (VCC) high. This will signal logically to Pi to stop what it was doing with flash, gracefully unmount it, and then to put the USUSB30 switch to connect the Flash with USB_EXT_DATA port. A good idea would be to put the Pi port into SUSPEND before making the switch. That's it.


The following events will occur on host side after the switch: since the USB traffic from Pi will cease, the Flash will go into SUSPEND, which will result in D+ = high. The host will see the "connect event", and since it was a fresh connect, it will issue USB_RESET, erasing all previous configuration to defaults. Then the host will enumerate the Flash in a normal way, and get access to its file system (if it is correct).

When the EXT cable is removed, the opposite process will happen: in 3 ms the Flash will go into suspend, D+ will go HIGH, the Pi side will sense a new connect event, and enumerate the Flash from its side. You might need to wait 5-10 ms before making the switch back after VBUS disappears. Job done.