Stm32f103 blue pill and stlink power
3V3 on SWD is intended as a target supply sense. Probably it has some current-sourcing capability, but it's rather limited (hence you see the voltage drop to 2.6V). 

As Chris suggested, you shouldn't connect programmer to powered-off board. You could try to add diode there, but I'm not entirely sure, how programmer will deal with the diode voltage drop - it could assume, that voltage rail is at level insufficient for reliable operation and refuse to program.

As for putting a diode on power supply lines of each of boards, an LDO probably wouldn't hurt much and prevent current flow back into power rail.