Using a breadboard power supply along with USB to Serial chip
The probable problem is that the USB VCC and BPSS (Breadboard Power Supply Stick) 5V output are hard connected together and the regulator on the BPSS acts as a load to the USB 5V supply. See "Vcc back-feed" diagram below. 

Breaking the 5V feed between the two power supplies MAY be all that is required.
 This may cause "issues if the two nominal 5V supplies are substantially different in voltage but will probably work OK. 

Care needs to be taken in choosing which items work on which supply.
 An active switching arrangement that connected the two supplies only when both were powered may be useful but should not be necessary.